This article discusses how to fix a failed Hive query. Hive query fails with the following error:
"Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient".
The problem results in Out Of Memory (OOM).
hive> show tables; FAILED: Error in metadata: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
The Out Of Memory (OOM) error, in this case, is a result of the server not having enough memory to start HiveMetaStore service.
HiveMetaStore service is configured in hive-site.xml on the client side, but the service is not started.
/etc/gphd/hive-0.11.0_gphd_2_1_0_0/conf/hive-site.xml <property> <name>hive.metastore.uris <value>thrift://hdw1.viadea.com:9083 </property>
However on hdw1:
-bash-4.1$ service hive-metastore status hive-metastore dead but pid file exists
After observing the hive-metastore.log, you can see that reason why the service failed to start was due to an Out of Memory error, as shown in the example below.
/var/log/gphd/hive/hive-metastore.log: 14/04/07 16:43:13 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore. WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. library initialization failed - unable to allocate file descriptor table - out of memory
- Firstly resolve the root cause why HiveMetaStore service is not started. In this case, increase the physical memory of the server.
- Then start the HiveMetaStore service manually using root user:
service hive-metastore start