QueryDB

I was running oozie jobs and came across strange behavior where sometimes a Pig latin job gets successfully executed and other times it fails with below exception: 2015-11-02 14:31:20,241 FATAL [main] org.apache.hadoop.mapred.YarnChild: Error running child : java.lang.NoSuchFieldError: HIVE_LAZYSIMPLE_EXTENDED_BOOLEAN_LITERAL at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initSerdeParams(LazySimpleSerDe.java:318) at org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initialize(LazySimpleSerDe.java:195) at org.apache.hcatalog.mapreduce.InternalUtil.initializeDeserializer(InternalUtil.java:155) at org.apache.hcatalog.mapreduce.HCatRecordReader.createDeserializer(HCatRecordReader.java:127) at org.apache.hcatalog.mapreduce.HCatRecordReader.initialize(HCatRecordReader.java:92) at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.initialize(PigRecordReader.java:181) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.jav

You may face this error with Hive during query execution or Hive  installation. This is basically related to MySQL. Caused by: java.sql.SQLException: Access denied for user 'hive'@'hivehost' (using password: YES) This error is caused because of insufficient privliges to your user follow below step to solve this :- mysql> CREATE USER 'hive'@'hivehost' IDENTIFIED BY 'mypassword'; ... mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'hive'@'hivehost'; mysql> GRANT SELECT,INSERT,UPDATE,DELETE,LOCK TABLES,EXECUTE ON metastore.* TO 'hive'@'hivehost'; mysql> FLUSH PRIVILEGES; mysql> quit; Note that create user for each host from where you are going to access the database. Refer given url for Hive installtion: http://querydb.blogspot.in/2015/10/hive-installation-step-by-step-with.html

While using Hive version less than < 0.14. You may get below exception: Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:326) at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:471) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:162) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1477) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:157) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.GroupByOperator.closeOp(GroupByOperator.java:1141) at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:577)

1) Download tar file "hbase-0.96.2-hadoop2-bin.tar.gz" 2) Unpack this tar on each machine of your HBase installtion. 3) Edit .bashrc export HBASE_HOME=/opt/ds/app/hbase-0.96.2-hadoop2 export PATH=$PATH:$HBASE_HOME/bin 4) Execute >source .bashrc 5) Verify hbase version > hbase version 2015-10-29 13:33:48,002 INFO  [main] util.VersionInfo: HBase 0.96.2-hadoop2 6) Edit "hbase-env.sh" Update JAVA_HOME export HBASE_MANAGES_ZK=true export HBASE_PID_DIR=/var/hbase/pids 7) Update "hbase-site.xml" on Master  <configuration>   <property>     <name>hbase.rootdir</name>     <value>hdfs://abcdHost:54310/hbase</value>   </property>    <property>     <name>hbase.cluster.distributed</name>     <value>true</value>   </property>   <property>     <name>hbase.zookeeper.property.dataDir</name>     <value>hdfs://

1) Download "oozie-4.1.0.tar.gz" 2) Gunzip and Untar @ /opt/ds/app/oozie 3) Change directory to  /opt/ds/app/oozie/oozie-4.1.0 4) Execute      bin/mkdistro.sh -DskipTests -Dhadoopversion=2.2.0 5) Change directory to /opt/ds/app/oozie/oozie-4.1.0/distro/target/oozie-4.1.0-distro/oozie-4.1.0 6) Edit '.bashrc' and add export OOZIE_VERSION=4.1.0 export OOZIE_HOME=/opt/ds/app/oozie/oozie-4.1.0/distro/target/oozie-4.1.0-distro/oozie-4.1.0 export PATH=$PATH:$OOZIE_HOME/bin 7) Change directory to /opt/ds/app/oozie/oozie-4.1.0/distro/target/oozie-4.1.0-distro/oozie-4.1.0 8) Make directory 'libext' 9) Execute: >cp /opt/ds/app/oozie/oozie-4.1.0/hcataloglibs/target/oozie-4.1.0-hcataloglibs.tar.gz . >tar xzvf oozie-4.1.0-hcataloglibs.tar.gz >cp oozie-4.1.0/hadooplibs/hadooplib-2.3.0.oozie-4.1.0/* libext/ >cd libext/ 10) Download 'ext-2.2.zip'and place it in 'libext/' directory 11) Add below properti