Spark Hive ORC Exception Caused by: java.util.concurrent.ExecutionException: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.
Exception
Caused by: java.util.concurrent.ExecutionException: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1865)
... 17 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.
at com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1865)
... 17 more
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message tag had invalid wire type.
at com.google.protobuf.InvalidProtocolBufferException.invalidWireType(InvalidProtocolBufferException.java:99)
Reason
You might receive above error while performing SQL operations using Spark or Hive. Above error results because there may be some corrupt ORC Files or Unsupported version of ORC on HDFS.
Solution
- Identify and remove corrupt or incorrect files from HDFS.
- Or, With Spark:- You can ignore such files by setting following property -
- set spark.sql.hive.convertMetastoreOrc=true
Comments
Post a Comment