I 'm using Hadoop 2.2 as a sink in Flume 1.4
If you try to use HDFS then you might get below exception:-
[ERROR] HDFSEventSink. process failed
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$RecoverLeaseRequestProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
Above Exception is due to incompatiblity of protobuf libraries. So, a possible solution that, I followed is to rename 2 Jar files in /opt/ds/app/flume-1.4.0/lib
>protobuf-java-2.4.1.jar-1
>guava-10.0.1.jar-1
Renaming this 2 Jars will cause Flume not to load it and can be kept as back up in lib itself. Then try to run again this problem should disappear.
If you try to use HDFS then you might get below exception:-
[ERROR] HDFSEventSink. process failed
Exception in thread "SinkRunner-PollingRunner-DefaultSinkProcessor" java.lang.VerifyError: class org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$RecoverLeaseRequestProto overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
Above Exception is due to incompatiblity of protobuf libraries. So, a possible solution that, I followed is to rename 2 Jar files in /opt/ds/app/flume-1.4.0/lib
>protobuf-java-2.4.1.jar-1
>guava-10.0.1.jar-1
Renaming this 2 Jars will cause Flume not to load it and can be kept as back up in lib itself. Then try to run again this problem should disappear.
Comments
Post a Comment