Normally, one can write syntax like below to create a JDBC connection with Hive -
input { jdbc { jdbc_driver_library => "hive-jdbc-2.0.0.jar,hive2.jar,hive-common-2.3.1.jar,hadoop-core-1.2.1-0.jar" jdbc_driver_class => "org.apache.hive.jdbc.HiveDriver" jdbc_connection_string => "" } } output { # Publish out in command line stdout { codec => json } }
But, you will get problem if you need to do Kerberos authentication for using Hive JDBC. Relating to this, set following JVM Options. Note that these can be set with either within config/jvm.options file or setting the LS_JAVA_OPTS variable will additive override JVM settings. Refer - https://www.elastic.co/guide/en/logstash/current/jvm-settings.html
- -Djava.security.auth.login.config=<Jass_config_file_path> (Required)
- -Djava.security.krb5.conf=<Path to krb5.conf> (if it is not in default location under /etc/)
if KRB5.conf is not specified then you can manually specify KDC and Realm using below options -
- -Djava.security.krb5.kdc=<KDC_Server>
- -Djava.security.krb5.realm=<Kerberos REALM>
Sample Jaas.conf file can look something like below -
Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
doNotPrompt=true
useTicketCache=false
serviceName="hive"
keyTab="/path/to/awsemraccount.keytab"
principal="awsemraccount@EMR.LOCAL"
storeKey=true
client=true;
};
Sample Hive JDBC URL would look like -
jdbc:hive2://myhiveserver:10000/default;principal=hive/_HOST@REALM;;AuthMech=1;KrbRealm=REALM;KrbHostFQDN=hostfqdn;KrbServiceName=hive
Comments
Post a Comment