QueryDB

  While doing SSH, we received below error -  Unable to negotiate with 22.33.18.90 port 22: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 Solution Update ssh command as below -  ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 username@22.33.18.90 Or, permanently adding below to ~/.ssh/config Host 22.33.18.90 KexAlgorithms +diffie-hellman-group1-sha1

  Talend TSSH Component failing with error as below -  java.io.IOException: There was a problem while connecting to localhost:22        at ch.ethz.ssh2.Connection.connect(Connection.java:805)        at ch.ethz.ssh2.Connection.connect(Connection.java:595)    ......        ...... Caused by: java.io.IOException: Key exchange was not finished, connection is closed.        at ch.ethz.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:78)        at ch.ethz.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:281)        at ch.ethz.ssh2.Connection.connect(Connection.java:761)        ... 5 more Caused by: java.io.IOException: Cannot negotiate, proposals do not match.        at ch.ethz.ssh2.transport.ClientKexManager.handleMessage(ClientKexManager.java:123)        at ch.ethz.ssh2.transport.TransportManager.receiveLoop(TransportManager.java:941)        at ch.ethz.ssh2.transport.TransportManager$1.run(TransportManager.java:510)        at java.lang.Thread.run

While trying  SFTP/ SSH: One may observe below error : @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:7nYKseap6dAOZ2jb+ExnHUluNtqbz46AUXw14NCO1hk. Please contact your system administrator. Add correct host key in /home/myuser/.ssh/known_hosts to get rid of this message. Offending RSA key in /home/myuser/.ssh/known_hosts:42 RSA host key for mysftp.sftp.com has changed and you have requested strict checking. Host key verification failed. Couldn't read packet: Connection reset by peer Solution - Do one of the following- use ssh-keygen to delete the invalid key             ssh-keygen -R "you server h

  We wrote the Spark code to read data from using HBase-Connector as below -  val sql = spark.sqlContext val df = sql.read.format("org.apache.hadoop.hbase.spark")  .option("hbase.columns.mapping",    "name STRING :key, email STRING c:email, " +      "birthDate DATE p:birthDate, height FLOAT p:height")  .option("hbase.table", "person")  .option("hbase.spark.use.hbasecontext", false)  .load() df.createOrReplaceTempView("personView") val results = sql.sql("SELECT * FROM personView") results.show() Above code works fine. But, if we add a where clause to SQL above, it gives error as below -  val results = sql.sql("SELECT * FROM personView where name='Jaiganesh'") results.show() Error -  Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.ClassNotFoundException: org.apa

  Several integrations for accessing HBase from Spark have occurred in the past. The first experimental connector was developed by Cloudera Professional Services, which was called Spark on HBase. Cloudera included a derivative of this community version ( called hbase-spark)   in both CDH 5 and CDH 6 Hortonworks also came up with an implementation, it was called SHC (Spark HBase connector). SHC was supported by HDP & CDP. But with CDP 7: Spark HBase Connector (SHC) is no longer supported in CDP. Refer https://docs.cloudera.com/runtime/7.2.0/hbase-overview/topics/hbase-on-cdp.html Refer below for compatibility- Implementation Spark Distribution hbase-spark 1.6 CDH 5 hbase-spark 2.4 CDH 6 hbase-spark 2.3 HDP 2.6, HDP 3.1 SHC 2.3 HDP 2.6, HDP 3.1 hbase-connectors 2.4 CDP Reference - https://community.cloudera.com/t5/Community-Articles/HBase-Spark-in-CDP/ta-p/294868