QueryDB

  While doing SSH, we received below error -  Unable to negotiate with 22.33.18.90 port 22: no matching key exchange method found. Their offer: diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1 Solution Update ssh command as below -  ssh -oKexAlgorithms=+diffie-hellman-group1-sha1 username@22.33.18.90 Or, permanently adding below to ~/.ssh/config Host 22.33.18.90 KexAlgorithms +diffie-hellman-group1-sha1

  Talend TSSH Component failing with error as below -  java.io.IOException: There was a problem while connecting to localhost:22        at ch.ethz.ssh2.Connection.connect(Connection.java:805)        at ch.ethz.ssh2.Connection.connect(Connection.java:595)    ......        ...... Caused by: java.io.IOException: Key exchange was not finished, connection is closed.        at ch.ethz.ssh2.transport.KexManager.getOrWaitForConnectionInfo(KexManager.java:78)        at ch.ethz.ssh2.transport.TransportManager.getConnectionInfo(TransportManager.java:281)        at ch.ethz.ssh2.Connection.connect(Connection.java:761)        ... 5 more Caused by: java.io.IOException: Cannot negotiate, proposals do not match.        at ch.ethz.ssh2.transport.ClientKexManager.handleMessage(ClientKexManager.java:123)       ...

While trying  SFTP/ SSH: One may observe below error : @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:7nYKseap6dAOZ2jb+ExnHUluNtqbz46AUXw14NCO1hk. Please contact your system administrator. Add correct host key in /home/myuser/.ssh/known_hosts to get rid of this message. Offending RSA key in /home/myuser/.ssh/known_hosts:42 RSA host key for mysftp.sftp.com has changed and you have requested strict checking. Host key verification failed. Couldn't read packet: Connection reset by peer Solution - Do one of the following- use ssh-keygen to delete the invalid key     ...

  We wrote the Spark code to read data from using HBase-Connector as below -  val sql = spark.sqlContext val df = sql.read.format("org.apache.hadoop.hbase.spark")  .option("hbase.columns.mapping",    "name STRING :key, email STRING c:email, " +      "birthDate DATE p:birthDate, height FLOAT p:height")  .option("hbase.table", "person")  .option("hbase.spark.use.hbasecontext", false)  .load() df.createOrReplaceTempView("personView") val results = sql.sql("SELECT * FROM personView") results.show() Above code works fine. But, if we add a where clause to SQL above, it gives error as below -  val results = sql.sql("SELECT * FROM personView where name='Jaiganesh'") results.show() Error -  Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.ClassNotFoundException: org.apa...

  Several integrations for accessing HBase from Spark have occurred in the past. The first experimental connector was developed by Cloudera Professional Services, which was called Spark on HBase. Cloudera included a derivative of this community version ( called hbase-spark)   in both CDH 5 and CDH 6 Hortonworks also came up with an implementation, it was called SHC (Spark HBase connector). SHC was supported by HDP & CDP. But with CDP 7: Spark HBase Connector (SHC) is no longer supported in CDP. Refer https://docs.cloudera.com/runtime/7.2.0/hbase-overview/topics/hbase-on-cdp.html Refer below for compatibility- Implementation Spark Distribution hbase-spark 1.6 CDH 5 hbase-spark 2.4 CDH 6 hbase-spark 2.3 HDP 2.6, HDP 3.1 SHC 2.3 HDP 2.6, HDP 3.1 hbase-connectors 2.4 CDP Reference - https://community.cloudera.com/t5/Community-Articles/HBase-Spark-in-CDP/ta-p/294868