我似乎在流中遗漏了一些东西—Spark 2.2中的静态连接。
手册上说这样的连接是可能的,但是我不能得到正确的语法。奇怪。没有使用水印。
val joinedDs = salesDs
.join(customerDs, "customerId", joinType="leftOuter")
得到的错误如下,但我很肯定我有正确的一面:
<console>:81: error: overloaded method value join with alternatives:
(right: org.apache.spark.sql.Dataset[_],joinExprs:
org.apache.spark.sql.Column,joinType: String)org.apache.spark.sql.DataFrame <and>
(right: org.apache.spark.sql.Dataset[_],usingColumns: Seq[String],joinType: String)org.apache.spark.sql.DataFrame
cannot be applied to (org.apache.spark.sql.Dataset[Customer], String, joinType: String)
.join(customerDs, "customerId", joinType="left_Outer")
^