如何在scala中为like操作符添加开关盒?

regexp world中的SQL符号“%”可以替换为“.*”。可以创建自定义项以使值与模式匹配:

val originalSqlLikePatternMap = Map("Item % not found%" -> "NOT_FOUND",
  // 20 other patterns here
  "%purchase count % is too low %" -> "TOO_LOW_PURCHASE_COUNT")
val javaPatternMap = originalSqlLikePatternMap.map(v => v._1.replaceAll("%", ".*") -> v._2)

val df = Seq(
  "Item foo not found ", "Foo purchase count 1 is too low ", "#!@"
).toDF("raw_type")

val converter = (value: String) => javaPatternMap.find(v => value.matches(v._1)).map(_._2).getOrElse("Unknown")
val converterUDF = udf(converter)

val result = df.withColumn("updatedType", converterUDF($"raw_type"))
result.show(false)

输出:

+--------------------------------+----------------------+
|raw_type                        |updatedType           |
+--------------------------------+----------------------+
|Item foo not found              |NOT_FOUND             |
|Foo purchase count 1 is too low |TOO_LOW_PURCHASE_COUNT|
|#!@                             |Unknown               |
+--------------------------------+----------------------+

使用 when 和 like

import org.apache.spark.sql.functions.when

val df = Seq(
  "Item foo not found",  "Foo purchase count 1 is too low ", "#!@"
).toDF("raw_type")

val newdf = df.withColumn(
  "updatedType",
  when($"raw_type" like "Item % not found%", "NOT_FOUND")
    .when($"raw_type" like "%purchase count % is too low%", "TOO_LOW_PURCHASE_COUNT")
    .otherwise("Unknown")
)

结果:

newdf.show
// +--------------------+--------------------+
// |            raw_type|         updatedType|
// +--------------------+--------------------+
// |  Item foo not found|           NOT_FOUND|
// |Foo purchase coun...|TOO_LOW_PURCHASE_...|
// |                 #!@|             Unknown|
// +--------------------+--------------------+

参考文献:

• Spark Equivalent of IF Then ELSE
• Filter spark DataFrame on string contains