def robscal(sdf, col_name):
lst_quantile = sdf.select(col_name).toPandas().quantile([0.25, 0.5, 0.75]).values.tolist()
q1, q2, q3 = [y for x in lst_quantile for y in x]
return sdf.withColumn(col_name, (sf.col(col_name)-q2)/(q3-q1))
我想用熊猫udf来避免这种转换,但我不知道怎么做。你能帮帮我吗?
