I am trying to convert below SQL code to pyspark. Can someone please help me
UPDATE B SET col = 'a' FROM table B WHERE (case when left(util,1) in ('A', 'B') then count else PROCS end < 1)
Here, util, count, procs are column names.
while coding in pyspark, i can create a new column ‘col’ like this:
df1 = df1.withColumn("col", case condition ,a)
Advertisement
Answer
You can use when
for doing the equivalent of update
:
df1 = df1.withColumn( 'col', F.when( F.expr("case when left(util,1) in ('A', 'B') then count else PROCS end < 1"), F.lit('a') ).otherwise(F.col('col')) )