I am unable to pass a date string in spark sql
When I run this
spark.sql(""" SELECT cast ('2021-04-12' as date) """)
> DataFrame[CAST(2021-04-12 AS DATE): date]
However I get error when I want to pass the date string as variable
 date_str = '2021-04-12'
 spark.sql(""" SELECT cast ({} as date) """.format(date_str))
 > AnalysisException: cannot resolve 'CAST(((2021 - 4) - 12) AS DATE)' 
 > due to data type mismatch: cannot cast int to date; line 1 pos 7;
I am not sure, how to pass that variable.
Advertisement
Answer
you can just add single quotes to the query and it should work for you
spark.sql(""" SELECT cast ('{}' as date) """.format(date_str))