The code in cell 1 works just fine.
The problem
I just don’t know the correct keywords to search to see how I can make the code in cells 2 & 3 work. Basically, in a %sql cell, can I select into a variable that can be later used in a python cell? I realize this may not even be possible. Cheers!
cell 1
ds = spark.sql("select * from duamonds")
display(ds)
cell 2
%sql select * from diamonds
cell 3
display({ sql result })
Advertisement
Answer
It cannot be done. However, you can do this:
Cell1
sqlresult= sqlContext.sql("select * from diamonds")
Cell2
display(sqlresult)
You can find more details on using SQL with dataframes HERE.