Skip to content
Advertisement

spark [dataframe].write.option(“mode”,“overwrite”).saveAsTable(“foo”) fails with ‘already exists’ if foo exists

I think I am seeing a bug in spark where mode ‘overwrite’ is not respected, rather an exception is thrown on an attempt to do saveAsTable into a table that already exists (using mode ‘overwrite’).

Below is a little scriptlet that reproduces the issue. The last statement results in a stack trace reading:

Any advice much appreciated.

Advertisement

Answer

saveAsTable doesn’t check extra options, use mode directly

or

User contributions licensed under: CC BY-SA
5 People found this is helpful
Advertisement