After updating from 3.6.3 to 4.1.1 a weekly script I run broke. Specifically it triggers an error:
In file.remove("temp.csv") :
cannot remove file 'temp.csv', reason 'Permission denied'
To manually delete temp.csv I get a file in use by rsession.exe error from windows. If I restart R in Rstudio, then both file.remove() and manually deleting work.
It might be that I’m using a bad stratgy in the first place, but the code causing the error is this bit:
write_csv(VPTS, "temp.csv")
VPTS <- read_csv("temp.csv", col_types = cols())
file.remove("temp.csv")
For context VPTS comes in from an SQL import using sqlFetch() and the formats are all wonky, dumping it into a CSV and using read_csv() imports it perfectly for future use.
I’m confident this isn’t a permissions issue, but something in the updated R not releasing the file it created. I’m just looking to get pointed in a direction to continue trouble shooting at this point.
Advertisement
Answer
readr version >=2.0 and < 2.1 uses vroom for fast data import and defaulted to lazy loading the data which kept a lock on the file preventing you from deleting it. You needed to explicitly disable to lazy reading with
VPTS <- read_csv("temp.csv", col_types = cols(), lazy=FALSE)
This info was mentioned in the changelog under the 2.0.0 version.
Deleting files after reading is also impacted by laziness. On Windows open files cannot be deleted as long as a process has the file open. Because readr keeps a file open when reading lazily this means you cannot read, then immediately delete the file. readr will in most cases close the file once it has been completely read. However, if you know you want to be able to delete the file after reading it is best to pass lazy = FALSE when reading the file
Update Note as of version readr 2.1, lazy reading is no longer the default (precisely because of errors like this). This will prevent the permission denied error from occurring. But if you want to read data more quickly, you will now need to set lazy=TRUE explicitly.