I’ve got a table defined in Snowflake as:
GLPCT
BATCH_KEY NUMBER(38,0) NULL CTACCT VARCHAR(100) NULL CTPAGE NUMBER(38,0) NULL
and a file that looks like this:
GLPCT.csv
CTACCT VARCHAR(100) CTPAGE NUMBER(38,0)
example:
CTACCT,CTPAGE "Test Account",100 "Second Account", 200
My copy into command looks like this:
copy into GLPCT_POC from 'azure://ouraccount.blob.core.windows.net/landing/GLPCT' credentials=(azure_sas_token='<SAS_TOKEN') file_format=(TYPE=CSV, SKIP_HEADER = 1, FIELD_OPTIONALLY_ENCLOSED_BY='"');
Problem
Snowflake is throwing an error due to a column number mismatch. How can I get Snowflake to ignore the column that isn’t present in the file and not throw an error? I can move BATCH_KEY
to the end of the table if that will help.
Advertisement
Answer
It appears it’s possible to indicate what columns to insert into with a COPY INTO statement, so ours becomes:
copy into GLPCT_POC (CTACCT, CTPAGE) from 'azure://ouraccount.blob.core.windows.net/landing/GLPCT' credentials=(azure_sas_token='<SAS_TOKEN') file_format=(TYPE=CSV, SKIP_HEADER = 1, FIELD_OPTIONALLY_ENCLOSED_BY='"');
We could not use a transformation as mentioned in a previous answer, due to this being an external file.