I have a partitioned table with daily snapshots from from glue. When I use athena to query it queries across all partitions. Is there a way to get Athena to automatically only get the latest snapshot? Or do I have to explicitly state what partition I want to query if I want to avoid querying across all snapshots?
Advertisement
Answer
If you have created tables as Hudi tables then you can use Athena to run Snapshot queries. For more please refer to this link.
If the tables are directly created in Glue catalog then you have to explicitly mention the partition that you want to query.If you have a table with large number of partitions then try partition projection which is well explained in this link.