What happens when we add new records to a SQL table while the Azure Search Indexer is running

Say for example that I’ve got an Indexer running every day and it pulls down 2 million records from the SQL table every time it gets run. It takes around 30 min to finish off. During that time we keep adding/updating new records in the table, does the Indexer will pick up these new changes? Or it creates like a snapshot of the DB table with the data that had when we first triggered off?

I’m testing it and when the Indexer is about 60 % still picks up the changes that I’m doing to the table but not sure whether this will behave the same until it finishes.

I’m using the RowVersion column to keep track of the changes in the table.

Thanks!

Answer

If rows are updated/inserted in the data source while the indexer is running, any changes may or may not be picked up by the indexer in the current execution. Changes are guaranteed to be picked up on the next run that starts after the updates/inserts.

If you want to ensure a change is captured by the indexer, you can set a more shorter schedule, then the indexer will run more frequently but do less work each time.