We are processing 30-50K records from a Flat File. These record contain User ID in it.
We have a requirement to check existence of user in database (MS SQL Server) as a validation of this incoming file.
I have tried with SQL Lookup function but it is taking long time e.g. processing only 5k records in 1 hour.
User Table has around 400K-500K users.
Is there any better solution for handling this kind of scenario?