I am fetching data from CSV file using disk connector, it's taking too much time while fetching 500000 records, it there any way to fetch data in batch mode for better performance?
No, this is not possible. You can't read read a file in batches from disk.
If there are multiple files, you can configure Maximum Files
yes, I saw that option but we don't have any other way to resolve or achieve this thing?
No, without reading the file you cannot do anything.
After reading the file, you can do some batching using flow control.
Thanks for your previous response.
I tested that process yesterday with 1 lac records, so it took 12 minutes to complete the whole process.
so it means if we have more than 1 lac records so it will take to much time to execute the process.
don't you think that will be very problematic things for that client or company?
Retrieving data ...