The data process shape - split document is running slow for 10 millions records.is there any alternative to achieve the same result with performance improvement?.
What is slow?
What format is the document? Are you using linking options?
How big is the document?
Is this a cloud Atom or Local? If Local, how much RAM is available to the Atom?
Is the document coming from a connector? If so, can you bring it in smaller batches?
What are you trying to do in DP?
Assuming you are pushing all the 10 million records to DP that might result in slow performance. I would suggest you to do the following
1. As soon as you get these records from a connector, split the document each document containing say 1000 or 2000 records
2. Send these documents to a flow control where you batch these documents into smaller batches
3. Then push them to next DP function
Let me know this helps
Performance depends on many factors.
Give us more details about atom hardware configuration and records that you would like to process.
Retrieving data ...