Input Flat file has more than 500K records(In other case it can be much much bigger as well) but due to business requirements, we had to use Flow control node to run each document individually. Now if we run the process it takes very long time to complete the process but its definitely not a good idea to have a process taking that long in PROD.
So the real question I have is:
- Is there anyway to make this process run faster??
- Is there a way to see the exact %age of the process completed (i.e. 50% through, 90% through) (not the process state)??
- Is there a way to see every individual record in logs, so we can see that till this point the file has been processed? Usually with such a big file it becomes difficult to see the individual records in platform UI (due to huge data flow??).