One of my processes behaves strange dropping a batch when using data process for search and replace. Scenario is here
- A SOAP response is received from a vendor system with unwanted double quotes making the XML not well formed.
- For Boomi to proceed further the XML should be well formed. The vendor system is not ready to give us well formed XML but asked us to do the necessary removal.
- We decided to use a data process to remove these double quotes ('''' & =") from incoming SOAP response.
We were successful in removing the unwanted double quotes and tested ok. However, when pulled data from PROD (Old code) to compare the new code tested data, we found some records were missing. This has put us in trouble. Again after few tests i found that the data process that removes double quotes, is removing a batch of records. Since we get thousands of records as SOAP response, we split documents as 150 records and in flow control 150 documents per batch. Which means 150 records get dropped. If 200 records, then 200 records get dropped after passing through the data process. When I remove the data process, all look good no records missing. Few screens for better understanding
Batching of records
Data process handling double quotes
Note that it is not random records lost but a document is lost with 150 records if split documents as 150 records.
Appreciate your help