AnsweredAssumed Answered

Batch of records lost when using data process for search and replace

Question asked by britto.parthalomew on Jan 16, 2017
Latest reply on Jan 18, 2017 by britto.parthalomew

Hi Folks,
One of my processes behaves strange dropping a batch when using data process for search and replace. Scenario is here
- A SOAP response is received from a vendor system with unwanted double quotes making the XML not well formed.
- For Boomi to proceed further the XML should be well formed. The vendor system is not ready to give us well formed XML but asked us to do the necessary removal.
- We decided to use a data process to remove these double quotes ('''' & =") from incoming SOAP response.
We were successful in removing the unwanted double quotes and tested ok. However, when pulled data from PROD (Old code) to compare the new code tested data, we found some records were missing. This has put us in trouble. Again after few tests i found that the data process that removes double quotes, is removing a batch of records. Since we get thousands of records as SOAP response, we split documents as 150 records and in flow control 150 documents per batch. Which means 150 records get dropped. If 200 records, then 200 records get dropped after passing through the data process. When I remove the data process, all look good no records missing. Few screens for better understanding

 

Batching of records

Data process handling double quotes

Note that it is not random records lost but a document is lost with 150 records if split documents as 150 records.

 

Appreciate your help

 

Thank you

Britto

Outcomes