AnsweredAssumed Answered

How to merge two flat files into a single file with unique records?

Question asked by abanotra075558 on Jul 5, 2018
Latest reply on Jul 9, 2018 by abanotra075558

Hello Everyone,

 

I am new to Boomi platform and I pretty much tried everything before posting my question here but unfortunately, nothing worked out for me.

 

Expectation - One single merged file from 2 flat files with no duplicate records.

 

Current situation -  Getting 2 separate files (even though they are combined using data process) and the check for duplicates is not working.

 

I am basically facing 2 different issues and kind of stuck right now, I went through multiple blogs in the Boomi community but unfortunately, nothing worked out for me.

 

First Issue- I am getting 2 flat files as an output from the map and using the data process component to combine the flat files into single merged file  The data process combines the files but creates 2 files as an output of data process. As per my requirement, I need a single file with unique records.

 

a) I did try Message shape as well and checked the option to combine documents into a single message but the message shape is not producing any output (No error as well).

 

 

b) Tried data cache component as well, I am getting 2 separate flat files as expected at the document cache component from 2 Maps but when I am using the load from cache component then it is not producing any data (Which makes me feel like add to cache component is working fine but load from the cache is not working).

 

 

c) Also tried using data cache in the  input data source in the map in order to cache document data from "Add to cache component" but I am getting an error here which says that "Unable to create JSON files from data, the document may not be well-formed JSON"

 

 

 

 

Second Issue -  There are some duplicate records in both the flat files which are generated from 2 different Map components, As per my requirement I need unique values in my single merged file after combining both the flat files with the duplicate records. I referred to the groovy script mentioned in the link below but it's not working for me. I am getting the same data even after using this script.

How to filter a unique set of documents using Groovy 

 

b) Tried using the enforce unique option in the output file of the map but still not getting unique results.

 

 

 

Sorry for the mess in the screenshots because of too many components.

 

Any help or suggestion is greatly appreciated.

 

Thanks in Advance,

Arun

Outcomes