Suppose you have a Flat file with duplicate data :
The process picks up this data and maps it to a JSON profile. By default, this only generates only 3 files (outbound documents from the map), but you were expecting to get 4 files with the duplicate data, as it is needed later in the process and duplicates are handled there.
In order to generate the 4 separate files, the process needs at least 2 additional shapes:
- Data Process - to split the data by line
- Flow Control - to send each individual row or record as a separate input file to the map.
In the screenshot above, Branch 1 will generate 3 files without any duplicates, but Branch 2 will generate 4 files, ignoring duplicates.