JSON profile ignores duplicate data. Expected or not expected behavior ?

Document created by anton_serbanescu Employee on Dec 19, 2016Last modified by chris_stevens on Jan 4, 2017
Version 2Show Document
  • View in full screen mode

Suppose you have a Flat file with duplicate data :





The process picks up this data and maps it to a JSON profile. By default, this only generates only 3 files (outbound documents from the map), but you were expecting to get 4 files with the duplicate data, as it is needed later in the process and duplicates are handled there.


In order to generate the 4 separate files, the process needs at least 2 additional shapes:

  • Data Process - to split the data by line
  • Flow Control - to send each individual row or record as a separate input file to the map.


In the screenshot above, Branch 1 will generate 3 files without any duplicates, but Branch 2 will generate 4 files, ignoring duplicates.

1 person found this helpful