I have requirement to filter the duplicates from Source file and take the best record out of it.
For Ex: the given below flat file profile has duplicate record of "Norred Chris" (email@example.com)
If you take a closer look on the 2nd record of 'Norred Chris' you could see it has extra information 'New York' as its StateorProvince, which the earlier one does not have. So my aim is to pick this one (2nd record of Norred Chris) among the duplicate. I can filter by "Enforce Unique" on the field "ExternalEmailAddress", which may not guarantee me the best record.Hope you get my question. Please let me know your thoughts on this and feel free to ask me any question in case you need more information.
firstname.lastname@example.org,Darren,Parker,4/11/1973,1234 23rd Ave,Denver,,12345,,5852345431,,,Xerox,,,Printer
email@example.com,Chris,Norred Chris,11/11/1967,"Norred PO Box 1234 Greeley CO",Rochestor,,87649,,5855854235,,,Accenture,,,IT
firstname.lastname@example.org,Chris,Norred Chris,11/11/1967,"Norred PO Box 1234 Greeley CO",Rochestor,New York,87649,,5855854235,,,Accenture,,,IT
Thanks in Advance..!