We have a scenario where we use a script to pick files from Amazon S3 into a disk directory from where BOOMI picks it up for processing. We have noticed that BOOMI has started picking files from the disk directory even before the script completes writing the same. Due to which we more often get Zero Byte files or incomplete files for processing.
As a workaround we are trying to modify the script to write the file as a hidden file (name starting with a '.' (dot)) until such time that the file is copied / moved entirely. We tried to test and see if this worked, but we found that BOOMI was picking these hidden files as well.
Can someone help me with an option that we could use in BOOMI to ensure these hidden files are not picked?