Delete Files From FTP Only When Files Get Processed Successfully

Document created by sheng_liao462475 Employee on Nov 11, 2016
Version 1Show Document
  • View in full screen mode

Use Case

Retrieve files from FTP and process the files, if files get processed successfully, delete them from FTP, otherwise don't delete them.

Currently we don't have DELETE action in FTP operation. But there are 2 alternative techniques you can use to achieve this.

 

Approach 1

Your process design will be something like this.

On Branch 1, you will need to use an FTP connector with GET action to retrieve files.

 

On branch 2, use a Data Process shape with Custom Scripting to delete files from FTP. When files get processed successfully, it will hit branch 2 and delete files, otherwise process will fail at branch 1.

Step 1

Download jar file(Binaries) from the below link.

Apache Commons Net – Download Apache Commons Net 

Step 2

Put the jar file under <atom installation>/userlib/script folder and restart the atom.

Step 3

Copy and paste the below script to data process shape.

import java.util.Properties;
import java.io.InputStream;
import java.io.IOException;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.OutputStream;
import java.util.Properties;
import com.boomi.execution.ExecutionUtil;

 

import org.apache.commons.net.ftp.FTPClient;
import org.apache.commons.net.ftp.FTPReply;
import org.apache.commons.net.ftp.FTPFile;

 

logger = ExecutionUtil.getBaseLogger();

 

for( int i = 0; i < dataContext.getDataCount(); i++ ) {
InputStream is = dataContext.getStream(i);
Properties props = dataContext.getProperties(i);

 

//FTP connection details
String server = "host";
int port = port number;
String user = "user";
String pass = "password";

FTPClient ftpClient = new FTPClient();
try {
ftpClient.connect(server, port);
if (!ftpClient.login(user, pass)) {
logger.error("Unable to login to FTP server");
}
String fileToDelete = "file path";
ftpClient.deleteFile(fileToDelete);
} catch (FileNotFoundException e) {
logger.error("file not found" + e);
} catch (IOException ioe) {
logger.error("Exception while deleting the file" + ioe);
} finally {
//logout and disconnect - optional
//ftpClient.logout();
//ftpClient.disconnect();
}

 

dataContext.storeStream(is, props);
}

Notes: please replace the red portion with the correct values. The above script only delete specific file, if you would like to delete ALL files from folder, you will need to use a for loop to loop through all the files in the folder and delete them. The above script will NOT work on Atom Cloud or Test Atom Cloud because we don't have the custom jar file installed there.

 

Approach 2

Your process design will be something like this.

On branch 1, you will need to use an FTP connector with GET action to retrieve files.

On branch 2, you will need to use an FTP connector with GET and DELETE action to delete files.

2 people found this helpful

Attachments

    Outcomes