subra

Work-Around for Salesforce Bulk API V2.0

Discussion created by subra on Feb 5, 2018
Latest reply on Apr 8, 2018 by mallikarjun.ratala563207

Summary: Salesforce Bulk API Usage, though offers performance benefits over regular APIs is a scarce resource for organizations. It is important to conserve and optimally use the Bulk API.

-------

As I understand, Boomi Salesforce connector does not support the Bulk API V2.0 (which essentially makes a determination automatically whether to use the regular or bulk API). This feature is much sought in the forums since organizations hit their Bulk API usage limits frequently. Processing much lower volumes of documents through the Bulk API (Bulk API accepts 10000 documents in a single go)  may be a wasted opportunity to use the regular API (Regular accepts 200 document in a batch) which generally has a much higher usage limit and mostly remains underused. A work-around that can mimic the Salesforce Bulk API V2.0 can be built in the lines described below:

 

Step 1: Create a Reusable Sub process that accepts any document shape and makes a determination if the documents need sent down a "Regular" vs. a "Bulk" path. This is done by a data shape with custom scripting that counts the number of documents and a subsequent decision shape that sends it down a regular path for a count less than, say , 2000 vs. a bulk path when the count exceeds 2000. I picked a arbitrary number but experimentally one can determine where the bulk APIs are wasted in their process.

 

 

Step 2: In the Main process add a Flow Control shape to group and process in batches of 10000 documents

Step 3: Duplicate the Sales force Operation and name the copy with a Prefix or Suffix of Bulk. Ensure to check the Bulk Operation check box and set batch size of 10000 (or your process specific optimal number).

Step 4: Wire the Return paths from the re-usable sub process to the Bulk Vs. Regular API Calls (with any transformations if needed)

 

 

This approach may need tweaked specific to the volumes you hit but hope this explains the general idea. I've personally have had great success using this approach to eliminate wastage of Bulk API calls.. I appreciate any suggestions and feedback.

Outcomes