When integrating Salesforce with other systems through Boomi Cloud I noticed many requests result in a timeout.
I either use custom Apex HTTP Callouts or Salesforce native Outbound Messages to send data from Salesforce to Boomi Web Service Server on the Cloud Atom (realtime integration). Using custom Apex code I can set the maximum timeout for a response at 2 minutes. Outbound Messages 'only' have a 10 seconds timeout and this cannot be changed.
The problem I'm facing is that Boomi barely responds within 10 seconds and frequently takes more than 1 or 2 minutes to finish a process and sent an 'Acknowledge' response back to Salesforce. Because of this, the messages in Salesforce are marked to be retried at a later time. However, Boomi does (eventually) process the message from Salesforce. This causes the following problems:
- Duplicate messages and data are sent to Boomi. This means a lot of performance overhead, which in turn only increases the chance of Boomi being too late with a response.
- Duplicate data is sent to the destination system. We have always have to find some workaround and (if possible) use guards to avoid duplicate data creation in the destination system.
- In Salesforce there is a limit of how many outbound requests can be sent daily. Because messages are continually being stacked up in the 'retry' queue, at some point we actually reached our 250000 (two-hundred-fifty-thousand) request limit. This causes serious issues with other integration flows.
- Error handling gets more complex than it should be: 'Is the duplicate data actually an error or part of the integration fault?' Should we return an error or success message?
I created a process to demonstrate the problem (see image below). Breakdown of the process:
- The Web Services Server receives an Outbound Message from Salesforce.
- The message is split into multiple documents using the Data Process Shape.
- An Account is retrieved from Salesforce using the Salesforce connector (I do this twice to add a little more work in the process, ignore this).
- The documents are then combined using another Data Process shape
- Finally a return message is sent back to Salesforce. (Note that moving the 'Return Documents' shape to the front does not increase the response time of Boomi. According to the documentation, all documents that reach this shape will be batched and only be returned after the whole process has finished).
In below image you see a screenshot of the 'Process reporting' page in Boomi where above process is shown as executed multiple times. The first thing that I noticed is that there is a great variation in the 'Elapsed Time': ranging from 7sec to 2min:21sec (average 35sec). At first I did not understand how this is possible "Is Salesforce sometimes slow with responding to the 'Get Account' request?" Well, yes.. but this only adds a couple of seconds at maximum. The real difference in Elapsed Time comes from the moment the process shows up in the 'Process reporting' page and the moment where the process is actually starting the 'Initializing...' Shape (see screenshot). The process took 2min:9sec to start up!
I already tried enabling the 'Low Latency' option in the process build editor, but this did not result in a visible decrease in timeouts.
I hope that this issues can be solved or that I'm doing something wrong. Does anyone else experience these issues or any suggestions to help me solve them? Because right now, the situation is really not ideal.
Thanks in advance,