Hi fellow community users!
I have for a while now been running into a strange issue with my processes that are scheduled to run early in the morning. The processes are defined as a Salesforce Get operation as the start shape to make a query against Salesforce and retrieving ~200-300 records each run and process these further. The issues I am facing is when these are scheduled to run at 4.00 AM in the morning I am facing issues like this one:
Severe errors occurred during start shape execution, terminating process.; Caused by: Error received executing Salesforce Get; Caused by: com.boomi.util.retry.RetryException: com.boomi.model.connector.common.salesforce.SalesforceException: Error received for Salesforce Query; Caused by: com.boomi.model.connector.common.salesforce.SalesforceException: Error received for Salesforce Query; Caused by: Error received for Salesforce Query; Caused by: Error invoking soap operation; Caused by: java.net.SocketTimeoutException: Read timed out; Caused by: Read timed out
I found out that is was only happening in the mornings, amongst the first executions of the process (executing it late in the evening works fine). It is also occuring the same issue for processes that does not retrieve any data at all, I know this since I have ~4 processes where two have data to retrieve everyday and two does currently have no data to retrieve at the moment at all.
So this is occuring regardless of the data load for the process, I have looked into the following: https://community.boomi.com/docs/DOC-2471#jive_content_id_Error_Caused_by_comboomimodelconnectorcommonsalesforceSalesfor… and the https://community.boomi.com/docs/DOC-2471#jive_content_id_Error_Error_received_executing_Salesforce_Get_Caused_by_javane… to solve it by I cannot get the following out of my head:
- Why does it seem to be an issue with this in the mornings run? If there were some issues with the query running for too long or too much data/records being retrieved, why does it then work if you execute it manually later on?
Furthermore, the issue seems to be present for our Production Environment in Boomi but not for our QA/Test environment, they are towards different Salesforce isntances and no issues occurs at the Salesforce instances that I can see. This leads me into the direction that there is some issues on the Server side, that goes wrong at early executions, Our IT department looked into this but could not find anything here.
My requirement is that this transfer needs to be done before 6AM each morning, which is why I have tested with setting up a scheudling to run every 20 minutes between 4AM and 5.40AM. For this it got the same issue with the first runs but the one at 5.40AM ran succesfull for about two weeks (with one or two days with the same issue even in the last execution). This week however, the issue has been occuring all days, meaning that none of the runs in the cheudling window have executed correctly.
I need some pointers towards if this is something that I can fix in Boomi or if this is something that the IT deprtment needs to take another look at and if so, what they should look for?
Measures I have taken myself to try to mitigate this on a process level:
- Remove any unused fields from the query - why this would make a difference I do not know since like stated above, it executes fine when running it manually (even one day with ~3 times the regular load).
- Scheduled it according to the above example.
The Salesforce Get query filter is two Formula checkboxes, but I cannot see how this can be an issue since (and I apologize for repeating myself like a parrot) this only seems to occurs in the early runs, not if I were to execute it at 4PM or even if there is no records at all to retrieve.
Any pointers here is very appreciated!