AnsweredAssumed Answered

How to count documents returned from Salesforce upsert operation

Question asked by abupp137129 on Nov 6, 2017
Latest reply on Nov 7, 2017 by abupp137129

I am attempting to add error handling to a process which upserts data to SF.  I have configured the operation to "Return Application Error Responses".  For anyone who has experience working w/ Salesforce's SOAP API, they know that the upsert response messages that Salesforce returns have a somewhat funny (meaning strange, not comical) XML implementation.  In the case of a success, it's pretty straight-forward and simple, there are id & success fields which indicate the SF object upserted, and if the upsert was successful.  Here's an example:




However, in the case of a record failure, the returned XML document looks somewhat different:


      <message>OBJ: Asset - insufficient access rights on cross-reference id: 0011W00001rYoA7</message>


Now, in terms of error handling this message leaves something import missing, namely the SF ID, or any other way to identify which source record which caused the error.  This is probably because the record didn't already exist in SF, so its ID could not be put into the error message.  In any event, how can anything meaningful be made of an error such as this to an end-user without some identifier?  If one tries to see the "cross-reference id" shown above in SF, it quickly becomes clear what the problem is, namely that the record no longer exists in SF. 


Still, the question is which source record is the culprit.  If the process only happens to be syncing a handful of records, this could be determined manually by inspecting the documents returned from SF in the process report.  Unfortunately, the process in question typically processes hundreds of records at a time, batched together in calls to Salesforce, 200 at a time, so this is not a practical approach.


Fortunately, there is a way to tie the Salesforce response to a unique identifier base on the fact that the order of documents returned from Salesforce matches the order in which they are upserted.  This means that if one can know the record number of the returned document, that can be used as an index into the source data, from which a unique ID can be extracted.  The problem is how to determine that record number when processing the responses.


I thought I could increment a counter of sorts based on the output of a decision shape that compares the value of the success element in a given document.  The results of this decision then direct processing to maps where either a success or an error counter are incremented.  In either case, these maps also increment a "record number" counter, the idea being to keep track of the "current" record index, as well as both the total number of failures and successes.  Here's a screen shot of this part of the process:



Here's how I set the record number (and other) counters, using a map function to set a process property:



The trouble is that the current record counter doesn't match the order in which the records come back from Salesforce.  Here's a screen shot from the source data of the decision shape of a process test where the input data set has been limited to three records, in which the second record is known to cause an error when upserted to Salesforce:



Note the 2nd document is significantly bigger, which is a clue that it is in fact an error response.  However, the decision shape appears to group the documents together which match the success condition, and process those first before sending any documents down the failure path.  The evidence for this is that despite the fact that the 2nd document has the error, the current record index when the process goes down the failure path is 3:



Just in case one wonders (as I did) that this is a random result, or one specific to the given input data set, I also ran test (after deploying the process) where I had the process handle 145 records, which had just one Salesforce error producing record.  Here is the output from the Notify shape shown above in the failure branch above in this case, note that the counter is again set to the total number of records processed:



That Object Id value shown above was taken from a lookup on a document cache created on the input data which maps the document index to a source system identifier.  The value shown (32433001-SBC) is, of course, not the correct one since the index being used for the lookup (145) should be 123, 


So, in the hopes that there are still some who have gotten this far after the lengthy explanation, is there some way to do this, so that the process can know that in the first test the error producing record had index #2 and had index #123 in the second test?


Thanks in advance.