The article provides best practices for how to configure and use Document Tracking to facilitate document reporting and troubleshooting.
Document Tracking allows you to capture key business data values from the documents being processed to help find those specific documents in Process Reporting for tracking and troubleshooting purposes. Using Tracked Fields is an important part of your functional monitoring strategy. You can think of them as custom fields for Process Reporting.
Document Tracking allows you to answer questions like "what happened with Order Number 123?" By configuring connector operations to capture specific values, you can trace an individual record through a process execution or even across multiple executions. This is extremely useful in decoupled design patterns.
For example, consider the following scenario. You have a series of individual processes that:
- Sync new orders from your website to the ERP application
- Send the order details from the ERP to the shipping application
- Updates the ERP with shipping tracking information
- Sends an invoice from the ERP to the customer
By configuring the same Tracked Fields on each inbound and outbound connector shape, you can easily search for a specific order number and trace its history through this workflow.
How it Works
- Tracked Fields are first defined at your AtomSphere account level. This is because Tracked Fields can be used across processes and are not tied a specific connector type, operation, or profile.
- Once defined in your account, you can configure individual connector operations to populate values for those Tracked Fields based on the data they receive or send. You will typically populate the Tracked Fields from profile elements of the document data being processed.
- At run time, these tracked values are captured as part of the process execution metadata and made available through Process Reporting similar to other system-generated document information such as file name and document size. Additionally you can search historical execution records by these custom Tracked field value to quickly find a specific document, and then drill down into the associated process execution for additional details and troubleshooting information.
For more details, see the User Guide Document Tracking.
How to Configure
In this example we will show how to configure Document Tracking to capture the "Customer ID" value when syncing account records from Salesforce to NetSuite. The objective is to capture the same value on both the inbound Salesforce query and the outbound NetSuite upsert calls for trace-ability.
1. Define Tracked Fields
Configure a new Tracked Field for your AtomSphere account.
- Navigate to Setup > Document Tracking.
- Click to add a new field.
- Enter the Field Label and Data Type. The Data Type determines the available filter options later.
You will need the Account Administration privilege to configure Document Tracking.
2. Configure Connection Operations
The following example process extracts account records from Salesforce and sends them to NetSuite. We want to configure Tracked Fields to the primary connector operations components to capture identifying record information.
Add Tracked Fields to the inbound Salesforce operation.
- Edit the Salesforce Get operation component and go the Tracking tab.
- Add the "Customer ID" Tracked Field from the previous step.
- Configure the value to populate the Tracked Field. In this case choose the "Name" field from the response XML profile (the one displayed on the operation's Options tab).
- Save the operation.
Perform the similar steps for the outbound NetSuite Upsert operation.
- Add the same "Customer ID" Track Field.
- For the value, choose the "entityId" field from the request XML profile.
When populating the Tracked Field value, it is possible to concatenate multiple dynamic and/or static parameter values.
3. Process Reporting: View Executions Details
Deploy and execute the process then go to Process Reporting to view the execution details.
- Go to Manage > Process Reporting and find the execution result.
- Drill down to the execution details and select the Salesforce query operation.
- In the document details table, note the new Customer ID Tracked Field with the value captured from the document that was executed.
- You can do the same for the NetSuite Upsert operation and note the Customer ID Tracked Field and value as well.
4. Process Reporting: Search Documents
Search for a specific Tracked Field value across all documents and executions.
- Within Process Reporting, switch to the Documents view.
- Add a filter for Tracked Fields, enter a search term (use * or ? for wildcards), and select the specific "User Defined" field(s) to search.
- The results show documents across connector operations and process executions matching the Tracked Field search term.
- Use generic names - Avoid process- or endpoint-specific names for Tracked Fields because they will be used across processes, applications, and data formats. For example, consider "Account Number" vs. "Salesforce Customer ID".
- Only track a few fields - Capture only the key values required to identify and troubleshooting documents.
- Do not store sensitive values - Tracked Field values are uploaded and stored in the AtomSphere platform database as part of the execution metadata. They are not stored in your local Atom runtime. Be aware of this and avoid tracking sensitive values such as credit card numbers, social security numbers, or any other protected information because they are stored unencrypted outside your local environment. Also note that the View Data privilege (which prevents users from viewing document data) does NOT apply to Tracked Fields values.
- There is a slight performance impact - Because the connector needs to parse the data to extract the values, there will be a slight performance decrease. Typically this is negligible and the ability to easily troubleshoot records outweighs the costs, but it is good to keep in mind not to track too many fields for a given connector operation.
- Do not use for batches or repeating values - Generally if the incoming data contains a list of values, it does not make sense to capture a value from a repeating item because it will only capture the first instance. For example, if your process reads a single large batch file containing many customer records, it would not be appropriate to track customer-level values. Similarly if your process sends an invoice record with multiple line items, it would be appropriate to track the "invoice number" and "customer number" from the "header", but do not track "item number" from the repeating line details.
- Create unique names - Be sure to use unique names for Tracked Fields.
- Use consistently - To get the most benefit from Document Tracking, be sure to consistently configure Tracked Fields for new connector operations.
- Tracked Field values are only captured for connector operations used within Connector shapes. They are not captured for connector operations used in map functions or in other shape parameters, such as a Connector Call lookup within a Decision shape, even if configured within the operation.
- Tracked Field values are only captured for deployed process executions. Values are not captured for Test Mode executions.
- Tracked Field values are available to view in Process Reporting for 30 days, same as all execution records.
- If you have a Low Latency listener process, remember execution information is not captured which includes Tracked Fields.
- Deleting a tracked field is permanent. Because very careful if you choose to delete a Tracked Field.
- The maximum value length is 1000 characters. Do not store the entire document data (i.e. Current Data).
- The maximum number of Tracked Fields per account is 20.
- Tracked Field values are not available through the AtomSphere platform API. In other words, you cannot use the platform API to query executions by a given Tracked Field value. The exception to this is the X12 Connector Record object used for EDI processing.
Are tracked profile fields taken from the request or response document?
The most common use case for Tracked Fields is to capture values from the document data using the Profile Element parameter type. Because some types of connectors actions both take in and return documents, it is important to know which document will be used for Tracked Fields.
- For connector "get" actions (e.g. Get, Query, Listen), the response document is available for tracking, that is, the document data returned from the connector shape into the process.
- For connector "send" actions (e.g. Send, Create, Update, Upsert, Delete, Execute), the request document is available for tracking, that is, the document sent to the connector shape.
However there are a few limited exceptions:
- A small number of connectors behave slightly differently for "send" actions when the "Return Application Error Responses" or "Return HTTP Responses" setting is enabled in the operation. In this specific situation the response document is available for tracking instead of the request. Because typically the request fields are desired for tracking, consider disabling "Return Application Error Responses" and instead handle errors using a Try/Catch shape. Connectors exhibiting this behavior include Salesforce, SAP, HTTP Client, and Web Services SOAP Client, OData Client, Red Prairie, QuickBooks, QuickBase, and any "legacy" connector.