This article provides a step-by-step guide to configuring and deploying the Process Route shape and component. You should be familiar with the Process Call shape and subprocesses in general before attempting to use the Process Route.
The Process Route allows you to dynamically choose a subprocess to execute at runtime based upon some document value. The "routing rules" are configured separately which define the mapping between a given value and the subprocess to execute, as well as the "interface" for how the subprocess should behave such as Passthrough or not and the number of expected Return Document paths.
The feature can greatly reduce the configuration and maintenance effort when building processes that must handle many different types of documents and therefore many different execution paths. However additional upfront design and coordination is required to use effectively. If your process only needs a few routes that are relatively static, using a Process Route may be overkill. If the process has many routes that are constantly changing, the extra design and configuration sophistication will pay off in the long run with respect to ongoing maintenance.
The two key aspects that make Process Route different than a regular Process Call are:
- Ability to dynamically determine which subprocess to execute based on some runtime criteria
- Ability to deploy the main and subprocess(es) independently
IMPORTANT Deployment independence is a critical concept to understand. See Understanding Deployment Implications below for more details and considerations.
The scenario for this tutorial is as follows:
- Inbound files for many different types of records are placed in a single disk directory.
- The format (and consequently the Profile), mapping rules, and destinations vary by record type.
The files follow a consistent file naming convention: <recordtype>_<date>.<file extension>.
- For this tutorial there will be two record types:
- Orders - Inbound sales orders in a CSV format that should be sent to the Order Management database
- Employees - Employee events such as new hires and terminations in an XML format that should be sent to the HR application via web services
Without the Process Route shape there are several alternative approaches for accommodating this situation:
- Create a single process that reads all the files and uses a Route shape to send each file to a record type-specific subprocess.
- Create a separate process for each record type that only reads those files.
However these approaches have several drawbacks:
- With the first option, every time a new record type is added or if one of the subprocesses needs to be modified, the parent process needs to be modified and redeployed which will also redeploy all the other subprocesses. This can be very difficult to manage if changes to another subprocess are underway but not ready to be deployed yet. Additionally when the number of potential routes exceeds 20-30, the Route shape can be impractical to use.
- With the second option, there would simply be a lot of processes to create, deploy, and moreover schedule. Any common elements would need to be maintained and deployed repeatedly.
With the Process Route feature, the main process and individual subprocesses can be added, maintained, and deployed independently. Additionally it requires only a single main process to develop, deploy, and schedule, which is more efficient with respect to the maximum concurrent executions on the Atom and provides more control over throttling.
See Understanding Deployment Considerations below for more information on deployment dependencies and implications.
The general approach for using the Process Route is as follows:
- Design considerations including the type of subprocess, route key value, and return paths
- Create subprocesses
- Create Process Route component
- Create main process with a Process Route shape
- Deploy main process, subprocesses, and Process Route component
Note the Process Route paths within a given process execution are still executed sequentially, NOT in parallel. Return paths are executed sequentially as well.
The example processes below are used to illustrate the relevant concepts and are not intended to be fully configured.
The Process Route component essentially defines the interface or contract between the main process and subprocesses so it is important to think through the desired interaction upfront.
Below are the relevant design considerations for this example:
- General: There should be a single process that reads in all files, and routes each file to a specific subprocess that can handle the mapping, logic, and target destination for the given record type.
- Process Type: The documents from the main process should be passed directly into the subprocess so the subprocesses will need to be configured as Data Passthrough.
- Route Key: Because each subprocess will handle a single record type, it makes sense to use the record type value as the Route Key. The record type is available as part of the incoming file name so the main process can parse the file name to determine the record type. The resulting value can be stored in a dynamic document property named "RECORD_TYPE" to use for the Route Key lookup.
- IMPORTANT The Route Key is an exact value match. The main process is responsible for preparing the value appropriately to match the Route Key exactly.
- Return Paths: The main process should do some generic logging upon successful or failed completion so it needs to know the outcome of the subprocess. If separate return paths "Success" and "Failure" are used, each subprocess can implement logic as necessary to indicate success or failure.
- Additional Design (optional): To consistently return information about the failure--whether due to invalid data, connectivity issue, or application response, the main process will expect the subprocess to populate a dynamic document property named "FAILURE_MSG" when returning documents via the "Failure" path.
2. Create Subprocess(es)
- Create a subprocess for each record type. The Start shape should be configured as Data Passthrough.
- Each subprocesses should include validation, mapping, destination connector calls, etc. specific to the respective record type.
- Each subprocess should have two Return Document shapes, Success and Failure, and perform routing logic to ensure documents are sent to the appropriate Return Documents shape.
- Additionally the subprocess should populate the FAILURE_MSG dynamic document property as necessary, for example using the business rules validation result, try/catch message, application response field, etc.
- Save the process.
3. Create Process Route Component
- Create a new Process Route component.
- On the General tab, enable "Passthrough" and add two Return Paths, "Success" and "Failure" (names are arbitrary):
- On the Process Routing tab, add two Keys, "ORDER" and "EMPLOYEE". For each record type, select the respective subprocess as the Route to Process, and map the subprocess's Return Documents shapes to the Process Route's Return Paths appropriately:
- Save the process.
4. Create Main Process
- Use the Disk connector to read all files.
- Add a Business Rules shape configured with a single map function rule to parse the record type value from the incoming file name and store in dynamic document property named "RECORD_TYPE". This will be used for the Route Key later.
- For more details on this Business Rules technique, see How to manipulate properties without a Map shape. Map function details:
- Add a Process Route shape configured with the Process Route component created above. Leave the defaults for the other General tab fields. On the Route Parameters tab, add a single parameter type for the dynamic document property "RECORD_TYPE."
- The Process Route's Return Paths are automatically drawn on the canvas. Perform subsequent steps as appropriate for your scenario. In this example, the Failure path throws an exception and references the common FAILURE_MSG dynamic document property:
In summary, it is important to note that none of the configuration is file- or record-type specific.
5. Deploy Components
On the Deploy tab:
- Deploy (and attach to the desired Environment/Atom) the main process.
- Deploy (and attach to the desired Environment/Atom) all subprocesses referenced in the Process Route.
- IMPORTANT This is mandatory. This is different than the regular Process Call in which the subprocesses are automatically included in the main process's deployment.
- Switch the Component type list to Process Route:
- Deploy (and attach to the desired Environment/Atom) the Process Route component.
6. Execute and View Logs
When the main process executes, keep in mind the subprocesses are Data Passthrough so they will not display as separate executions records in Process Reporting. However the individual subprocess(es) that were executed are captured in the main execution's Process Log:
Also remember the incoming file name will be captured on the inbound disk connector's tracked fields which can be viewed and searched in the Document view.
Understanding Deployment Implications
One of the greatest advantages of the Process Route component is the deployment flexibility. Because the Process Route component and subprocesses can be deployed independently, it greatly reduces the regression impact when changes arise. This flexibility allows you to modify and redeploy a single subprocess without having to redeploy the main process which would typically include all other subprocesses.
- If you need to modify one of the subprocesses, you only need to redeploy that subprocess. You do not need to modify or redeploy the main process or the Process Route component. However keep in mind any changes should be compatible with the design assumptions of the main process.
- If you need to add a new route and subprocess, you only need to create and deploy the new subprocess and then update and redeploy the Process Route component. You do not need to modify or redeploy the main process.
- If you need to change common logic in the main process (e.g. error handling), you only need to modify and redeploy the main process. You do not need to modify or redeploy the subprocesses or Process Route component. However keep in mind any changes should be compatible with the design assumptions of the subprocesses.
- If you need to make a change that affects the interaction/interface between the main process and subprocesses (such as adding a new Return Path or assuming the availability of some process property or document cache), you should do so thoughtfully to ensure backward compatibility. Depending on the nature of the change this could be as simple as providing default values for properties or as complex as full coordination and redeployment of all processes and components.
- Because there is no deployment dependency between the main and subprocesses, it is quite possible to have different versions of the same component used in different processes (for example, you could have Profile revision 1 in the subprocess but Profile revision 2 in the main process). Take care to ensure any changes to common components are non-destructive and backward compatible, or that you coordinate the re-deployments of both the main process and subprocesses in those situations.
- Extensions must be declared in each deployed process; in other words in both the main and subprocess. For example, if both the main and subprocess wish to use the same connection or process property extension, the extensions must be declared in both processes. This differs from the regular Process Call shape scenario in which all the extensible components from the subprocesses are "rolled up" and declared only once in the main process.
Understand Process Route's execution behavior:
- Documents will only be routed to a single path. Duplicate and wildcard keys are not allowed so documents cannot match to multiple paths.
- Documents are first grouped together by Route Key (similar to the regular Route shape) and then each group of documents is sent to the respective subprocess in the sequence defined within the Process Route component.
- The Return Paths are executed in the sequence defined within the Process Route shape, not the component.
- The Default path for documents that did not match a Route Key is executed last.
- Again Process Route subprocess executions behave the same as regular Process Call executions with respect to:
- Process Reporting - "Passthrough" subprocesses are logged as a continuation of the main process; there is not a separate execution record. Non-passthrough subprocesses will create a separate execution record.
- Execution "context" is shared so things like process properties and document caches are available and modifiable between the main and subprocesses.
- When performing a "deep" copy (Include Dependents=true) of a process with a Process Route shape, the Process Route component is NOT copied.