This article provides an overview of the options and best practices for monitoring your integration processes.
Knowing when integration processes fail or do not complete as expected is a critical part of managing the integration platform. Boomi Enterprise Platform provides a number of options to suit your needs depending upon the volume of executions, desired communication method, and sophistication of notification routing and delivery requirements.
For monitoring the general availability and health of your basic runtimes and runtime clusters, see Operational Monitoring Techniques for basic runtime, runtime cluster, and runtime cloud Runtimes.
The table below describes the various process monitoring options and best practices for use.
| Option | Description | Best Practices |
|---|---|---|
|
Process Reporting (UI) |
The Process Reporting console provides detailed information about process executions including process logs and documents.
Manage > Process Reporting |
The Process Reporting console can be used to manually monitor low volumes of executions but as activity increases, handling Events on an exception-basis becomes necessary. However regardless of how you are notified about an error, you will most likely use the Process Reporting console to research logs and document-level information, troubleshoot, and rerun documents. See also Overview of Boomi Enterprise Platform Logs for Troubleshooting.
Use Document Tracking to capture key values from document data to be able to quickly find and trace a given record through multiple process executions.
Important notes about Process Reporting:
|
|
Dashboards (UI) |
The Dashboards provide historical and summary information about process executions and inbound requests.
Dashboard > choose dashboard |
There are three dashboards that can be used for trending and historical analysis:
|
|
Email Alerts (Event Framework) |
Boomi Enterprise Platform users can choose to receive platform Events via email. Users can choose to subscribe to Events by Log Level and Type.
Account menu > Setup > Email Alerts |
Email alerts are best used for exception-based notifications that will be received and acted upon by individuals. As a general recommendation, configure the Log Level to WARNING to receive alerts for WARNING and ERROR events but not expected successful activity such as "process started" and "process completed". See Email alert management.
For higher executions volumes or if more advanced subscription/routing rules are desired, use the Event API to consume and handle Events as needed. See the Platform API section below.
Important notes about email alerts:
|
|
RSS Feeds (Event Framework) |
Platform Events are available via RSS feed per basic runtime or at the account level.
Account menu > Setup > Account Information or Manage > Runtime Management > choose basic runtime > basic runtime Information |
RSS are the simplest approach for receiving platform Events. If you have an incident reporting system or monitoring tool that can consume RSS feeds, consider using them.
For exception-based monitoring, use the "Alerts Only" feeds to only receive Warning and Error level Events. There are no other filtering options.
See Understanding the Event Framework. |
|
Platform API (Event Framework) |
Events can be consumed programmatically via the Event API.
There is no UI for the list of Events. |
The Event object provides the most flexibility for routing rules, message formatting, and delivery options. It is the best option for exception-based monitoring of larger numbers of executions.
For execution-related Events, you can use the Execution Record object and Download process log operation to retrieve additional details about the execution. Note the API does not provide access to document-level information including tracked fields.
You can create another Boomi Enterprise Platform process (or use an external monitoring tool) to periodically extract new Events and route and deliver as required. For example, Events could be sent to your incident reporting application to leverage existing communication channels and escalation rules. Note the connection to the destination system including your company's SMTP mail server will consume a connection license.
IMPORTANT: Event records are available for 7 days.
See Understanding the Event Framework. |
|
Custom (Built into Process Flow) |
Custom error handling and notification logic can be built into the process design/flow as required. |
If truly custom notification logic is required, you can design this into the process flow itself by capturing errors and validation warnings using Try/Catch and Business Rules steps and connector operation response handling. For example, runtime errors could be caught, formatted with a Message step, and emailed directly to recipients using a Mail connector and your company's SMTP mail server.
Alternatively, the errors could be mapped to custom alert message XML format, and written to an external database or queue for subsequent processing. This option may be desired when integrating with an existing notification/auditing/monitoring framework or to provide custom attributes in the notification message. It requires additional upfront design, development, and testing effort. It is still advised to use the platform Event subscriptions to be notified of problems that cannot be handled within the context of a process execution. |
The Process Reporting console displays execution results for the past 30 days. This period cannot be changed. The results include process- and document-level metadata such as execution statistics, document counts, and document tracked fields.
The Process Reporting console will always retain the summary execution history for 30 days, however the execution artifacts such as process logs and document data can be purged more frequently. This may be desirable for basic runtimes/runtimes processing a large volume of data. Note that purge schedules are configured for the entire runtime, not per-process.
The runtime can also be configured to "Purge Immediately" so that execution artifacts (most notably document data) are removed immediately upon completion of the execution. Again keep in mind the Purge Immediately setting applies to the entire runtime, not individual processes.
Execution artifacts for processes running on the Boomi runtime cloud are purged after 14 days. This can be decreased but not increased beyond 14 days.
While it is technically possible to increase the purge history beyond 30 days for local runtimes, it is not practical because the execution summary information in the Process Reporting console is only available for 30 days.
The purge frequency can be modified per runtime in Runtime Management > basic runtime Properties (or runtime cluster or Cloud Properties). The Basic Property sets a “global” purge frequency for all execution artifacts. Additionally/alternatively you can override this setting by configuring purge frequencies for logs (process and runtime), documents (also applies to basic runtime Queues), and temp data individually with Advanced Properties.
For more information see Purging of basic runtime, runtime cluster, or runtime cloud logs and data.
Processes configured to use Low Latency mode require special considerations for monitoring. As noted above, Low Latency process executions do not generate Events for process errors and therefore you cannot rely on email alerts, RSS feeds, or even the Event API. User notifications (from Notify steps) are created and therefore could be received via email alerts, however because Low Latency mode is typically used for high volume scenarios such as inbound web services and JMS messages, receiving an Event or email alert for each individual error can be impractical given the sheer volume of potential messages. Instead the strategy is to detect a general, persistent issue with the service and then use the available process logs and dashboards to investigate further.
Below are some additional considerations and recommendations for monitoring errors for Low Latency processes.