Getting and setting document properties in the Connector SDK

Document created by sdecima Employee on Apr 18, 2017Last modified by Adam Arrowsmith on Jun 14, 2017
Version 5Show Document
  • View in full screen mode

Sometimes it’s useful to have documents carry extra properties or metadata that isn’t part of the document payload. This article explains how Connector SDK developers can build that functionality into their connector descriptor file to give users greater flexibility and tracking with their connector processing.

 

 

Designing document properties

For general context and as a refresher, within AtomSphere integration processes there are two types of document properties (described in more detail in the AtomSphere Users Guide):

  • Standard document properties - Predefined properties declared by connectors and selectable by end users. These are the ones we will focus on in this article.
  • Dynamic document properties - Free-form text names defined by end user users. Within the SDK these are called "user-defined" properties. 

 

When developing custom connectors, it is best practice to declare your own standard document properties vs. relying on dynamic document properties because it makes the available options much clearer to end users and allows for document tracking within Process Reporting.

 

Within the SDK, standard document properties are broken down further into two sub-types:

  • dynamicProperty - "Inbound" values set by the user and passed into a connector shape to augment or override default connector component (typically the Operation component).
  • trackedProperty - "Outbound" values set by the connector and returned to the process to provide the user with additional metadata about the documents. These values are also captured and displayed as part of document tracking in the Process Reporting console.

 

Using a dynamicProperty

To get extra properties from the user, have the user set a value for the document property that you defined as an SDK dynamicProperty in the connector descriptor file like this:

 

<dynamicProperty type="string" id="MessageId" label="Message Id"/>

 

The dynamicProperty element supports these attributes:

  • @id — defines the name of the field that is passed as the property key to the connector.
  • @label — is an optional text string that is displayed in the GUI for this field. If this attribute is not defined, the id is used.
  • @type — is the type of property value that is passed to the connector. This attribute also restricts the values that can be entered in the GUI to: “string”, “password”, “boolean”, and “integer”.

 

From the code, you can get that property’s value using the getDynamicProperties() method of a TrackedData instance like this:

 

public static String getMessageId( TrackedData data )
{
    Map<String, String> dynamicProperties = data.getDynamicProperties();
    String propertyValue = dynamicProperties.get("MessageId");
    return propertyValue;
}

 

This new property will now show up in the list of standard connector document properties you can choose from the Set Properties shape > Properties to Set section as shown in the following screenshot:

 

 

The user can also set the new property from the Set Document Property map function and from a Groovy script within a custom scripting step.

Again, it’s important to not confuse getDynamicProperties with the platform’s Dynamic Document Properties.

 

Using a trackedProperty

Sometimes in addition to the content of the document returned by a connector, you may want to provide metadata values that your users can track or use in a process. To do that, define an SDK tracked property in the connector descriptor file like this:

 

<trackedProperty id="MessageId" label="Message Id"/>

 

The trackedProperty element supports these attributes:

  • @id — defines the name of the field that is passed as the property key to the connector.
  • @label — is an optional text string that is displayed in the GUI for this field. If this attribute is not defined, the id is used.

 

In the code we set the tracked properties values as metadata to the payload we return for each document like this:

 

public static void sendResponse( TrackedData data, 
                                 OperationResponse response,
                                 String output, String messageId )
{
    PayloadMetadata metadata = response.createMetadata();
    metadata.setTrackedProperty("MessageId", messageId);

    response.addResult(data, OperationStatus.SUCCESS, "0", "OK",
                        PayloadUtil.toPayload(output, metadata));
}

 

This new property will now show up in the list of standard connector document properties you can choose from the parameter value list for most process steps, as shown in the following screenshot:

 

 

 

 

The user can also get the new property from the Get Document Property map function or from a Groovy script within a custom scripting step. These values also show up in Process Reporting like any other tracked property.

 

Using both a dynamicProperty and a trackedProperty for the same value

If you want a value to be able to be set dynamically AND tracked for Process Reporting purposes, simply define both a dynamicProperty and a trackedProperty in the descriptor file. This effectively allows the user to see a single document property that can be set before an operation and read after the connector's execution.

 

Using a userDefinedProperty

Again as a best practice you should defined standard properties (i.e. dynamicProperty and trackedProperty) for your connector instead of relying on less-obvious user defined properties to make your connector easier for end users.

 

To get the dynamic document properties that a user set on a document, you don’t need to define anything on the connector. Instead, just get those property values directly using the getUserDefinedProperties() method of a TrackedData instance like this:

 

public static String getUserDocumentProperty( TrackedData data, 
                                              String propertyName )
{
    Map<String, String> userProperties = data.getUserDefinedProperties();
    String propertyValue = userProperties.get(propertyName);
    return propertyValue;
}

 

That covers document properties from an SDK developer perspective. You can read more about dynamic document properties and dynamic process properties and the difference between them.

 

Common scenarios for using document properties

In most cases a connector can handle all the operation inputs and outputs via the document profile, but there are some scenarios where you need extra properties. Following are common use cases where we need document properties.

 

Adding properties to a raw binary document

A common use case is when the document is binary data, and we need to attach some properties to it. Using document properties we can let the user set and get properties that are attached to the binary data document that the connector can use.

 

An example in AtomSphere is the Disk connector that reads and saves files from the file system, and the document data is the binary content of the file. The connector has a File Name document property that the user can set to define the name the file will have when the connector saves it, or the name of the file the connector read from the file system.

 

Another example is the Microsoft Azure Service Bus connector that sends and receives binary messages from/to a queue. The connector has a Message Id document property (both a dynamicProperty and a trackedProperty) so the user can set or get the unique identifier for a message. It also defines a Content Type, and Label read/write properties, and a Sequence Number and Lock Token read only properties (SDK tracked property). You can find more info on the document properties this connector supports in the connector's operation help page.

 

Allow values to be provided at run-time to override an operation component's field

Sometimes there are configuration values that are configured at the Connection or the Operation component but it may be useful to allow the user to override that per document. In this case generally we don't want to pollute the document profile with this value since is something that is not directly related to the document's content but to the document's processing. In this cases we can use document properties to attach this configuration to a document when in need to override one of the values and make the connector use them accordingly when setting up the values internally.

 

Display values in Process Reporting

If you want to have certain values display in Process Reporting as part of document tracking you can use trackedProperty document properties.

 

Reading custom properties

Another common use case is when the connector needs to support some custom properties that we can't define at the moment of the profile import or creation. This is one scenario in which using userDefinedProperty values is appropriate.

 

An example of this is the Google Pub Sub connector, that sends and receives binary messages from/to a queue, and allows the user to set any metadata to be sent along with the document set as dynamic document properties (a.k.a. SDK user-defined properties).

 

The ability to define dynamic and tracked properties can provide a lot of flexibility and convenience so consider using them in your next connector!

Attachments

    Outcomes