As part of our custom connector, create operations hit our servers to create a session before sending batches of data - this session is shared across batches. Maintaining this session context across batches is important when processing the data on our servers.
The problem arises when a Flow Control shape is used prior to our connector shape in a process (with batches enabled). This is sometimes necessary, as potentially large amounts of data (1Gb +) will be passed in by our customers. With batches happening on the process level, each time a batch hits our connector it creates a new session. So, instead of one session for the whole process run we get X sessions, where X = the number of batches created by the Flow Control shape.
Is there a way from our connector to read/write process level properties? We would like to store a session Id on the process level so that a whole process run would be associated with one session. This session would get generated the first time our create operation is called, and then grab the stored value for all subsequent calls.
If there is another approach you would recommend please let me know - thanks!