Overview

The document cache (doc cache or just cache) is a way to store documents retrieved in a process so you can use it later. Many use it to correlate different types of data to one another or to pull a value from another document based on the value in the current document. Documents are kept only during the process (in memory) and they do not carry on in the cache into later executions. There are two ways to add to the document cache:

Implementing the Add to Cache Step.
By using the cache option in most connector operation tabs, you can directly store documents retrieved from a connector. This is similar to adding an Add to Cache step immediately after the connector. Exercise caution to ensure that the data and document cache profiles are compatible, and handle any errors effectively, as discrepancies can lead to operational issues.

The doc cache is used in two distinct ways in the platform:

Storing documents to look up information inside of the document based on some value(s) that is already present but coming from a different source later on. For common scenarios, review Document Cache Best Practices and Common Scenarios
1. A profile is used so you can pick out elements as keys and values
2. A profile element key is defined
3. If you want to use a document property, it has to make sense to use it i.e.there's a different value associated with each document, otherwise stick to profile elements
Storing documents to be used as attachments in very specific connectors like the Web Services SOAP MIME attachments or HTTP connector
1. The profile type is None since it's dealing with attachments, not data types
2. A document property key is used to keep track of documents/attachments

Document cache data flow

Here's a typical, simple example of a document cache being used:
User-added image

In practical integrations, storing multiple documents is common, and a connector or alternative step usually replaces the message step to supply documents to the document cache. The second branch would also be data from another source and could be an entirely different data profile.

I've added the following documents to the cache:

<Header>
  <GenerationDateTime>20190417 133732.567</GenerationDateTime>
  <DocumentID>ATP_300-4600001987-0001</DocumentID>
  <IssueDateTime>2018-12-11 00:41:11.0</IssueDateTime>
  <Name>Billy</Name>
  <Age>27</Age>
</Header> 
--- 
<Header>
  <GenerationDateTime>20170417 133732.567</GenerationDateTime>
  <DocumentID>ADP_300-4600001987-0006</DocumentID>
  <IssueDateTime>2008-12-11 00:41:11.0</IssueDateTime>
  <Name>Paul</Name>
  <Age>30</Age>
</Header>

It's an XML document. I've configured my cache for that XML document and the elements in it. The doc cache is configured for one index (how data is categorized and retrieved later, it is like a filter or “where” clause) with one key (what value(s) to associate with a document going into the cache). In this case my key is the Name element (Billy and Paul). In the retrieve from cache step, I pick my cache and which index to use (you can have multiple indexes i.e. different ways to pull data. Maybe I want to pull data using Age in other scenarios). Then I pick how to define the value that is going to be used as the key, in this case I am extracting a element from a flat file called FirstName. Here's the full flow for the example above:

Some number of documents are retrieved
They are stored in a cache (Note: there is no more data flow after this step)
Later in another branch or map or somewhere else, I want to retrieve the document(s) stored in the cache
I have separate data flowing through the path, e.g. a flat file with a FirstName element and I want to pull a document out of the cache that relates to this flat file
In the flat file I have the FirstName element and it equals "Billy" (the cache is case sensitive) so the retrieve from cache step is going to use "Billy" as the value for the key Name since that was how I defined it earlier
The retrieve from cache step is going to look into the cache and see there are two documents, it will look at each key (which is the XML element Name) and see if there is a "Billy"
The cache sees there is a document with the key equal to "Billy", retrieves that and sends it out of the retrieve from cache step. This document will now be sent down the path to future step

Some important notes about caches:

You can retrieve more than one document from the cache based on the same key
The cache can be shared between a child and parent process. You can write to the cache in a parent process and it will still be available for a child process to use or vice versa
You should have data coming from another place that has some same value as the key element in the documents in the doc cache
You can mimic a key for testing by having a message step with the key value and setting the parameter value to be current data so the retrieve from cache picks it up and uses it
You can add another index to use a different key but you can only use one index at a time
Using multiple keys in the same index works exactly the same way as one key. The cache will look up the document's key element and see if all the keys have such a value
If nothing matches, the retrieve from cache returns nothing and your data flow stops
The cache does not filter out duplicates, you'll need to do it beforehand. You can use the enforce unique options in a flat file and mapping or another strategy to remove duplicates
Document properties stay with the document when it is cached and retrieved

This will be basically how the doc cache works everywhere else as well. You need to supply actual data (some data has to go into the retrieve from cache step) to retrieve data. Other places you can use the document cache:

Map's Document Cache Lookup function, give key values to retrieve as many elements from the document as you want
To the left side profile itself in the map, on one of the high level nodes (different node for each profile) you can click on the triangle to use the option "Add Cached Data", this will allow you to retrieve all of the document's elements using some key value that comes from the incoming profile. You can use the elements just like a regular profile element
Setting property values to use in naming or decision steps, wherever you can specific a parameter value

Enforce one index entry per document

If you have this option checked in the doc cache, this allows you to:

Retrieve all documents from the doc cache
Use the remove from cache step to remove specific documents

The enforce one index entry per document will make sure inside a document data itself there is only one element that corresponds with the key. If you try to add this to the cache with the option checked (notice there's multiple Name elements):

<Header>
  <GenerationDateTime>20190417 133732.567</GenerationDateTime>
  <DocumentID>ATP_300-4600001987-0001</DocumentID>
  <IssueDateTime>2018-12-11 00:41:11.0</IssueDateTime>
  <Name>Billy</Name>
  <Name>Bob</Name>
  <Age>24</Age>
</Header>

the cache will throw an error: "The Document Cache setting allows only one entry per document in a given index. Multiple document entries were produced for index: index1" because you have multiple keys in that document and the cache doesn't know which one to use.

If the option is unchecked, you can now add the document above into the cache and the cache will make one index entry for "Bob" and another index entry for "Billy". If you use "Bob" as the key value later, the cache will retrieve that document or if you use "Billy" as the key value later, the cache will also retrieve that same document. That one document has two (multiple) index entries for it now.

Optimizing Cache Management: The Procedure for Removing Documents

This allows you to remove either all documents currently in the cache or specific ones. The remove from cache step stops the data flow. If you want to remove specific documents, you need to set the key value going into the step like how you would do it for the retrieve from cache step.

Adding cache data to a the map

The source profile can pull a document from the document cache also. On one of the high level nodes (different node for each profile) you can click on the triangle to use the option "Add Cached Data", this will allow you to retrieve all of the document's elements using some key value that comes from the incoming data. You can use the elements just like a regular profile element. Note that in order to use this you need to make sure that only one document will match the key to pull the document from the cache. Having more than one match will cause the map step to error.

User-added image

FAQ and Common Issues

Do I need to split my document up before it goes into the cache?

Namely flat file and database profile need to be split up before going into the cache. Documents with multiple data rows that should recognized as separate need to be split (with a data process split) up before going into the cache or it'll be treated as one document.

Can I replace existing records in a document cache?

There's no automatic update records with the same keys or index functionality currently. You could look up a document(s) in the cache by key and remove just that one and then add in an updated document with the same key.

Why is no data being returned from the document cache?

This usually means either you do not have data in the cache or your keys do not match anything stored in the cache. Here's a couple of things to look out for:

Confirm the data going into the cache is individual documents e.g. like flat-files should be split into individual rows so each row can be indexed
The key actually has a value and it's pointing to the right profile element
There should be data coming into the Retrieve from Cache step, otherwise there won't be document flow and nothing will be retrieved
The data going to the Retrieve from Cache step has a value that corresponds with the key value of the documents stored in the document cache

Found more than 1 document in the document cache

The full error should include a little more information to let you know which index and which key had multiple results such as, "Found more than 1 document in the document cache (index: index1, keys: [Name = Billy])". You should only get this error during a Document Cache Lookup in parameters or in the map function. This is because the input has found multiple documents that match the key and it can't return the one value that is expected for the lookup. Adding cache data in the map or the document cache steps should not produce this error since they can pull multiple documents that match the key. A couple of things to look for:

Double check your input data is unique and see if any produced the same key. If your data is not unique and you want to use document cache lookup, you'll have to add more keys or update your data so it is unique
Use the add to map option or document cache steps if you meant to instead of the document cache lookup
You can also try placing a Decision step before the Document Cache step and only proceed to placing the data in the document cache if it does not already exist

Why is retrieve all documents from the cache not retrieving all documents?

This usually happens if your profile used for the document cache has instance identifiers. Instance identifiers will set aside data that matches the instance definition so that data won't be used. Make a separate profile for caching (have no instance identifiers) and mapping (which can include instance identifiers). If this is not the case, double check what data actually went into the cache or it might be a defect.

Does the data get mixed up if the same document cache component is used in multiple processes?

No, each process execution will have its own instance of the document cache. The document cache only lives for the length of an execution and does not persist across the process executions. The process call step does not reset the cache. Calling on multiple subprocesses which use the same document cache component will load the cache with a mix of data from the different subprocesses. Always be aware of which document cache you are using in a subprocess.

Why document cache retrieves document in a wrong sequence ?

The document Cache makes no promise about keeping the order of the items. Customers can use custom script or make use of a secondary key to retrieve the items they would like in certain order. This Oracle doc for Java HashMap: https://docs.oracle.com/javase/9/docs/api/java/util/HashMap.html explains further about HashMap and how it works.

How the Document Cache Step Works