How can I implement data caching between two different processes in Boomi?
It's not feasible. Cache is only in one process. You have to store as a file in one process and read from another process.
It is bad news. Disc read/write it is not a fast operation
It is relatively fast to use local disk (store in ./work dir). It is a lot faster than SFTP and web service calls!
You could use "Load from cache" and write all to disk just before process 1 ends, and then read from disk and use "Add to Cache" at the start of process 2.
All operation with disk slowly than in memory operation with in-memory cache
Maybe I don't understand why you are using 2 processes? I would think it is because 1 process does something, then process 2 takes over later?
How about just have one process call the other when it is done process (using a simple process call). In that way they can both use the same database cache?
Or are you talking about 2 invocations of the same process where it should persist the cache between runs?
Even though local disk is slower than memory, it is still many times faster than a web service call or a database query... did you try it?
I talking about both this examples.
For example case:
We have data with a long time of life such us Marketing offers. They are using for frontend(REST API) and stored in Salesforce. We would like to store this data in Boomi cache(for access speed) and update for example once time in month from Salesforce.
About storing data to disk. I dont want to control data on disk, i want give control data right to the cache provider such us coherence,redis(or boomi if it is posible) etc.
So you have a process that listens for incoming requests, and instead of having to forward queries to Salesforce, you want the data to be readily available in Boomi? And have a 2nd process refresh the cache?
Since Boomi does not have a persistent cache that would require a Boomi process that ran in an endless loop somehow. Not something I would recommend.
what is your use case?
The problem with integrating this directly in boomi is that the atom/molecule/cloud has many different modes of operation which might make an integrated cache less than useful. with multiple nodes in a molecule/cloud and separate jvms for cloud executions and atom workers, you could end up with scenarios where you are continuously repopulating your cache (and thus, not really gaining any benefits from the cache).
caches are really a commodity, and there a pile of great free cache servers out there with a variety of different deployment models (including fully cloud-based impls like aws elasticache). it would be much better for your infrastructure to setup a cache tuned to your needs than trying to wedge something into boomi.
That said, while it may not make sense to implement the cache within boomi, it might make sense to make caching "easy to use" from within boomi. e.g. if you could select an "external cache component" on a connector shape and boomi would automagically interact with an external cache, that could be really powerful. boomi could have a "plugin" layer with implementations that support various external cache implementations.
This would be highly beneficial for database queries as well. We have a parameter service which other processes invoke (via HTTP Client) to lookup parameters stored in a database. These values do not change often. By not caching (either the service results or the database connector call), we must connect to the database for every single request. This puts unnecessary load on the database for something which could be stored in memory on the process.
Retrieving data ...