AnsweredAssumed Answered

Map and Memory Dumps

Question asked by RanjiniElamkulammana2391 on Nov 4, 2016
Latest reply on Nov 4, 2016 by britto.parthalomew

40,000 documents going into a Map is causing a memory dump.

20,000 documents gets through the same Map without a memory dump

 

So we thought, lets batch them and send them through the map.

 

batch by 1000, still causes memory dump.

 

Any thoughts?

 

What kind of a map?

**********************

1. A very complex map

2. 5 to 6 SQL queries

3. 3 to 4 groovy scripts

4. flat file to a DB profile

 

We checked usual suspects

*********************************

1. To see if there is any groovy code that is accumulating objects. we don't see that

2. we batched the incoming documents and doesn't seem to impact the memory dump

 

Further challenge

************************

1. This takes 45 minutes to produce the dump

2. So every run is a long process

 

Questions

*****************

1. What could be impacting this behavior?

2. what should we look for in the map?

Outcomes