AnsweredAssumed Answered

We are currently using a queue to handle errors coming out of our batch processes. We tried to limit but were unsuccessful at limiting the number of process spawned.

Question asked by jtyson@targaresources.com on Oct 8, 2018
Latest reply on Oct 8, 2018 by trey.yeager826393

We created a queue file, which allows batch jobs to queue up their errors as they run. We then created a listener that when called, reads the queue, and writes to a database. We did this to give our batch processes a buffer to keep from being stopped by any issues in writing the errors to the error database. Unfortunately, despite having a limit of 20 spawns set on the operation, it spawns up potentially thousands of jobs, using all the resources on the atom, and then causes anything else to crash until such time as it has handled the errors and released the resources.

 

Attached are visuals of both the process, and the operation.

 

Any ideas would be greatly appreciated.

Outcomes