large dnc_* files are filling up the system tmp/ or temp/ directory

Document created by mike_aronson Employee on Nov 15, 2013
Version 1Show Document
  • View in full screen mode
Executing process(es) results in large folders or files getting created in the system \tmp or \temp folder with the following naming convention:

dnc_*
 
These folders/files are filling up this directory and causing the system to experience performance / disk space issues.

To address the issue with the system temp directory filling up, here are a few suggestions that may help:

1. Change the location of the Atom's JVM temporary directory.

This is a common situation for users running Atoms on Windows servers built with a "lean" C: drive with minimal disk storage for the operating system but have the Atom and other applications installed on a separate D: drive with more resources. However by default the Atom's JVM will use the C: drive for temporary files such as the dnc files, which can quickly consume disk space.

To change the directory, edit the ../<Atom installation directory>/bin/atom.vmoptions file and add the following line:

-Djava.io.tmpdir=<tmp_directory>

We recommend setting this to the Atom's local temp directory, ../<Atom installation directory>/tmp

After making this change, restart the atom/molecule/cloud. If you are unable to restart the Atom, restart the server.


2. Add more disk space to the C: drive.

This should alleviate the issue while allowing the processes to execute efficiently.


3. Increase the time between scheduled executions.

This will reduce the number of concurrent executions that may be contributing to the creation of dnc files.


4. Avoid known XML profile misconfiguration.

While there is no direct correlation between the dnc file and process execution, you may be able to determine which process is causing the largest dnc_* file(s) to be written by reviewing the timestamps on the large dnc file(s) and associate them with the process timestamp in Process Reporting. If a particular process or processes can be identified, additional investigation can be performed to see if improved efficiencies can be made to the process.

One thing to look for is a situation involving incorrect XML profile configuration that has been observed to contribute to the creation of very large dnc files. The issue is when the profile has elements explicitly replicated for a "repeating" set of data, like this:

0EM40000000PTZn



Note the "Item" and children elements are explicitly defined twice. This incorrect configuration is typically a result of either manually modifying the profile with the intent of being able to map the first item differently than the second item, or more commonly by importing a sample XML file with multiple items. The correct configuration is to have Item defined only once with Max Occurs=Unbounded. To map specific occurrences independently, instance identifiers should be used. After importing a sample XML, any repeated element definitions should be removed.

5. Increase the Atom's memory.

Follow the instructions here to increase the memory allocated to the Atom's JVM.


Additional Technical Information

The creation of the dnc folders/files is normal behavior of the Atom and are used to temporarily buffer the parsing of document data while performing actions such as mapping and profile element extraction. Most document parsing is performed in memory for performance however dnc files are leveraged when parsing very large documents or when the Atom enters a "low memory" state. Note that a "low memory" state may not be directly related to the size of the documents in a given process execution, such as approaching the number of concurrent executions for the Atom's current memory settings.

While the use of dnc files does NOT necessarily indicate a problem with the Atom's execution load or configuration, if you see them consistently during execution it may be an indication to investigate whether additional memory should be allocated to the Atom JVM.

These files are temporary and are removed automatically by the Atom after the process or process step completes. However if the files are being written to the system temp directories and the Atom unexpectedly crashes for some reason these files will not be cleaned up automatically and can be safely purged manually when the Atom is not running (not avoid accidentally removing a file that is actively being used). Alternatively we recommend setting the java.io.tmpdir to the Atom's local tmp directory, which will then be purged by the normal Atom file purge schedule.


 

Attachments

    Outcomes