Error storing data, file not found (No space left on device)

Document created by Nikita_nandakumar on Dec 29, 2014Last modified by Adam Arrowsmith on Sep 7, 2017
Version 8Show Document
  • View in full screen mode

Issue

A process fails with one of the errors below:

 

  • "Severe errors occurred during start shape execution, terminating process. (com.boomi.process.ProcessException) Caused by: Unable to create new index segment. ((com.boomi.store.DataStoreException)) Caused by: Error storing data, file not found. ((com.boomi.store.DataStoreException)) Caused by: ../<ATOM_INSTALL>/data/YYYY.MM.DD/###########.dat.pmeta.0 (No space left on device) ((java.io.FileNotFoundException))"
  • "Unable to create new index segment.; Caused by: Error storing data file not found.; Caused by: ../<ATOM_INSTALL>/data/YYYY.MM.DD/#########.dat.0 (Too many open files).
  • Error storing data, file not found.; Caused by: ../<ATOM_INSTALL>/data/YYYY.MM.DD/###########.dat.pmeta (No space left on device)

 

Cause

There can be several causes:

  • This error may indicate an issue reading or writing a file on the file system.
  • 'No space left on device' indicates that the server is running out of disk space or "Too many open files" may indicate that there are not enough file handles left.
  • Due to very high processing volumes, the number of files on disk or within a single directory (e.g. ../data and ../execution) are reaching the limits allowed by the OS.

 

Solution

For Linux, you may need to increase the nofile limit from the default (usually 1024) to something higher. For example: ulimit - What is the max opened files limitation on Linux? - Stack Overflow. Please check with your system admin regarding the utilization of the disk storage and the OS level settings for file limits. See also Sizing Your File System Disk Space and Inode Limit in Linux.

 

To avoid exceeding the maximum number of a files in a single directory, set the Atom Data Directory Level and Process Execution Directory Level properties to separate the execution artifacts files into a series of subdirectories. As a starting point, set the values to 2. Note that setting this too high can have an adverse effect by slowing down disk reads as it traverses the hierarchy.

5 people found this helpful

Attachments

    Outcomes