Molecule secondary node on linux does not start - Failed while trying to acquire single instance atom lock

Document created by mike_aronson Employee on Jun 10, 2014
Version 1Show Document
  • View in full screen mode

A molecule is installed with nodes on two hosts.  For example, primary node is installed on host with ip address 10.1.1.1 and runs fine.  Secondary node is installed on host with ip address 10.1.1.2 but will not start.

Messages similar to the following appear in the error.log file in the bin/ directory under the molecule installation folder:

     MON DD, YYYY HH:mm:SS com.boomi.container.config.ContainerConfig <init>
     INFO: Container base directory is /<MOLECULE_INSTALLATION_DIRECTORY>/MOLECULE_NAME/
     Could not acquire process lock, retrying...
     Jun 10, 2014 2:02:53 PM com.boomi.container.core.ContainerFileLock logError
     SEVERE: Could not acquire process lock, retrying...
     Failed while trying to acquire single instance atom lock, /<MOLECULE_INSTALLATION_DIRECTORY>/MOLECULE_NAME/bin/atomLock.lck.10_1_1_1
     java.nio.channels.OverlappingFileLockException
     at com.boomi.container.core.ContainerFileLock.acquireProcessLock(ContainerFileLock.java:112)
     at com.boomi.container.core.ContainerFileLock.access$200(ContainerFileLock.java:26)
     at com.boomi.container.core.ContainerFileLock$2.execute(ContainerFileLock.java:84)
     at com.boomi.util.retry.Retryable.call(Retryable.java:90)
     at com.boomi.container.core.ContainerFileLock.areMultipleInstancesImpl(ContainerFileLock.java:80)
     at com.boomi.container.core.ContainerFileLock.areMultipleInstances(ContainerFileLock.java:61)
     at com.boomi.container.core.BaseContainer.checkRunningInstance(BaseContainer.java:1023)
     at com.boomi.container.core.BaseContainer.start(BaseContainer.java:211)
     at com.boomi.container.core.BaseContainer.main(BaseContainer.java:1147)
     at com.boomi.container.core.Container.main(Container.java:264)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.boomi.launcher.Launcher.main(Launcher.java:196)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
     at com.install4j.runtime.Launcher.main(Unknown Source)
     Jun 10, 2014 2:03:29 PM com.boomi.container.core.ContainerFileLock logError
     SEVERE: Failed while trying to acquire single instance atom lock, /<MOLECULE_INSTALLATION_DIRECTORY>/MOLECULE_NAME/bin/atomLock.lck.10_1_1_1
     java.nio.channels.OverlappingFileLockException
     at com.boomi.container.core.ContainerFileLock.acquireProcessLock(ContainerFileLock.java:112)
     at com.boomi.container.core.ContainerFileLock.access$200(ContainerFileLock.java:26)
     at com.boomi.container.core.ContainerFileLock$2.execute(ContainerFileLock.java:84)
     at com.boomi.util.retry.Retryable.call(Retryable.java:90)
     at com.boomi.container.core.ContainerFileLock.areMultipleInstancesImpl(ContainerFileLock.java:80)
     at com.boomi.container.core.ContainerFileLock.areMultipleInstances(ContainerFileLock.java:61)
     at com.boomi.container.core.BaseContainer.checkRunningInstance(BaseContainer.java:1023)
     at com.boomi.container.core.BaseContainer.start(BaseContainer.java:211)
     at com.boomi.container.core.BaseContainer.main(BaseContainer.java:1147)
     at com.boomi.container.core.Container.main(Container.java:264)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.boomi.launcher.Launcher.main(Launcher.java:196)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
     at com.install4j.runtime.Launcher.main(Unknown Source)
     Jun 10, 2014 2:03:29 PM com.boomi.container.core.BaseContainer$1 uncaughtException
     SEVERE: Unhandled exception in thread main.
     java.lang.IllegalStateException: Atom instance already running.
     at com.boomi.container.core.BaseContainer.checkRunningInstance(BaseContainer.java:1024)
     at com.boomi.container.core.BaseContainer.start(BaseContainer.java:211)
     at com.boomi.container.core.BaseContainer.main(BaseContainer.java:1147)
     at com.boomi.container.core.Container.main(Container.java:264)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.boomi.launcher.Launcher.main(Launcher.java:196)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
     at com.install4j.runtime.Launcher.main(Unknown Source)
     java.lang.reflect.InvocationTargetException
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.boomi.launcher.Launcher.main(Launcher.java:196)
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
     at com.install4j.runtime.Launcher.main(Unknown Source)
     Caused by: java.lang.IllegalStateException: Atom instance already running.
     at com.boomi.container.core.BaseContainer.checkRunningInstance(BaseContainer.java:1024)
     at com.boomi.container.core.BaseContainer.start(BaseContainer.java:211)
     at com.boomi.container.core.BaseContainer.main(BaseContainer.java:1147)
     at com.boomi.container.core.Container.main(Container.java:264)
     ... 11 more
      
The error.log file shows a conflict in that the second node already saw an atom instance was running on node with ip address 10.1.1.1, but it was the node with ip address 10.1.1.2.

It appears that the secondary node (10.1.1.2) was trying to start with the ip address of the primary node (10.1.1.1)

When starting, the molecule node will read the etc/hosts file on a linux system.

Check the etc/hosts file on the host for the secondary node and confirm that the IP address is configured correctly.
In the past we have observed that this error can occur if the IP address on the second is configured incorrectly whereby the etc/hosts file contains the ip address of the primary node instead (perhaps due to a copy or clone of the file).

Try correcting the entry in the etc/hosts file to resolve the issue.

If the IP addresses are configured correctly in the etc/hosts file and it still does not work, double check that the permissions on the folders/files are correct accessible for read/write/execute by the atom user.
1 person found this helpful

Attachments

    Outcomes