OPL and Java: loading dynamic Linux libraries

When calling IBM ILOG OPL (Optimization Programming Language) from a Java application running on Linux, one will face some issues regarding loading dynamic OPL libraries. Typical error messages look like:

Native code library failed to load: ensure the appropriate library (oplXXX.dll/.so) is in your path.

java.lang.UnsatisfiedLinkError: no opl63 in java.library.path
java.lang.UnsatisfiedLinkError: no opl_lang_wrap_cpp in java.library.path
java.lang.UnsatisfiedLinkError: no cp_wrap_cpp_java63 in java.library.path
java.lang.UnsatisfiedLinkError: no concert_wrap_cpp_java63 in java.library.path

This article explains my considerations and some approaches how to fix it.

According to the OPL Java Interface documentation, granting access to the OPL should be as simple as:

this.oplFactory = new IloOplFactory();
this.errorHandler = oplFactory.createOplErrorHandler();
this.settings = oplFactory.createOplSettings(this.errorHandler);
...

However, at the first time Java reaches a reference to any class that provides OPL, it will try to load all C-compiled dynamic libraries that implement the OPL interface. Under linux, this library is called oplXXX.so (where XXX is the OPL version, eg. 63 for 6.3) and usually found as a file ./bin/YYY/liboplXXX.so from the OPL installation directory (where YYY is the name of your operating system and machine architecture).

The easiest way to assure that Java finds the OPL library is passing its path on the java command line with the -Djava.library.path JVM parameter:

java -Djava.library.path=/opt/ilog/opl63/bin/x86-64_debian4.0_4.1 -jar OptApplication.jar

On other ILOG products, I used to write code that forces loading the library to avoid requiring the user to care about the -Djava.library.path JVM parameter.

try { // (does not work)
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libopl63.so");
} catch (UnsatisfiedLinkError e) {
    throw new OplLibraryNotFoundException(e);
}

Unfortunately, there is a hidden trap: the oplXXX.so itself has binary dependencies to other ILOG libraries. Both approaches (System.load and JVM parameter) will fail with an error message like:

java.lang.UnsatisfiedLinkError: /opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libopl63.so: libdbkernel.so: cannot open shared object file: No such file or directory
java.lang.UnsatisfiedLinkError: no opl_lang_wrap_cpp in java.library.path
java.lang.UnsatisfiedLinkError: no cp_wrap_cpp_java63 in java.library.path
java.lang.UnsatisfiedLinkError: no concert_wrap_cpp_java63 in java.library.path

All required dependecies are, according to ldd:

libdbkernel.so, libdblnkdyn.so, libilog.so, libcplex121.so

One solution would be to load all the libraries in reverse order before referencing any OPL class:

try { // (does not work)
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libilog.so");
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libdbkernel.so");
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libdblnkdyn.so");
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libcplex121.so"); 
    System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libopl63.so");
} catch (UnsatisfiedLinkError e) {
    throw new OplLibraryNotFoundException(e);
}

Unfortunately, not all binary library dependencies conform to JNI and it is not possible to force pre-loading them.

It happens that the JVM, in order to load libopl63.so, passes control to LD GNU Linker, which is in charge to load libopl63.so and all its dependencies. The LD is a component of  Linux and runs under the scope of the operating system. It is completely unaware of the JVM that called it. Therefore, it has no knowledge of the JVM configuration nor class loading policies. It will not look within paths listed by the -Djava.library.path JVM parameter. Instead, it was programmed to look for paths listed in LD_LIBRARY_PATH.

I agree that this is really odd. I checked thoroughly reference manuals/documentation and talked to experienced Linux system administrators. There is really nothing one can do with Java coding or configuring to fix this issue. The only solution is configuring the LD_LIBRARY_PATH environment variable to instruct LD where to locate additional OPL libraries. In order to call ones application, a redundant command line is required as:

LD_LIBRARY_PATH=/opt/ilog/opl63/bin/x86-64_debian4.0_4.1 java -Djava.library.path=/opt/ilog/opl63/bin/x86-64_debian4.0_4.1 -jar OptApplication.jar

Even worse, one needs to set LD_LIBRARY_PATH on each Java invocation. Editing bash_profile.sh or .bashrc is of little use, since most setuid’ tools (as bash or gdm that starts your graphical interface) do reset LD_LIBRARY_PATH for security reasons. And you practically all log-in access relies on a setuid’ application, LD_LIBRARY_PATH will always be reseted.

Daniel Felix Ferber

System.load("/opt/ilog/opl63/bin/x86-64_debian4.0_4.1/libopl63.so");

2 Responses to OPL and Java: loading dynamic Linux libraries

  1. Anderas says:

    Thanks for sharing your solution!

    Instead of prepending the variable, I’ve added the variable to catalina.sh:
    export LD_LIBRARY_PATH=””

    • hs says:

      For anyone who is deploying app on WebSphere application server can create LD_LIBRARY_PATH under Application servers > server1 > Process definition > Environment Entries.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: