opengrok-indexer generates ClassNotFoundException on org.osgi.framework.BundleReference when using the URLClassLoader.findClass

[IamTHEvilONE]

When running an index task using opengrok-indexer, there is a failure to find some class for the URLClassLoader.

opengrok-indexer -J=-verbose:class -J=-Djava.util.logging.config.file=/usr/opengrok/doc/logging.properties
-J=-XX:+HeapDumpOnOutOfMemoryError
-J=-XX:HeapDumpPath=/usr/opengrok/heap_dumps
-a /usr/opengrok/lib/opengrok.jar --
-c /usr/local/bin/ctags
-s /usr/opengrok/src -d /usr/opengrok/data -H -P -S -G
-W /usr/opengrok/etc/configuration.xml -U http://localhost:8080/source/

The error is:

2019-06-20 17:02:28.349-0400 FINER t1 ReflectionHelper$2.run: Unable to load class org.osgi.framework.BundleReference using the current class loader.
java.lang.ClassNotFoundException: org.osgi.framework.BundleReference
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.glassfish.jersey.internal.util.ReflectionHelper$2.run(ReflectionHelper.java:268)
at org.glassfish.jersey.internal.util.ReflectionHelper$2.run(ReflectionHelper.java:253)
at java.security.AccessController.doPrivileged(Native Method)
at org.glassfish.jersey.internal.util.ReflectionHelper.(ReflectionHelper.java:1494)
at org.glassfish.jersey.internal.ServiceFinder.(ServiceFinder.java:166)
at org.glassfish.jersey.client.ClientConfig$State.(ClientConfig.java:162)
at org.glassfish.jersey.client.ClientConfig.(ClientConfig.java:515)
at org.glassfish.jersey.client.JerseyClientBuilder.(JerseyClientBuilder.java:100)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at javax.ws.rs.client.FactoryFinder.newInstance(FactoryFinder.java:114)
at javax.ws.rs.client.FactoryFinder.find(FactoryFinder.java:209)
at javax.ws.rs.client.ClientBuilder.newBuilder(ClientBuilder.java:94)
at javax.ws.rs.client.ClientBuilder.newClient(ClientBuilder.java:121)
at org.opengrok.indexer.index.IndexerUtil.enableProjects(IndexerUtil.java:53)
at org.opengrok.indexer.index.Indexer.main(Indexer.java:308)

Last line of code from opengrok is:

opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/index/IndexerUtil.java

Line 53 in cb82f66

final Invocation.Builder request = ClientBuilder.newClient()

After running the opengrok-indexer in verbose mode, I don't see any reference to loading org.osgi code.

URLClassLoader comes from here:

[Loaded java.net.URLClassLoader$3$1 from /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.212.b04-0.el7_6.x86_64/jre/lib/rt.jar]

I haven't seen anything in the issues (open/closed) that was recent, and other sources reference that it could be the jar running outside the IDE can frequently generate this. Not sure if it actually matters or where to go from here.

Environment Setup

OS: CentOS Linux release 7.6.1810 (Core)
Kernel: 3.10.0-957.21.2.el7.x86_64
OpenGrok: 1.2.16
opengrok-tools 1.2.16
Java: 1.8.0.212.b04-0.el7_6.x86_64

[vladak]

I wonder what makes Jersey demand org.osgi.framework.BundleReference cause I've not seen that exception before.

[vladak]

This might be caused by some dependency mismatch. Jersey docs on https://jersey.github.io/documentation/latest/deployment.html#osgi.httpservice say:

Make sure to replace with an appropriate version number. Which one is appropriate depends on the specific GlassFish 4.x version you are using. The version of the bundle cannot be higher than the version of Jersey integrated in your GlassFish 4.x server. Jersey bundles declare dependencies on other bundles at the OSGi level and those dependencies are version-sensitive. If you use example bundle from let's say version 2.5, but Glassfish has Jersey 2.3.1, dependencies will not be satisfied and bundle will not start. If this happens, the error will look something like this:
1234567891011 | gogo$ lb...303 | Installed  |    1| jersey-examples-osgi-http-service-bundle (2.5.0.SNAPSHOT)gogo$ start 303 org.osgi.framework.BundleException: Unresolved constraint in bundle

[vladak]

#2211 might be related.

[carbonrobin]

I'm finding this error and #2211 also.

Is this related to trying to use opengrok-projadm to add or delete a project and getting a message "could not delete project PROJECT in web application on http://localhost:8080/source"? The command is still able to delete the source code from /opengrok/src.

It seems all my deleted test projects are still listed in the opengrok/data/index directory. Even if I delete the opengrok/data directory and fully reindex the old, non-existent projects, their indexes come back. Restarting tomcat doesn't seem to change the results.

On the main home page for the opengrok app, I see one project listed under Project and the other five projects listed under Repositories.

[vladak]

The problem reported here is specific to the indexer. To bridge a gap between old version and new version the indexer sends a RESTful API request to the web app to enable projects in the configuration. This fails because of the above exception so it looks like the request is not delivered.

That said, keep in mind that once you modify the configuration in the running web app, you need to retrieve it and store in configuration.xml so that it is persistent. This is needed when using opengrok-projadm. There is a certain workflow required when using the project centric approach. Basically, index everything first and then avoid using project/repository discovery - assuming the source root still has everything.