Skip to content

Conversation

desruisseaux
Copy link
Contributor

When compiling a multi-release project in the old way, put previous versions on both the class-path and module-path for versions before the version where module-info.java is introduced.

…ersions

on both the class-path and module-path for versions before the version wbere
`module-info.java` is introduced.
@desruisseaux
Copy link
Contributor Author

The test case and the correction was provided by @fridrich. This is related to #945 in that those two pull requests aim to fix the compilation error demonstrated by the singleproject-modular test case added in this commit.

…ath and module-path.

Instead, ensure that the `--module-source-path` option is used for compiling modules.
@fridrich
Copy link
Contributor

There is still a problem in some projects. Once you reach the phase that the compilation is modular, you have to add ALL dependencies on --module-path instead of --class-path. If not, dependent modules are not found.

@desruisseaux
Copy link
Contributor Author

Do you mean that there is some non-modular dependencies that are on the class-path, and the code in those dependencies are no found?

@fridrich
Copy link
Contributor

Do you mean that there is some non-modular dependencies that are on the class-path, and the code in those dependencies are no found?

No, when you suddenly consider that whole thing as modular project because of the module-info.java, all its module dependencies must be on the --module-path. Currently, all the dependency artifact from the pom.xml are on the --class-path.

@desruisseaux
Copy link
Contributor Author

Is the following a correct description of the scenario?

  • A project has a dependency of thpe jar. It means that Maven decides itself if the dependency is put on the class-path or module-path. The rule is:
    • If the dependency has a module-info (or an automatic module name in MANIFEST.MF) and the project has a module-info, put on the module-path.
    • Otherwise, put on the class-path.
  • A multi-release project has no module-info in the base code. Therefore, the dependencies are put on the class-path.
  • When the code for a higher Java release is compiled and that code introduces a module-info, the project is now considered modular, therefore the decision made in previous point must be revisited.
    • You observed that it was not the case?

@fridrich
Copy link
Contributor

OK, just an example. Look for yourself. Check out https://github.com/eclipse-ee4j/yasson/releases/tag/3.0.4
Change the maven-compiler-plugin version to 4.0.0-beta-3-SNAPSHOT. Take your branch and make "mvn install" of the plugin. Then build with that plugin that tag of yasson.

You will have as a result:
[INFO] Compiling all files. [ERROR] module not found: jakarta.cdi [ERROR] module not found: jakarta.json.bind [ERROR] module not found: jakarta.json [INFO] Summary of compiler messages: 3 compiler.err.module.not.found Total: 3 errors [ERROR] COMPILATION ERROR: Cannot compile org.eclipse:yasson:jar:3.0.4 main classes. [INFO] For trying to compile from the command-line, use: javac @target/javac.args

@fridrich
Copy link
Contributor

the dependencies of that module will be on --class-path in target/javac.args. Change that to --module-path and the compilation will succeed.

@desruisseaux
Copy link
Contributor Author

Thanks for the link, I will test right now. What I'm trying to figure out is why the dependency is still on the class-path.

for `module-info` in previous versions of a multi-release project.
@desruisseaux
Copy link
Contributor Author

Pushed a fix to this pull request. It was indeed a bug in the plugin, which was not detecting that the project is modular when compiling for versions other than the base version, because the module-info file was not in the higher versions.

Note that this bug existed in the previous approach (putting everything on the class-path) as well, but it was unnoticed. I presume it was because putting everything on the class-path allowed javac to compile as a non-modular project. Being more strict about the class-path / module-path allowed us to identify this bug.

If there is other projects failing, please send me their links too.

@fridrich
Copy link
Contributor

Now, with this change, this whole thing is compiling all my test-set correctly. Besides one package, but that is due to the plexus-compiler-api -> javax.tools.JavaCompiler migration and not sure we can do anything useful there.

But I see one problem due to the incrementalCompilation. I will expose it in the next comment including the reproducing steps.

@fridrich
Copy link
Contributor

Ok, the reproducer that shows problems (that might be not in the scope of this PR) is following:
Checkout https://github.com/amaembo/streamex, modify the version of the maven-compiler-plugin to 4.0.0-beta-3-SNAPSHOT and compile the project with mvn package javadoc:aggregate which will trigger an additional compilation before the javadoc invocation and that compilation will fail.

@fridrich
Copy link
Contributor

But, this might not be really the problem of this particular PR. The changes in incremental compilation messed up quite a number of things.

@desruisseaux
Copy link
Contributor Author

Thank for the link. I have done a step by step debugging. The issue does not seem to come from the incremental compilation. It comes from the execution of the following plugin in the pom.xml:

      <plugin>
        <groupId>org.codehaus.mojo</groupId>
        <artifactId>build-helper-maven-plugin</artifactId>
        <version>3.0.0</version>
        <executions>
          <execution>
            <phase>test</phase>
            <goals>
              <goal>add-source</goal>
            </goals>
            <configuration>
              <!-- Necessary to add sources to the source-jar -->
              <sources>
                <source>src/main/java-mr/9</source>
                <source>src/main/java-mr/16</source>
              </sources>
            </configuration>
          </execution>
        </executions>
      </plugin>

Below is the list of steps executed by Maven with my analysis after the ⟵ symbol:

  • enforcer:3.0.0-M2:enforce (enforce-java)
  • jacoco:0.8.7:prepare-agent (default-prepare-agent)
  • resources:3.3.1:resources (default-resources)
  • compiler:4.0.0-beta-3-SNAPSHOT:compile (default-compile) ⟵ this plugin
  • compiler:4.0.0-beta-3-SNAPSHOT:compile (java9) ⟵ multi-release compilation
  • compiler:4.0.0-beta-3-SNAPSHOT:compile (java16)
  • resources:3.3.1:testResources (default-testResources)
  • compiler:4.0.0-beta-3-SNAPSHOT:testCompile (default-testCompile)
  • surefire:3.0.0-M3:test (default-test)
  • build-helper:3.0.0:add-source (default)Source code directories for Java 9 and 16 added here!
  • surefire:3.0.0-M3:test (java9-test) ⟵ not impacted by above addition (nothing has been recompiled yet)
  • surefire:3.0.0-M3:test (java8-test)
  • antrun:1.8:run (jacoco-report)
  • jar:2.4:jar (default-jar)
  • javadoc:3.0.1:jar (attach-javadocs) ⟵ Note: contains duplicated source classes for Java 8, 9 and 16.
  • source:3.0.1:jar-no-fork (attach-sources)
  • javadoc:3.0.1:aggregate (default-cli)
  • enforcer:3.0.0-M2:enforce (enforce-java) ⟵ seems to restart everything from the beginning
  • jacoco:0.8.7:prepare-agent (default-prepare-agent)
  • resources:3.3.1:resources (default-resources)
  • compiler:4.0.0-beta-3-SNAPSHOT:compile (default-compile)Maven Project has source directories for Java 8, 9 and 16!

Therefore, the plugin seems to work as instructed. The execution of build-helper:3.0.0:add-source adds the directories for Java 9 and 16 in the Maven Project. When compiler:4.0.0-beta-3-SNAPSHOT:compile is executed again in the last bullet, it asks for the source directories to Maven core. It is Maven core who send back the source directories for Java 8, 9 and 16, because of the execution of the build-helper plugin. This is independent of the compiler plugin.

The only impact of incremental build is that it correctly detects that all the files in the Java 8 directory were already compiler, and think that the files in the Java 9 and 16 directories are new. Therefore, the javac.args files contains only the latter, but this is correct.

If this compilation worked with Maven 3 plugin, we would need to run in a debugger in order to understand how it could be. But I suspect that it may be because of some fragile combination of circumstances.

@desruisseaux desruisseaux self-assigned this Jul 15, 2025
@desruisseaux desruisseaux added the bug Something isn't working label Jul 15, 2025
@fridrich
Copy link
Contributor

Personally I also think that this is not really maven-compiler-plugin problem. Sometimes you will have lower quality pom.xml files that somehow worked before and now they will not. But there is not a contract of being bug-to-bug compatible. I see from my point of view this PR as pretty mature to get in. Not like my opinion matters :)

@desruisseaux
Copy link
Contributor Author

Your opinion matter, you identified real bugs and your test cases helped a lot! Regarding the above issue, a workaround is to add the following block in the compiler plugin executions:

          <execution>
            <id>default-compile</id>
            <goals>
              <goal>compile</goal>
            </goals>
            <configuration>
              <release>8</release>
              <compileSourceRoots>
                <compileSourceRoot>${project.basedir}/src/main/java</compileSourceRoot>
              </compileSourceRoots>
            </configuration>
          </execution>

It causes the plugin to ignore the sources managed by Maven core and use the directory specified in <compileSourceRoot> instead. That way, the plugin is not impacted by the add-source execution.

However, when doing that, we hit another issue with the symbolic link created for redirecting javac output to the location expected by Maven 3. I will fix that now by creating the symbolic link elsewhere. (Note: these hacks exist only in order to mimic the behaviour of Maven 3. When using the new <sources> element, there is no such hacks, but the directory layout of the output is different).

…ic Maven 3 behaviour).

The previous approach using symbolic links caused the same files to appear in two locations,
which was another source of confusion.
@fridrich
Copy link
Contributor

Your opinion matter, you identified real bugs and your test cases helped a lot! Regarding the above issue, a workaround is to add the following block in the compiler plugin executions:

What I meant is that it does not determine whether something enters or not.

@desruisseaux
Copy link
Contributor Author

I pushed a commit which replace the symbolic links by a renaming of directories before and after compilation. The symbolic links caused confusion in some circumstances, because of same files visible from two locations. With this commit and the workaround in above comment, the compilation of streamex with aggregated javadoc succeed.

I will wait a little bit before to merge, in case you have a chance to test.

@fridrich
Copy link
Contributor

I will port all the patches and we will know soon. Thank goodness the build is bit-to-bit reproducible and does effective tree pruning :)

@fridrich
Copy link
Contributor

My tests are fine all. Everything compiles well.

@desruisseaux
Copy link
Contributor Author

Thanks! I will merge now then.

@desruisseaux desruisseaux merged commit 1ed03b8 into apache:master Jul 16, 2025
26 checks passed
@desruisseaux desruisseaux deleted the modular-multi-release branch July 16, 2025 12:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants