Checkstyle 10.0 upgraded on LTS 8.9.x, breaking analysis of Java 8 builds

[ianwilliams1]

Sonar version: LTS sonarqube-8.9.6 -> sonarqube-8.9.7
Checkstyle sonar plugin version: 9.2.1 -> 9.3 (intended) -> 10.0 (actual)
All builds to be analyzed are Java 8

Steps to reproduce or description of problem:
See SonarQube issue : Plugin Update does not update as expected for more details

Post upgrade, validation sonar-scan using Java 8 FAILED with:
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.7.0.1746:sonar (default-cli) on project pom-sampleApplication-all: The plugin [checkstyle] does not support Java 1.8.0_321: org/sonar/plugins/checkstyle/CheckstylePlugin has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 -> [Help 1]

The problem is two-fold:

• SonarQube button indicated "Upgrade to 9.3", but in fact upgraded to 10.0
• Plugin release 10.0 shows:

  <properties>
   <java.version>11</java.version> 
   <sonar.version>8.9.0.43852</sonar.version>

SonarQube 8.9 Requirements show:

Supported Platforms
Java
SonarQube scanners require version 8 or 11 of the JVM and the SonarQube server requires version 11. Versions beyond Java 11 are not officially supported.

SonarQube is able to analyze any kind of Java source files regardless of the version of Java they comply to.

Java	Ver	Server	Scanners
Oracle JRE	11	Y	Y
	8	X	Y
OpenJDK	11	Y	Y
	8	X	Y

As the plugin qualifies as a "Scanner", it is not compliant with the configuration. Had I known the App would jump to latest, I would not have upgraded plugin, remained on 9.2.1 or manually installed 9.3. I am confused by their statement in italics.

Not sure what can be done to address the issue, you may need guidance from SonarQube, but since the update was only to include check styles 10.0 which sets minimum Java to 11 anyway, my recommendation is release a 10.0.1, configured as:

  <properties>
   <java.version>11</java.version> 
   <sonar.version>sonarqube-9.0.1.4610</sonar.version>

and burn 10.0 release. Hopefully that will prevent the automatic update within SonarQube LTS 8.9.x beyond check-style-9.3 as it does not meet the minimum SonarQube version.

[muhlba91]

thanks for your feedback, and sorry to hear you encountered such issue.

however, reading the platform requirements we still fulfill SQ LTS requirements.
as you posted, the SQ requirements clearly state scanners require "java 8 or 11". this expression is a disjunction (only one of them must be true) and not a conjunction where both conditions must be met (that would be expressed by "java 8 and 11").

additionally, our release notes state the minimum required java version - which is 11 as of 10.0:
https://github.com/checkstyle/sonar-checkstyle/blob/master/README.md?plain=1#L24
https://github.com/checkstyle/sonar-checkstyle/releases/tag/10.0
https://community.sonarsource.com/t/new-release-sonar-checkstyle-10-0/59436
https://github.com/SonarSource/sonar-update-center-properties/pull/277/files#diff-e3c71559bbcf09fc1f44e9e4f4b22e8855e570a76b4a443ed309217dbc6ca011R10
those are also shown in the SQ marketplace:

moreover, we also increased the plugin version to a new major version indicating at least one major change. in this case dropping java 8 support, which is also a backwards compatibility breaking change.

hence, i don't see any violation of SQ scanner requirements in this case and neither an unexpected change because we indicated this through a new major version and stated this breaking change in the changelog which is also clearly shown in the SQ marketplace, and can be reviewed easily before upgrading the plugin.

the solution in your case is to downgrade the plugin by replacing the jar file with an older release (e.g. https://github.com/checkstyle/sonar-checkstyle/releases/tag/9.3).

outside of this issue: updates (no matter of what kind) should always be done carefully by reading the release notes/changelogs, and possibly also test such upgrades in a dedicated environment. additionally, major version changes might introduce not backwards compatible changes.

[ianwilliams1]

@muhlba91 , Thank you for your considered feedback. As I noted in the issue raised with SonarQube, the primary failure is in that the SonarQube "Upgrade to ..." button clearly showed upgrade to 9.3 and that was my intent. To my surprise I ended up with the incompatible 10.0 release. I agree you did the right thing in semver and documentation; I simply had no intention crossing the threshold.

The DepCheck plugin on the other hand, went from 2.0.8 to 3.0.0 without documenting the min. Java 11 in the Release Notes and it's only mentioned deeply in the PR details - (Sonarqube 8 by @Reamer).

The secondary issue, which obviously requires feedback from SonarSource is that if a "plugin" is compiled with Java 11, it simply will not function on a scanner infrastructure running Java 8. So, I don't understand their requirements / supported version table.The only solutions I know of are to either provide two plugins variants, one for each supported target version or perhaps build with 11 but specify a target of 8, and I don't even know if that would work, depending on plugin behavior. Alternatively, the sonar-scanner should omit processing for plugins dependent on a higher Java platform that what is present (tricky). The problem being though, the Server side has no idea what's available on the scanner side until a scan is initiated.

Again, these are not issues to be addressed by the plugin community (other than by supporting the lowest common denominator), rather from SonarSource itself. It's just odd that we've been using SonarQube with a laggard Java since 4.1.1 and never encountered this problem until now, where 2 of the 4 3rd party plugins with upgrades failed us (8 total) in two weeks!

At least the issue is documented for others. Until SonarQube addresses the misleading "Upgrade to ..." <implied latest, not what the label says>, which got me here, you may refer other victims to this issue as you and I documented. If SonarSource provide alternate guidance, I will create a new issue.