Leave jar and war tasks enabled by default and differentiate their output locations from those of bootJar and bootWar by configuring them with a classifier

[novettaberin]

I cannot publish my spring boot jar to our maven repository (which is helpful when creating a package to go on our customer's private network) with the way the plugin operates now. I'm using the Spring Boot 2.1.17.RELEASE plugin due to some customer requirements at the moment. Bottom line is that I need to include this in my Gradle build script to make it work:

configurations {
    [apiElements, runtimeElements].each {
        it.outgoing.artifacts.removeIf { it.buildDependencies.getDependencies(null).contains(jar) }
        it.outgoing.artifact(bootJar)
    }
}

If your plugin did this for me, then our builds would work out of the box without failure. In fact your plugin would now do no harm to the expected behavior of Gradle.

[philwebb]

Can you elaborate some more on what those lines do and why you need them? Perhaps a sample project might help to show exactly the problem that you're facing.

[novettaberin]

Without those lines, when I type gradle publish I get an error saying there was nothing built--even when there was. The simple original jar needs to be removed from the list of artifacts (removeIf dependency is the Jar task), and the bootJar needs to be added to the artifact list.

[wilkinsona]

In fact your plugin would now do no harm to the expected behavior of Gradle.

I don't agree with this. While we disable the jar task by default, we have seen users that re-enable the task. In some cases, they are choosing to publish the normal jar while not publishing the artifact produced by bootJar. The change being proposed here would hinder this use case.

[wilkinsona]

Having discussed this with the Gradle team (thanks again, @jjohannes), the preferred solution is to have both jar and bootJar enabled, with one configured with a classifier so that their output does not clash. Ideally, we'd configure the jar task with a classifier so that the output of bootJar continues to be written to its current location but I'm not sure how possible that is. We'd need to experiment a bit.

We should also register the output of bootJar as an additional variant for publishing as part of the Java component. We'd use a separate configuration for this with no dependencies (as is appropriate for a fat jar where all of the dependencies are built in) and add the bootJar task to it as an artifact.

[novettaberin]

Awesome. The bottom line I care about is to be able to publish the fat spring boot jar to the maven repository so that when we do our CI/CD deployment package it can pull all of those dependencies for us.

[novettaberin]

Looks like this won't be introduced until 2.5.x, any chance it would be backported to an also supported version? To be clear for Spring Boot 2.1.x and Spring Boot 2.3.x I would still need to use my hack in Gradle, correct?

[wilkinsona]

The changes proposed here won't be back ported. Until Spring Boot 2.5 is released, you should continue to use the workaround you've noted above and the Gradle team have documented.

[nucatus]

@bericoberin we faced a similar situation and we ended up doing this:

jar {
    enabled = true
}

bootJar {
    layered()
    archiveAppendix = 'boot'
}

In this way, the expected behavior is met when running both, jar and bootJar tasks without perturbing the downstream tasks.

The classifier attribute is deprecated for awhile and replaced with archiveClassifier. However, we didn't use this one because the classifier is put after the version in the archive name and we wanted a suffix to be put before the version so that this file can be easier identified in downstream systems without having to parse/grep versions.

Here is how gradle is composing the name of the archive in the jar task:

    ${archiveBaseName}-${archiveAppendix}-${archiveVersion}-${archiveClassifier}.${archiveExtension}

I kind of agree that this should be a default behavior when using spring boot plugin so that the default flow of gradle is not altered by the plugin.

[nucatus]

@wilkinsona I'm not sure whether publishing the bootJar to maven makes any sense, since the purpose of maven is to manage dependencies while keeping the size of the artifact minimal. If your code is 300kB, why publishing to maven artifacts that are two orders of magnitude bigger? I would leave this as an opt-in for the user, while keeping the default behavior where only the application jar is published by default, and not the fat jar.

[wilkinsona]

@nucatus It absolutely makes sense for certain use cases. @bericoberin describes one in the opening description of this issue.

This issue is primarily about leaving the jar task enabled as disabling it is surprising for some and has some unpleasant knock-on effects. Enabling both the jar task and the bootJar task will require us to do something to avoid the clash in their output locations. A qualifier is one way to do that and seems the most obvious but that may change as we start work on implementing this.

Gradle doesn't publish anything by default. When implementing any changes for this issue, we'll be aiming to make it straightforward to configure the publication of the normal jar, the fat jar, or both jars. To get the desired flexibility, this may require a separate software component or just a new variant on the existing component.

[novettaberin]

@nucatus That code example looks cleaner and would be more easily implemented in a company plugin if we needed to.

As to the concern about publishing to Nexus, it has to do with our deployments going to a disconnected network. We need to be able to quickly and easily assemble distribution packages. Pulling the specified versions from Nexus into a location on disk to pack on a DVD helps us tremendously since our customer has archaic processes, and we can't deploy directly from where we develop the code. Granted, this is a temporary solution until we are able to get completely containerized, but that is a process we can't start in the near future.

Our microservices are built from several separate repositories. So the timing of when one is deployed vs. another is not something we can directly control.

[nucatus]

@wilkinsona what I wanted to stress on is that the odds of publishing a Spring Boot fat jar to maven are much lower than the likelihood of publishing a classic maven artifact where the dependencies are described in the POM file. In my opinion, the latter should be the default, where the former would be an opt-in.

When I mentioned what gets published to maven by default, I referred to using the default gradle publish task that assumes that the outcome of the components.java is actually the outcome of the jar task, and this is inferred by gradle. In the gradle publish config below, gradle will publish the outcome of the jar task to maven. If the user wants another jar to be published, that has to be explicitly specified.

publishing {
    publications {
        myLibrary(MavenPublication) {
            from components.java
        }
    }
}

[snicoll]

@wilkinsona what I wanted to stress on is that the odds of publishing a Spring Boot fat jar to maven are much lower than the likelihood of publishing a classic maven artifact where the dependencies are described in the POM file.

Yet, the Maven Plugin works this way by default. If you're building an app and use the repackage goal of the Maven Plugin, it will replace the main artifact and publish that by default. You can opt-in for publishing both or only the regular module jar. While build systems are different and may lead to different way of configuring things, I think that what we do is the right behaviour considering you have to opt-in explicitly for it.

[wolfs]

I found out that the jar task and the bootJar task write to the same location without changing any configuration, as shown by the snippets above for enabling the jar task. It seems to me a good idea to configure the disabled jar task to write to a different location than the bootJar task, so the location is not picked up by default and it is easier to enable the task when necessary.

Moreover, there are some tasks by the application plugin, namely the startScripts, distTar and distZip task, all which will end up packaging up the bootJar without depending on it and without being meant to be used (see e.g. this build scan). The assemble lifecycle tasks also depends on those tasks, though normally you wouldn't want to run them. So maybe those tasks should be disabled by default as well?

[oehme]

I just wanted to throw in that having two jar tasks with the same output file can also confuse IntelliJ, so +1 for differentiating the two.

[wilkinsona]

Thanks, @wolfs.

In an earlier discussion with @jjohannes, he recommended leaving the jar task enabled but to also configure it or bootJar with a classifier. In your comment above, you're recommending leaving jar disabled and also disabling some downstream tasks as well as configuring jar or bootJar to avoid the output location clash.

We expect the majority of users to only be interested in the output of bootJar as it's unusual for a Spring Boot application to also be used as a dependency (where the output of the jar task would be useful). Because of this I'm leaning towards the leaving jar disabled approach but I'm wondering if I'm overlooking something.

[wilkinsona]

Expanding a bit on the problem mentioned above by @wolfs and the application plugin, with our current arrangement, assemble results in the output of distZip and distTar being faulty. They accidentally include the output of bootJar in their lib directory (as bootJar writes to the same location as jar) and then try to use it on the classpath. This doesn't work as the application's code is in BOOT-INF/classes. The distribution is also twice as big as it needs to be as the dependencies are both in the lib directory directly and in the BOOT-INF/lib/ directory of the fat jar.

With a clean build that doesn't run bootJar, distTar and distZip produce archives with the application's code missing entirely as the jar task is skipped. I wonder if Gradle should fail these tasks if an expected input is absent?

[wolfs]

In an earlier discussion with @jjohannes, he recommended leaving the jar task enabled but to also configure it or bootJar with a classifier. In your comment above, you're recommending leaving jar disabled and also disabling some downstream tasks as well as configuring jar or bootJar to avoid the output location clash.

I am not recommending leaving the jar task disabled. I think the best solution would be to have the jar and the bootJar task enabled, both producing sensible output in separate locations. Though if the jar task is disabled, then the downstream tasks depending on the jar task should also be disabled, so they don't accidentally pick up things from the bootJar task or run without any reason.

With a clean build that doesn't run bootJar, distTar and distZip produce archives with the application's code missing entirely as the jar task is skipped. I wonder if Gradle should fail these tasks if an expected input is absent?

Yeah, I think Gradle should fail in that case. I would need to look closer into how tasks are wired up currently why it doesn't. I suppose we ignore missing files (aka locations where the file does not exist) for archive tasks like Zip and Tar in general.

[wilkinsona]

Thanks again, @wolfs. We'll look at leaving things enabled by default in 2.5 and configuring separate output locations.

[wilkinsona]

The publishing side of this is a bit of a mess at the moment. With a clean project, if you run a publish task in isolation it'll fail because the jar isn't there:

> Task :compileJava NO-SOURCE
> Task :processResources NO-SOURCE
> Task :classes UP-TO-DATE
> Task :jar SKIPPED
> Task :generateMetadataFileForMavenPublication
> Task :generatePomFileForMavenPublication
> Task :publishMavenPublicationToMavenRepository FAILED

This is happening because the jar task is disabled.

If you explicitly run bootJar as well, it still fails:

$ ./gradlew clean bootJar publishAllPublicationsToMavenRepository --console=plain
> Task :clean
> Task :compileJava
> Task :processResources
> Task :classes
> Task :bootJarMainClassName
> Task :bootJar
> Task :jar SKIPPED
> Task :generateMetadataFileForMavenPublication
> Task :generatePomFileForMavenPublication
> Task :publishMavenPublicationToMavenRepository FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':publishMavenPublicationToMavenRepository'.
> Failed to publish publication 'maven' to repository 'maven'
   > Artifact publish-test-0.0.1-SNAPSHOT.jar wasn't produced by this build.

If you enable the jar task the publish will succeed by the jar that's published depends on whether jar runs before or after bootJar. The last one that runs will win. When the fat jar wins the pom and Gradle module metadata are wrong as they shouldn't have any dependencies.

[wilkinsona]

For consistency, we should also leave the war task enabled and differentiate its output using a classifier.

When the Spring Boot plugin's applied, I think it makes sense to consider the artifact generated by bootJar or bootWar to be the main artifact. Therefore, the classifier should be configured on the jar and war tasks.

When the plain jar task's output is being used, it's typically because parts of the application are being used as a library/dependency elsewhere. I think library may be a reasonable classifier.

It's less clear why the plain war task's output would be used, given that the output of bootWar can be deployed to a servlet container or executed via java -jar. This makes an appropriate classifier harder to name. Something like standard or plain is the best I've managed to think of thus far.

Flagging for team attention to see if anyone has any suggestions for the classifiers' names.

[philwebb]

We're going to try plain as the classifier for both jar and war.

[RobbanHoglund]

This breaks existing builds by creating an additional jar-file.
When we are starting the jars assembled in the container/pods it fails with the following message:

no main manifest attribute, in /deployments/myapplication-SNAPSHOT-plain.jar

We could do a work around by adding this to the build.gradle:
jar.enabled = false

[snicoll]

@RobbanHoglund how does it break existing builds? It looks like a a custom copy command of yours is too agressive and take any jar file from the build/libs directory perhaps?

[RobbanHoglund]

Yes we are deploying whatever jar that is produced by the "gradle assemble". And that was Ok with the previous behavior with the assumption that only one Springboot application jar was created by the assemble. Now, with the changed behavior, 2 different jars are created and where we happens to start the "non Springboot application jar" with the resulting failure posted above.

We will fix our deploy mechanism to handle the fact that there may be multiple artifacts created and that we only deploy the Springboot application jar....

[mauro1855]

Hi,

We just updated to 2.5, and indeed we also experienced the issue mentioned by @RobbanHoglund with our existing CI config. It is setup to get any .war that matches the application name (specifically, matches name*.war), but now that both wars are generated, it doesn't find one single war file to deploy as before, thus failing our deployment plan. I either have to explicitly disable the plain war in my project, or adjust my CI config.