Create a new layer for loader classes

[nebhale]

This one is an observation for discussion rather than a requirement for improvement. At this time, Spring Boot's default OCI image layering strategy describes four layers:

1. dependencies
2. snapshot-dependencies
3. resources
4. application

There is, however, an implicit fifth layer containing (in a typical case):

• META-INF/MANIFEST.MF
• META-INF/maven/**
• BOOT-INF/classpath.idx
• BOOT-INF/layers.idx
• org/springframework/boot/loader/**

My observation is that this final implicit layer mixes lifecycles. The contents of META-INF/MANIFEST.MF, META-INF/maven/**, and BOOT-INF/classpath.idx are broadly application-specific and change for every tagged version of an application while BOOT-INF/layers.idx and org/springframework/boot/loader/** are much more stable, and generally would be duplicate even across applications within an enterprise (given consistent versions of Boot).

The concrete result of this is that every single version of an application using this layering would end up with a layer that isn't de-duplicatable but contains (at the current time) 364K of duplicated files and 12K of unique files. I think we can all agree that this isn't the biggest inefficiency in a typical enterprise deployment system, but there is room for a big win both in cumulative storage and cumulative data transfer (i.e. deployment speed) if these files can be de-duped in a registry and cached at edge nodes.

[mbhave]

At the moment, if layertools is used, the list above ends up in the application layer. The loader classes are indeed not tied to the application and we did discuss moving them to a separate layer (maybe dependencies?).

[nebhale]

Yeah, you're absolutely right (I misread some documentation). Upon re-examination with the proper layout, I think that they only change to my observation would be that the duplication probably wouldn't matter if the application layer was vastly larger than ~346K most of the time. Do you have any feel for how large the application-contributed classes would be in a typical Spring Boot application.

[mbhave]

Good question. I don't but someone else on the team might. Flagging for team attention so that we can decide if we want to do anything about this.

[wilkinsona]

I can offer some anecdata.

Sagan's sagan-site has over 800KB of content that would go in the application layer. If we moved BOOT-INF/classes/templates by default into the resources layer it drops to ~500KB. I don't think of Sagan as being particularly huge in real-world terms.

I suspect plenty of apps would have more than ~346KB in the application layer. That said, we'd also have plenty of apps where it's almost empty as pretty much everything's either in the resources layer or one of the dependencies layers.

It feels intuitive to me for the loader classes to go in the dependencies layer by default. That's where other Boot jars will go and the loader classes will change at the same frequency as those jars. We probably also need to consider some changes to the Maven XML and Gradle DSL to allow users to move the loader classes to a different layer if they wish.

[nebhale]

The dependencies layer feels right to me as well as the loader is tied strongly to the version of Boot, which strongly influences the rate of change of the dependencies layer.

Note, the loader is particularly troublesome for Boot's current layering mechanism because it builds layers by moving files and those are immovable. So, if this turns out to be a bear to implement/configure it probably won't be worth it.

[mbhave]

We probably also need to consider some changes to the Maven XML and Gradle DSL to allow users to move the loader classes to a different layer if they wish.

The Maven XML and Gradle DSL determines where something ends up in the layered jar. I'm not sure if we can do anything about the loader classes in that case since they are at the root of the jar rather than nested in BOOT-INF/layers.

[wilkinsona]

We going to place everything in /org/springframework/boot in a new spring-boot-loader layer by default. It will be sandwiched between the default dependencies and snapshot-dependencies layers by default. If the user customises the layers and does not define a spring-boot-loader layer we'll create our own fallback layer that will be the upper-most layer.

[nebhale]

@wilkinsona How will that work given the classpath implications?