Migrating out of spring.profiles.include into spring.profiles.group

[blacelle]

While migrating to SpringBoot 2.4.X, I encountered an issue while migrating from spring.profiles.include. I drop my questions and some supposed answers here. Fact this Upgrading to SpringBoot 2.4 is a pain in our case as we massively rely on spring.profiles.include to chain profile activation, and the documentation/migration notes are unclear for our specific-case.

(Sorry for the quite long ticket. This ticket has been turned from I'm completely lost to Thanks for confirming I'm going the right way, others might be interested)

First, very helpful links:
https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4
https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-Config-Data-Migration-Guide
#22944

Second, would it be relevant to WARN on such deprecated piece of configuration remaining in the project configuration? With a second thought, this looks a bad idea since #21697 restored this property in some usages. But in my case, it took my a while just to understand this property was removed (i.e. my bad, I did not consider going through Release notes, and the Migration Guide).

In our case, we rely massively on spring.profiles.include to triggers profiles chained activation. Typically, we have the following profiles:

• prod profile, which includes prod-db, prod-queue and prod-common.
• prod-db includes common-db and prod-common
• prod-queue includes common-queue and prod-common
• etc.

The new group features looks great to ensure prod triggers prod-db, prod-queue and prod-common, but I'm not sure what's the proper way to handle the rest of the chain of profiles.

Should I introduce a prod-db groups over common-db and prod-common, and a prod-queue groups over common-queue and prod-common? I guess so.

To confirm what it would look like:

spring.profiles.group.prod-db: common-db,prod-common

However, I also understand I have to pack all these groups into some main application.properties: this is not nice for us as we have many such groups, which are currently described as spring.profiles.include within each profile specific file.

In #22944 (comment), I don't get what is a not in a profile-specific document. Does not in a profile-specific document refers to file being imported through spring.config.import? I guess so.

In addition, given we have multiple applications, each with their own application.yml (and quite a bunch of tests also activating sub-sets of profiles, relying on the chains of profile), I would prefer to push these groups definition into a different specific file. Is spring.config.import the proper way to handle such a case?

I wonder if spring.profiles.include could be restored for simple-single profile-specific documents (e.g. document associated a single and plain profile, not meta-profiles like !dev, nor multi-profile documents).

[philwebb]

Thanks for opening the issue @blacelle. We suspected that there would be a few users with complex setups like yours that unfortunately will find the upgrade difficult. In the short-term, you can try setting spring.config.use-legacy-processing to true which should restore the old behavior. This will at least allow you to upgrade.

Should I introduce a prod-db groups over common-db and prod-common, and a prod-queue groups over common-queue and prod-common?

Yes, you can define multiple groups, so you should be able to do something like this:

spring:
  profiles:
    group:
      prod: prod-db, prod-queue, prod-common
      prod-db: common-db
      prod-queue: common-queue

I don't get what is a not in a profile-specific document. Does a not in a profile-specific document refers to file being imported through spring.config.import?

This means that you can use spring.profiles.include but you can't combine it with spring.config.activate.on-profile in any way. We now process configs in distinct sweeps and once we've activated the profiles, we don't allow any new config files to be imported.

I would prefer to push these groups definition into a different specific file. Is spring.config.import the proper way to handle such a case?

Yes, you can define your profile groups in a distinct file and import it if you want to. For example you can have an application.yaml file like this:

spring:
  config:
    import: classpath:profile-groups.yaml

Then have a profile-groups.yaml file that defines the groups.

I wonder if spring.profiles.include could be restored for simple-single profile-specific documents (e.g. document associated a single and plain profile, not meta-profiles like !dev, nor multi-profile documents).

This would unfortunately negate most of the benefit that we're trying to achieve with the changes. We're really trying to make our own code much more robust and remove as much complexity as possible.

Please let us know if the profile groups feature works for you. If not, perhaps you can share a sample project that shows your current setup and we can work out if there's something else we can do.

[blacelle]

We're really trying to make our own code much more robust and remove as much complexity as possible.

I completely catch the point of this evolution.

Still, I'm quite disappointed as I was very happy of leveraging spring.profiles.include as an elegant solution to manage 50+ profiles (various environments, various sub-systems, various 3rd-parties each of them having multiple environments). The bigger pain for me will be to centralised the groups in a single file, making it less readable than current configuration.

About:

I wonder if spring.profiles.include could be restored for simple-single profile-specific documents (e.g. document associated a single and plain profile, not meta-profiles like !dev, nor multi-profile documents).

This would unfortunately negate most of the benefit that we're trying to achieve with the changes.

I supposed it would be quite straightforward if considering only single-simple profiles (just like I may end writing my own spring.profiles.include scanner, to activate the proper profiles before booting). You know better.

Also, what about warning/failing if spring.profiles.include is used in profile-specific documents (and spring.config.use-legacy-processing is not toggled)?

On my side, I'm pretty sure I'll succeed to comply with the new standard. Thanks for the feedback. Feel free to close.

[philwebb]

Also, what about warning/failing if spring.profiles.include is used in profile-specific documents (and spring.config.use-legacy-processing is not toggled)?

You should get an exception for that already.

Thinking some more about your use-case. You might be able to use an EnvironmentPostProcessor to activate the profiles programmatically. It's a bit hard to say for sure without seeing a sample app that shows your current setup.

[blacelle]

@philwebb In https://docs.spring.io/spring-boot/docs/2.4.0/reference/htmlsingle/#spring.profiles.include, the lack of impact related to discussed change mislead me in the first step of our migration. I suppose some note may be added there.

You should get an exception for that already.

I definitely did not get one. Especially in unit-tests relying on @ActiveProfiles (i.e. I face this issue by having tests behaving weirdly, it appeared chained profiles were not applying -> no exception, but test broken).

The source of the test is:

@RunWith(SpringRunner.class)
@SpringBootTest
@ActiveProfiles({ "someDevProfile", "someAppProfiles" })
@ContextConfiguration(classes = { TestConfigFromAzureVault.SpringFromAzureVaultTest.class })
public class TestConfigFromAzureVault {

	@BeforeClass
	public static void assumeInternetIsAvailable() {
		// We need internet access to connect DEV KeyVault
		PepperTestHelper.assumeInternetIsAvailable();
	}

	@Autowired
	Environment env;

	@SpringBootApplication
	public static class SpringFromAzureVaultTest {

	}

	@Test
	public void testSomeSecretFromAzureVault() {
		// This property is defined in Azure KeyVault
		String secretKey = env.getRequiredProperty("azk-keepme-unittest");

		Assert.assertEquals("someSecret", secretKey);
	}
}

Hope it helps.

[philwebb]

@blacelle Can you share a small project that shows that full test? I think I need to see the contents of the application.yaml as well.