Skip to content

Generate configuration metadata for types in other modules #18366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jjoslet opened this issue Sep 26, 2019 · 16 comments
Closed

Generate configuration metadata for types in other modules #18366

jjoslet opened this issue Sep 26, 2019 · 16 comments
Assignees
Labels
type: enhancement A general enhancement
Milestone

Comments

@jjoslet
Copy link

jjoslet commented Sep 26, 2019

Spring Boot version: 2.1.8.RELEASE

When using a class imported from a dependency as a @ConfigurationProperties, the generated configuration properties meta-data does not contains the description for this class' properties.

Scenario to reproduce:
I test the example described in the documentation to create two data sources : https://docs.spring.io/spring-boot/docs/current/reference/html/howto-data-access.html#howto-two-datasources.

The produced spring-configuration-metadata.json does not include the description for the properties of DataSourceProperties class.

Here is a sample of the produced meta-data:

{
   "name": "app.datasource.first.name",
   "type": "java.lang.String",
   "sourceType": "org.springframework.boot.autoconfigure.jdbc.DataSourceProperties"
},

Sample project: demo.zip

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Sep 26, 2019
@snicoll
Copy link
Member

snicoll commented Sep 26, 2019

This is working as expected I am afraid. If the source code of DataSourceProperties is not available, we can't extract the documentation and default value as the annotation processor doesn't have access to the source model.

However, I can see how that's annoying and how the doc example is misleading. Flagging for team attention.

@snicoll snicoll added the for: team-attention An issue we'd like other members of the team to review label Sep 26, 2019
@jjoslet
Copy link
Author

jjoslet commented Sep 26, 2019

Since this is an annotation processor, this is hard to retrieve the JavaDoc from a compiled class.

I think this pattern can have a different behavior for the computation of the default value too.

Have you planned another way to produce the description which does not suffer from this limitation ?

Maybe another annotation that is used to describe the property ?

@snicoll
Copy link
Member

snicoll commented Sep 26, 2019

@jjoslet I think I wrote exactly that in my previous comment, didn't I? We haven't scheduled anything at this point but I'd like to discuss it a bit more which is why I've flagged it.

@philwebb philwebb added type: enhancement A general enhancement and removed for: team-attention An issue we'd like other members of the team to review status: waiting-for-triage An issue we've not yet triaged labels Sep 27, 2019
@wilkinsona
Copy link
Member

We may be able to find the existing metadata that was generated for DataSourceProperties from the classpath and splice it into the metadata that is being generated.

@philwebb philwebb added this to the 2.x milestone Sep 27, 2019
@jjoslet
Copy link
Author

jjoslet commented Sep 30, 2019

That will solve the issue for the particular DataSourceProperties example since it already produces meta-data. But it'll not handle the case where a third party class is not originally annotated with @ConfigurationProperties and is used in an application with @ConfigurationProperties.

Example in Spring Boot: HikariDataSource, org.apache.tomcat.jdbc.pool.DataSource. org.apache.commons.dbcp2.BasicDataSource, etc.

@snicoll
Copy link
Member

snicoll commented Sep 30, 2019

We have metadata for that too. Ssl is more what you meant and we'll have to figure out how we can update the annotation processor to handle it. I think the issue is well understood, we now need to design it.

@jjoslet
Copy link
Author

jjoslet commented Sep 30, 2019

Yes, metadata are present but they miss the description.

{
  "name": "spring.datasource.dbcp2.username",
  "type": "java.lang.String",
  "sourceType": "org.apache.commons.dbcp2.BasicDataSource"
}

@snicoll
Copy link
Member

snicoll commented Sep 30, 2019

It's a third party type, they don't have description as it's not a @ConfigurationProperties type to begin with.

@cachescrubber
Copy link
Contributor

Since #19500 is closed: My use-case is not to generate metadata for external 3rd-party classes outside my control. I am the maintainer of the jar providing the @ConfigurationProperty type and would cooperate in providing the required metadata - for example by using annotations instead of JavaDoc or by means of manually publishing the required metadata within my jar.

@snicoll
Copy link
Member

snicoll commented Dec 31, 2019

I got that. Generating the metadata and reusing it is covered here too.

@wilkinsona
Copy link
Member

#34210 is related to this. It proposed allowing the configuration processor to consume multiple additional metadata files.

@wilkinsona wilkinsona added the status: pending-design-work Needs design work before any code can be developed label Feb 21, 2023
@snicoll snicoll changed the title ConfigurationProcessor : description missing from dependency Generate configuration metadata for types in other modules May 5, 2025
@snicoll snicoll self-assigned this May 5, 2025
@snicoll snicoll modified the milestones: 3.x, 4.0.x May 5, 2025
@snicoll snicoll removed the status: pending-design-work Needs design work before any code can be developed label May 5, 2025
@snicoll
Copy link
Member

snicoll commented May 5, 2025

I am investigating the option to source configuration metadata for any arbitrary types in the form of META-INF/spring/com.example.MyType.json - When this file is present, the annotation processor looks it up for any processing of MyType and add the necessary metadata.

The next step is to offer a way to run the annotation processor on MyType in its module to generate the file above. Files can also be contributed manually for types that can't run with the AP. This would centralize the description, default, etc in one place.

@snicoll
Copy link
Member

snicoll commented May 5, 2025

We're brainstorming where to put those files and wondering if META-INF/spring/configuration-metadata could be a good home for everything configuration-metadata related. One idea is to put the metadata for the "module" (the one we have now) alongside the one generated for a given type. Then an additional sub-directory could be used for the additional (manual) metadata.

This would give something like this:

META-INF
├── spring
│   └── configuration-metadata
│       ├── additional
│       │   ├── configuration-metadata.json
│       │   └── org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json
│       ├── configuration-metadata.json
│       └── org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json

We're not 100% happy with this just yet. The per-type metadata is only used as a temporary artifact for anyone willing to embed the org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties type in a @ConfigurationProperties-annotated type. As such, tooling such as IDE don't really need to handle them.

@philwebb
Copy link
Member

philwebb commented May 5, 2025

Another option:

META_INF/
  spring-configuration-properties-metdata.json (v1 compat)
  spring/
    configuration-properties-metadata.json (introduce only when v2 is needed)
    configuration-properties-metadata-source/
      additional.json
      org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json
      org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties-additional.json

@philwebb
Copy link
Member

After some more discussion we've landed on:

4.0.x:

META-INF/
  spring-configuration-metdata.json
  additional-spring-configuration-metdata.json
  spring/
    configuration-properties/
      org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json
      additional/
        org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json

If we need a v2 metadata later:

META-INF/
  spring/
    configuration-properties/
      metdata.json
      org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json
      additional/
        metdata.json
        org.springframework.boot.autoconfigure.jms.JmsPoolConnectionFactoryProperties.json

snicoll added a commit that referenced this issue May 13, 2025
Previously, if a ConfigurationProperties had a nested type or was
extending from a type located outside the compilation unit, no
metadata discovered on the source code was available (documentation and
explicit default value, if any). This typically happens when such a type
resides in another module.

This commit introduces `@ConfigurationPropertiesSource` as a way to
annotate such type and have metadata generated for them in their own
module.

Type-metadata is generated as one file per type and is reused
transparently whenever that type is used. As for module metadata, an
additional file can be crafted manually and will be merged when the
metadata for the type is generated.

The following is an example structure with two types where one has
an additional metadata:

META-iNF/
  spring/
    configuration-properties/
      additional/
        com.example.SourceOne.json
      com.example.SourceOne.json
      com.example.SourceTwo.json

Those files are used only by the annotation processor and are not meant
to be public API.

See gh-18366
snicoll added a commit that referenced this issue May 13, 2025
Closes gh-18366
@snicoll
Copy link
Member

snicoll commented May 13, 2025

Closed by 8c56624

@snicoll snicoll closed this as completed May 13, 2025
philwebb added a commit that referenced this issue May 13, 2025
See gh-18366

Co-authored-by: Phillip Webb <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

6 participants