Multiple Datasources and JPA obey Principle of Least Astonishment

[cosmocracy]

A surprising amount of work--entirely boilerplate--is currently necessary for a Spring Boot app to utilize two (or more) Datasources with JPA. There is fantastic support for a single implicit Datasource (and autowiring it for JPA use) by configuring a handful of properties. By comparison an extraordinary amount of work--annotations and interfaces/classes--are requried to support a second Datasource with JPA (EnableTransactionManagement, EnableJpaRepositories, EntityManagerFactory, EntityManagerFactoryBuilder, PlatformTransactionManager, JpaTransactionManager, DataSourceBuilder, LocalContainerEntityManagerFactoryBean, ...)

This issue proposes that Spring Boot configuration guidelines--and supporting changes to autowiring--be developed such that adding additional Datasources for JPA doesn't require departing from a configuration-over-code mindset. The current approach certainly falls afoul of the Principle of Least Astonishment:

In general engineering design contexts, the principle can be taken to mean that a component of a system should behave in a manner consistent with how users of that component are likely to expect it to behave; that is, users should not be astonished at the way it behaves

The fact that so many developers spend time troubleshooting and discussing how to accomplish this is good evidence that a simplification is in order.

See also:

• https://medium.com/@joeclever/using-multiple-datasources-with-spring-boot-and-spring-data-6430b00c02e7
• https://stackoverflow.com/questions/27614301/spring-boot-multiple-datasource
• http://www.onlinetutorialspoint.com/spring-boot/spring-boot-multiple-data-sources-example.html
• http://roufid.com/spring-boot-multiple-databases-configuration/
• https://scattercode.co.uk/2016/01/05/multiple-databases-with-spring-boot-and-spring-data-jpa/
• http://www.baeldung.com/spring-data-jpa-multiple-databases
• http://www.javaoptimum.com/how-to-configure-multiple-datasources-with-spring-boot/
• ...
• Issue Improve documentation showing how to use two different databases #3456
• Issue Provide an example of how to configure multiple datasources and configuration consumption #7652

[wilkinsona]

Do you have a suggestion as to how this could be simplified? How would you expect Spring Boot to infer that you want multiple DataSources, multiple EntityManagers etc?

[spring-projects-issues]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.

[spring-projects-issues]

Closing due to lack of requested feedback. If you would like us to look at this issue, please provide the requested information and we will re-open the issue.

[OrangeDog]

Reopen please? I have suggestions for the simple case of multiple databases.

Defining multiple DataSources isn't too bad but could be easier. I'm doing it like this

private static void copyNonNullProperties(Object source, Object target, String... ignoreProperties) {
    Collection<String> ignoredSet = Arrays.asList(ignoreProperties);
    BeanWrapper sourceWrapper = PropertyAccessorFactory.forBeanPropertyAccess(source);
    BeanWrapper targetWrapper = PropertyAccessorFactory.forBeanPropertyAccess(target);

    Arrays.stream(sourceWrapper.getPropertyDescriptors())
            .map(pd -> new PropertyValue(pd.getName(), sourceWrapper.getPropertyValue(pd.getName())))
            .filter(pv -> pv.getValue() != null)
            .filter(pv -> !ignoredSet.contains(pv.getName()))
            .forEach(targetWrapper::setPropertyValue);
}

@Bean
@Primary
@ConfigurationProperties("spring.datasource.hikari")
public HikariConfig defaultHikariConfig() {
    return new HikariConfig();
}

@Bean
@ConfigurationProperties("second.datasource")
public DataSourceProperties secondDataSourceProperties() {
    return new DataSourceProperties();
}

@Bean
@ConfigurationProperties("second.datasource.hikari")
public DataSource secondDataSource() {
    DataSource dataSource = secondDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class).build();
    copyNonNullProperties(defaultHikariConfig(), dataSource, "class");
    return dataSource;
}

@Bean
public Object telemetryDataSourceInitializer(
        @Qualifier("secondDataSource") ObjectProvider<DataSource> dataSource,
        ApplicationContext ctx
) throws ReflectiveOperationException
{
    Class<?> clazz = Class.forName("org.springframework.boot.autoconfigure.jdbc.DataSourceInitializerInvoker");
    Constructor<?> constructor = clazz.getDeclaredConstructor(
            ObjectProvider.class, DataSourceProperties.class, ApplicationContext.class
    );
    constructor.setAccessible(true);
    return constructor.newInstance(dataSource, secondDataSourceProperties(), ctx);
}

It would be nice if I didn't have to specify the type, and Boot would automatically apply the auto-configured type, corresponding vendor properties, and initialization.

It's the EMFs that are the real pain. If you are separating everything by package scan then it's not too bad, though you need to do some copy-pasting of private methods and hard-coding of Hibernate to get the vendor properties with defaults and customizers. Some duplication would be saved if the EntityManagerFactoryBuilder bean came with more defaults in place (properties, mappingResources, jta).

Downsides of package separation include having to refactor packages to change datasources, and not being able to have multiple EMFs for the same type. Any other method for assigning entities becomes a bit of a nightmare (for repositories it's manageable, because @EnableJpaRepositories accepts include and exclude filters). If the builder accepted similar scan filters, or even a custom PersistenceUnitInfo, it would be a lot easier to use. The SpringPersistenceUnitInfo should also be public so these entities don't miss out on LTW.

I think an even better solution is to make use of the @PersistenceUnit and @PersistenceContexct annotations on entity and repository classes. I'm not sure of the technical details of how that should be wired up, but it should at least be possible to filter the EMF's scan by classes annotated with a matching unitName. Perhaps it would also be possible to then only require a single @EnableJpaRepositories which could use the annotated name to do the respository-emf bindings, as long as there's a matching bean.

[wilkinsona]

@OrangeDog Thanks for sharing your thoughts. We've now got #15732 which has superseded this issue.

[OrangeDog]

@wilkinsona that doesn't cover the JPA stuff though?

[wilkinsona]

It says:

We'd also need similar functionality for components that consume a DataSource such as JPA and transaction management.

We can't really know what JPA, transaction management etc will look like until we're figured out the DataSource piece as everything else will build on top of it.

[OrangeDog]

I see. I'm not looking for full autoconfiguration, just slightly easier custom configuration that doesn't have to duplicate everything the Boot would've done. As I mentioned above, a small first change that would be helpful is if the EntityManagerFactoryBuilder and DataSourceBuilder came with more of the autoconfig defaults baked in.

If you do put in full multi-JPA autoconfiguration and I want something slightly different, I don't want to be back to doing everything myself again.