Improve testcontainers lifecycle to Initialize containers first and destroy them last

[philwebb]

Container instances should be started as early as possible and stopped as late as possible. This will help with #35201 and also reduce the likelihood of issues such as the one reported by @odrotbohm :

023-05-01T14:44:31.221+02:00  INFO 64148 --- [localhost:51526] org.mongodb.driver.cluster               : Exception in monitor thread while connecting to server localhost:51526

com.mongodb.MongoSocketReadException: Prematurely reached end of stream
	at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:115)
	at com.mongodb.internal.connection.SocketStream.read(SocketStream.java:138)
	at com.mongodb.internal.connection.InternalStreamConnection.receiveResponseBuffers(InternalStreamConnection.java:735)
	at com.mongodb.internal.connection.InternalStreamConnection.receiveMessageWithAdditionalTimeout(InternalStreamConnection.java:596)
	at com.mongodb.internal.connection.InternalStreamConnection.receiveCommandMessageResponse(InternalStreamConnection.java:440)
	at com.mongodb.internal.connection.InternalStreamConnection.receive(InternalStreamConnection.java:397)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:227)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:159)
	at java.base/java.lang.Thread.run(Thread.java:833)

2023-05-01T14:44:31.316+02:00 DEBUG 64148 --- [ionShutdownHook] o.s.data.mongodb.core.MongoTemplate      : find using query: { "completionDate" : null} fields: Document{{}} for class: class org.springframework.modulith.events.mongodb.MongoDbEventPublication in collection: org_springframework_modulith_events
2023-05-01T14:44:31.316+02:00  INFO 64148 --- [ionShutdownHook] org.mongodb.driver.cluster               : Cluster description not yet available. Waiting for 30000 ms before timing out
2023-05-01T14:44:31.729+02:00  INFO 64148 --- [localhost:51526] org.mongodb.driver.cluster               : Exception in monitor thread while connecting to server localhost:51526

com.mongodb.MongoSocketOpenException: Exception opening socket
	at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:73)
	at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:204)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:199)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:159)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542)
	at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597)
	at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327)
	at java.base/java.net.Socket.connect(Socket.java:633)
	at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107)
	at com.mongodb.internal.connection.SocketStream.initializeSocket(SocketStream.java:82)
	at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:68)
	... 4 common frames omitted

2023-05-01T14:45:01.322+02:00  WARN 64148 --- [ionShutdownHook] o.s.b.f.support.DisposableBeanAdapter    : Invocation of destroy method failed on bean with name 'eventPublicationRegistry': org.springframework.dao.DataAccessResourceFailureException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=localhost:51526, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketOpenException: Exception opening socket}, caused by {java.net.ConnectException: Connection refused}}]

This appears to be a lifecycle issue cause because a DisposableBean wants to use a container service in their destroy method.

[odrotbohm]

In my particular case, Boot didn't control the lifecycle of the service instances. I was using @Container annotated, static fields on a test class annotated with @Testcontainers. In that case, Testcontainers manages the service instance and ties its lifecycle to the class, i.e., shuts down the service in a JUnit AfterAllCallback. At that point, the ApplicationContext instance created for a test class still lives in the test context cache. If that AC is reused from a different test class that doesn't set up similar infrastructure, the components in the AC will not have access to any services. In my particular case, it was code running in a destruction callback of a Spring bean.

Declaring the containers as Spring bean is a decent workaround, but I wonder if we should outline the lifecycle problem in the reference documentation.

[philwebb]

If that AC is reused from a different test class that doesn't set up similar infrastructure, the components in the AC will not have access to any services

I probably need to test the latest code with whatever sample you have. In theory the second test shouldn't be able to reuse the AC unless it shares exactly the same Container instance.

[odrotbohm]

In my particular case, it was code in a destruction callback that caused the problem. Those callbacks are only triggered on JVM shutdown. If the service instance is managed by TC and shutdown in the BeforeAllCallback the service will be missing when the AC instances held in the cache are shut down.

The documentation on the fundamental TC support in Boot shows exactly that container consumption programming model. I wonder if we should leave a note there on the potential problem or rather recommend the "container as a bean" model instead right away.

[philwebb]

@odrotbohm Thanks Ollie, I hadn't considered the shutdown side of things. I've opened #35236