Skip to content

Kafka in native-image fails when using SSL bundles #44435

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mhalbritter opened this issue Feb 26, 2025 · 5 comments
Closed

Kafka in native-image fails when using SSL bundles #44435

mhalbritter opened this issue Feb 26, 2025 · 5 comments
Assignees
@mhalbritter
Labels
type: bug A general bug
Milestone
3.3.10

Comments

@mhalbritter
Copy link
Contributor 

Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.] with root cause

org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.
        at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:778) ~[na:na]
        at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:531) ~[na:na]
        at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:524) ~[na:na]
        at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:114) ~[kafka-producer-sample:na]
        at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:134) ~[kafka-producer-sample:na]
        at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:643) ~[na:na]
        at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:295) ~[na:na]
        at org.springframework.kafka.core.DefaultKafkaProducerFactory.createRawProducer(DefaultKafkaProducerFactory.java:944) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.DefaultKafkaProducerFactory.createKafkaProducer(DefaultKafkaProducerFactory.java:826) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.DefaultKafkaProducerFactory.doCreateProducer(DefaultKafkaProducerFactory.java:793) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.DefaultKafkaProducerFactory.createProducer(DefaultKafkaProducerFactory.java:768) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.DefaultKafkaProducerFactory.createProducer(DefaultKafkaProducerFactory.java:762) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.KafkaTemplate.getTheProducer(KafkaTemplate.java:976) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:828) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.KafkaTemplate.observeSend(KafkaTemplate.java:805) ~[kafka-producer-sample:3.3.3]
        at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:582) ~[kafka-producer-sample:3.3.3]
        at com.ks.kafka.sample.service.sender.Sender.send(Sender.java:18) ~[kafka-producer-sample:na]
        at com.ks.kafka.sample.service.MessageService.send(MessageService.java:16) ~[kafka-producer-sample:na]
        at com.ks.kafka.sample.controller.KafkaController.sendMessage(KafkaController.java:18) ~[kafka-producer-sample:na]
        at [email protected]/java.lang.reflect.Method.invoke(Method.java:568) ~[kafka-producer-sample:na]
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:257) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:190) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:986) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:891) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1088) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:978) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:903) ~[kafka-producer-sample:6.2.3]
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:564) ~[kafka-producer-sample:6.0]
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885) ~[kafka-producer-sample:6.2.3]
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658) ~[kafka-producer-sample:6.0]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51) ~[kafka-producer-sample:10.1.36]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[kafka-producer-sample:6.2.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[kafka-producer-sample:6.2.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:114) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[kafka-producer-sample:6.2.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201) ~[kafka-producer-sample:6.2.3]
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116) ~[kafka-producer-sample:6.2.3]
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164) ~[na:na]
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140) ~[na:na]
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167) ~[na:na]
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90) ~[na:na]
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483) ~[kafka-producer-sample:10.1.36]
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115) ~[na:na]
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93) ~[kafka-producer-sample:10.1.36]
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74) ~[na:na]
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344) ~[na:na]
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:397) ~[na:na]
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63) ~[kafka-producer-sample:10.1.36]
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:905) ~[na:na]
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1743) ~[na:na]
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52) ~[kafka-producer-sample:10.1.36]
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1190) ~[na:na]
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) ~[na:na]
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:63) ~[na:na]
        at [email protected]/java.lang.Thread.run(Thread.java:842) ~[kafka-producer-sample:na]
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:885) ~[kafka-producer-sample:na]
        at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:860) ~[kafka-producer-sample:na]

There's a runtime hint missing for org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory.
@mhalbritter mhalbritter added the type: bug A general bug label Feb 26, 2025
@mhalbritter mhalbritter added this to the 3.3.x milestone Feb 26, 2025
@mhalbritter mhalbritter mentioned this issue Feb 26, 2025
Closed
@mhalbritter
Copy link
Contributor Author

Workaround until this is fixed:

class MyHint implements RuntimeHintsRegistrar {
    @Override
    public void registerHints(RuntimeHints hints, ClassLoader classLoader) {
        hints.reflection().registerType(SslBundleSslEngineFactory.class, MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS);
    }
}

and @ImportRuntimeHints(MyHint.class).

@mhalbritter mhalbritter self-assigned this Feb 26, 2025
@mhalbritter mhalbritter mentioned this issue Feb 26, 2025
Closed
@mhalbritter mhalbritter modified the milestones: 3.3.x, 3.3.10 Feb 26, 2025
@patpatpat123
Copy link

Hello @mhalbritter ,

Just wanted to say thank you for this, you beat me to this.
I reached out with the same issue here

reactor/reactor-kafka#418

and here:

#44414 (comment)

I was going to send the minimal https://github.com/patpatpat123/issuekafkasslbundle

And saw this post only after.

Again, thank you for doing this. Good day!

@mhalbritter mhalbritter mentioned this issue Feb 28, 2025
Closed
@patpatpat123
Copy link

patpatpat123 commented Mar 26, 2025 •
edited
Loading

Hello @mhalbritter ,

Apologies for breaking the etiquette and posting on a closed thread.

Just wanted to confirm this fix should be within 3.4.4 right?

Because we just bumped everything to 3.4.4 and still seeing this issue:

i.m.c.i.binder.jvm.JvmGcMetrics : GC notifications will not be available because no GarbageCollectorMXBean of the JVM provides any. GCs=[young generation scavenger, complete scavenger]
WARNING: A restricted method in java.lang.System has been called
WARNING: java.lang.System::loadLibrary has been called by io.netty.util.internal.NativeLibraryUtil in an unnamed module (file:/workspace/com.Application)
WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module
WARNING: Restricted methods will be blocked in a future release unless native access is enabled

Started Application in 0.121 seconds (process running for 0.127)
o.s.c.f.c.c.BeanFactoryAwareFunctionRegistry : Failed to invoke function 'sendToKafka'
org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.
	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:778)
	at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:531)
	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:524)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:114)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:134)
	at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:643)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:295)
	at reactor.kafka.sender.internals.ProducerFactory.createProducer(ProducerFactory.java:34)
	at reactor.kafka.sender.internals.DefaultKafkaSender.lambda$new$2(DefaultKafkaSender.java:102)
	at reactor.core.publisher.MonoCallable.call(MonoCallable.java:72)
	at reactor.core.publisher.FluxSubscribeOnCallable$CallableSubscribeOnSubscription.run(FluxSubscribeOnCallable.java:228)
	at reactor.core.scheduler.ImmediateScheduler.schedule(ImmediateScheduler.java:52)
	at reactor.core.publisher.MonoSubscribeOnCallable.subscribe(MonoSubscribeOnCallable.java:52)
	at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8876)
	at reactor.core.publisher.Flux.subscribeWith(Flux.java:9012)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8856)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8780)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8698)
	at com.Service.run(Service.java:53)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:788)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:86)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:796)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:787)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:772)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:186)
	at java.base@24/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base@24/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:571)
	at java.base@24/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:153)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:176)
	at java.base@24/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base@24/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:325)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1361)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1350)
	at com.Application.main(Application.java:18)
	at java.base@24/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)


reactor.core.Exceptions$ErrorCallbackNotImplemented: org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.
Caused by: org.apache.kafka.common.config.ConfigException: Invalid value org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory for configuration ssl.engine.factory.class: Class org.springframework.boot.autoconfigure.kafka.SslBundleSslEngineFactory could not be found.
	at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:778)
	at org.apache.kafka.common.config.ConfigDef.parseValue(ConfigDef.java:531)
	at org.apache.kafka.common.config.ConfigDef.parse(ConfigDef.java:524)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:114)
	at org.apache.kafka.common.config.AbstractConfig.<init>(AbstractConfig.java:134)
	at org.apache.kafka.clients.producer.ProducerConfig.<init>(ProducerConfig.java:643)
	at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:295)
	at reactor.kafka.sender.internals.ProducerFactory.createProducer(ProducerFactory.java:34)
	at reactor.kafka.sender.internals.DefaultKafkaSender.lambda$new$2(DefaultKafkaSender.java:102)
	at reactor.core.publisher.MonoCallable.call(MonoCallable.java:72)
	at reactor.core.publisher.FluxSubscribeOnCallable$CallableSubscribeOnSubscription.run(FluxSubscribeOnCallable.java:228)
	at reactor.core.scheduler.ImmediateScheduler.schedule(ImmediateScheduler.java:52)
	at reactor.core.publisher.MonoSubscribeOnCallable.subscribe(MonoSubscribeOnCallable.java:52)
	at reactor.core.publisher.MonoCacheTime.subscribeOrReturn(MonoCacheTime.java:143)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8876)
	at reactor.core.publisher.Flux.subscribeWith(Flux.java:9012)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8856)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8780)
	at reactor.core.publisher.Flux.subscribe(Flux.java:8698)
	at com.Service.run(Service.java:53)
	at org.springframework.boot.SpringApplication.lambda$callRunner$5(SpringApplication.java:788)
	at org.springframework.util.function.ThrowingConsumer$1.acceptWithException(ThrowingConsumer.java:82)
	at org.springframework.util.function.ThrowingConsumer.accept(ThrowingConsumer.java:60)
	at org.springframework.util.function.ThrowingConsumer$1.accept(ThrowingConsumer.java:86)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:796)
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:787)
	at org.springframework.boot.SpringApplication.lambda$callRunners$3(SpringApplication.java:772)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:186)
	at java.base@24/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
	at java.base@24/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:571)
	at java.base@24/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:153)
	at java.base@24/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:176)
	at java.base@24/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base@24/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:636)
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:325)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1361)
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1350)
	at com.Application.main(Application.java:18)
	at java.base@24/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

Wondering what did I do wrong. I did not apply (just bump the version and built a native image)

<plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
                <configuration>
                    <image>
                        <env>
                            <BP_JVM_VERSION>24</BP_JVM_VERSION>
                            <BP_NATIVE_IMAGE>TRUE</BP_NATIVE_IMAGE>
                        </env>
                        <buildpacks>
                            <buildpack>paketobuildpacks/java-native-image:11.9.0</buildpack>
                        </buildpacks>
                    </image>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.graalvm.buildtools</groupId>
                <artifactId>native-maven-plugin</artifactId>
            </plugin>

To add more details, GraalVM24. 3.4.4 NON native image will work fine.

Once it is native image GraalVm24 + 3.4.4, exception is reproducible

@mhalbritter
Copy link
Contributor Author

Yeah, that fix is in 3.4.4.

I just looked at your sample and you're using reactor-kafka. The fix I applied was to register the runtime hints on KafkaAutoConfiguration, which doesn't apply to your case. Spring Boot has no auto-configuration for reactor-kafka, so you have to apply the workaround yourself.

@patpatpat123
Copy link

I see. Understood. Many thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mhalbritter mhalbritter

Labels
type: bug A general bug
Projects
None yet
Milestone
3.3.10
Development

No branches or pull requests
2 participants
@patpatpat123 @mhalbritter