HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141. #7401

eshwitha · 2025-02-18T16:06:25Z

Description of PR

JIRA: HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141.

Upgrade Kafka to 3.9.0 to fix CVE.

How was this patch tested?

Built on local, ran the UTs on local for hadoop-tools, hadoop-kafka.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

pjfanning · 2025-02-18T18:13:21Z

@eshwitha you need to update

hadoop/LICENSE-binary

Line 321 in 22b8fcd

org.apache.kafka:kafka-clients:3.4.0

pjfanning · 2025-02-18T18:15:31Z

CVE-2024-31141

hadoop-yetus · 2025-02-19T03:37:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	37m 41s		trunk passed
+1 💚	compile	16m 49s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	14m 47s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	mvnsite	23m 29s		trunk passed
+1 💚	javadoc	10m 0s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 26s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	55m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	35m 6s		the patch passed
+1 💚	compile	17m 41s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	17m 41s		the patch passed
+1 💚	compile	17m 19s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	17m 19s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	19m 36s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	9m 57s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 25s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	shadedclient	56m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	383m 0s		root in the patch passed.
+1 💚	asflicense	1m 22s		The patch does not generate ASF License warnings.
		689m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/1/artifact/out/Dockerfile
GITHUB PR	#7401
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint shellcheck shelldocs
uname	Linux e405033e81b1 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `1a1b864`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/1/testReport/
Max. process+thread count	2797 (vs. ulimit of 5500)
modules	C: hadoop-project . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

eshwitha · 2025-02-19T03:40:20Z

@eshwitha you need to update

hadoop/LICENSE-binary

Line 321 in 22b8fcd

org.apache.kafka:kafka-clients:3.4.0

@pjfanning I’ve already updated the same.

eshwitha · 2025-02-19T04:46:55Z

@pjfanning Updated the patch to address the system property setting.

hadoop-yetus · 2025-02-19T17:14:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 50s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	6m 20s		Maven dependency ordering for branch
+1 💚	mvninstall	36m 12s		trunk passed
+1 💚	compile	16m 47s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	14m 51s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	4m 44s		trunk passed
+1 💚	mvnsite	21m 48s		trunk passed
+1 💚	javadoc	9m 59s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 26s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+0 🆗	spotbugs	0m 20s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	68m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 34s		Maven dependency ordering for patch
+1 💚	mvninstall	35m 45s		the patch passed
+1 💚	compile	16m 36s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	16m 36s		the patch passed
+1 💚	compile	15m 0s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	15m 0s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 31s		the patch passed
+1 💚	mvnsite	19m 55s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	9m 53s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	7m 23s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+0 🆗	spotbugs	0m 19s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	69m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	384m 40s		root in the patch passed.
+1 💚	asflicense	1m 21s		The patch does not generate ASF License warnings.
		760m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/2/artifact/out/Dockerfile
GITHUB PR	#7401
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle shellcheck shelldocs
uname	Linux 2f10771cf296 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e24403a`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/2/testReport/
Max. process+thread count	3136 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-tools/hadoop-kafka . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7401/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-02-20T04:47:29Z

@eshwitha Thanks for the contribution! @pjfanning Do you have any other suggestions?

pjfanning · 2025-02-20T10:24:47Z

@eshwitha Thanks for the contribution! @pjfanning Do you have any other suggestions?

@slfan1989 seems ok to me

slfan1989 · 2025-02-20T10:30:51Z

@eshwitha Thanks for the contribution! @pjfanning Thanks for the review!

slfan1989 · 2025-02-20T10:31:21Z

@eshwitha Can we submit another PR to branch-3.4?

eshwitha · 2025-02-20T13:38:46Z

@eshwitha Can we submit another PR to branch-3.4?

Sure @slfan1989, raised the PR for branch-3.4: Github Pull Request #7416

@slfan1989, @pjfanning Thanks for the review!

) Contributed by Palakur Eshwitha Sai. Reviewed-by: PJ Fanning <[email protected]> Signed-off-by: Shilun Fan <[email protected]>

… (apache#7401) Contributed by Palakur Eshwitha Sai. Reviewed-by: PJ Fanning <[email protected]> Signed-off-by: Shilun Fan <[email protected]>

github-actions bot added build trunk labels Feb 18, 2025

HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141.

e24403a

eshwitha force-pushed the HADOOP-19456 branch from 1a1b864 to e24403a Compare February 19, 2025 04:32

github-actions bot added the TOOLS label Feb 19, 2025

slfan1989 approved these changes Feb 20, 2025

View reviewed changes

slfan1989 merged commit ec6c08b into apache:trunk Feb 20, 2025
1 of 3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141. #7401

HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141. #7401

Uh oh!

eshwitha commented Feb 18, 2025 •

edited by slfan1989

Loading

Uh oh!

pjfanning commented Feb 18, 2025

Uh oh!

pjfanning commented Feb 18, 2025

Uh oh!

hadoop-yetus commented Feb 19, 2025

Uh oh!

eshwitha commented Feb 19, 2025

Uh oh!

eshwitha commented Feb 19, 2025

Uh oh!

hadoop-yetus commented Feb 19, 2025

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

pjfanning commented Feb 20, 2025

Uh oh!

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

eshwitha commented Feb 20, 2025

Uh oh!

Uh oh!

HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141. #7401

HADOOP-19456. Upgrade kafka to 3.9.0 to fix CVE-2024-31141. #7401

Uh oh!

Conversation

eshwitha commented Feb 18, 2025 • edited by slfan1989 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

pjfanning commented Feb 18, 2025

Uh oh!

pjfanning commented Feb 18, 2025

Uh oh!

hadoop-yetus commented Feb 19, 2025

Uh oh!

eshwitha commented Feb 19, 2025

Uh oh!

eshwitha commented Feb 19, 2025

Uh oh!

hadoop-yetus commented Feb 19, 2025

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

pjfanning commented Feb 20, 2025

Uh oh!

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

slfan1989 commented Feb 20, 2025

Uh oh!

eshwitha commented Feb 20, 2025

Uh oh!

Uh oh!

eshwitha commented Feb 18, 2025 •

edited by slfan1989

Loading