Skip to content

YARN-10503. Support queue capacity in terms of absolute resources wit… #3814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

YARN-10503. Support queue capacity in terms of absolute resources wit… #3814

wants to merge 1 commit into from

Conversation

brumi1024
Copy link
Member

…h custom

resourceType. Contributed by Qi Zhu.

Description of PR

How was this patch tested?

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 48s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.2 Compile Tests _
+0 🆗 mvndep 3m 26s Maven dependency ordering for branch
+1 💚 mvninstall 24m 5s branch-3.2 passed
+1 💚 compile 7m 41s branch-3.2 passed
+1 💚 checkstyle 1m 11s branch-3.2 passed
+1 💚 mvnsite 1m 43s branch-3.2 passed
+1 💚 javadoc 1m 21s branch-3.2 passed
+1 💚 spotbugs 3m 31s branch-3.2 passed
+1 💚 shadedclient 13m 59s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 24s Maven dependency ordering for patch
-1 ❌ mvninstall 0m 36s /patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌ compile 2m 30s /patch-compile-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch failed.
-1 ❌ javac 2m 30s /patch-compile-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 58s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 38 unchanged - 0 fixed = 39 total (was 38)
-1 ❌ mvnsite 0m 38s /patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚 xml 0m 2s The patch has no ill-formed XML file.
+1 💚 javadoc 1m 13s the patch passed
-1 ❌ spotbugs 0m 44s /patch-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
-1 ❌ shadedclient 6m 55s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 41s hadoop-yarn-api in the patch passed.
-1 ❌ unit 0m 42s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch failed.
+1 💚 asflicense 0m 31s The patch does not generate ASF License warnings.
76m 40s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/1/artifact/out/Dockerfile
GITHUB PR #3814
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname Linux bf4a46b24240 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.2 / 60f480516d06fa2a817ddfa153fece21442c21c4
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/1/testReport/
Max. process+thread count 411 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 44s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.2 Compile Tests _
+0 🆗 mvndep 3m 31s Maven dependency ordering for branch
+1 💚 mvninstall 23m 33s branch-3.2 passed
+1 💚 compile 7m 15s branch-3.2 passed
+1 💚 checkstyle 1m 15s branch-3.2 passed
+1 💚 mvnsite 1m 49s branch-3.2 passed
+1 💚 javadoc 1m 30s branch-3.2 passed
+1 💚 spotbugs 3m 36s branch-3.2 passed
+1 💚 shadedclient 14m 26s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 25s Maven dependency ordering for patch
+1 💚 mvninstall 1m 23s the patch passed
+1 💚 compile 6m 30s the patch passed
+1 💚 javac 6m 30s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 12s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 38 unchanged - 0 fixed = 39 total (was 38)
+1 💚 mvnsite 1m 40s the patch passed
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+1 💚 javadoc 1m 15s the patch passed
+1 💚 spotbugs 3m 48s the patch passed
+1 💚 shadedclient 15m 19s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 55s hadoop-yarn-api in the patch passed.
-1 ❌ unit 75m 14s /patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 48s The patch does not generate ASF License warnings.
166m 45s
Reason Tests
Failed junit tests hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCSAllocateCustomResource
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/2/artifact/out/Dockerfile
GITHUB PR #3814
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname Linux b7dfe7d89322 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.2 / f48feb2dba948fe0803ffdf727a56657a3dfd9c3
Default Java Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/2/testReport/
Max. process+thread count 812 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/2/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@ericbadger @brumi1024
YARN-10503. Support queue capacity in terms of absolute resources wit…
2ef34e2 
…h custom

resourceType. Contributed by Qi Zhu.
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 11m 12s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.2 Compile Tests _
+0 🆗 mvndep 3m 20s Maven dependency ordering for branch
+1 💚 mvninstall 26m 2s branch-3.2 passed
+1 💚 compile 7m 22s branch-3.2 passed
+1 💚 checkstyle 1m 13s branch-3.2 passed
+1 💚 mvnsite 1m 36s branch-3.2 passed
+1 💚 javadoc 1m 13s branch-3.2 passed
+1 💚 spotbugs 3m 28s branch-3.2 passed
+1 💚 shadedclient 15m 53s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for patch
+1 💚 mvninstall 1m 20s the patch passed
+1 💚 compile 6m 46s the patch passed
+1 💚 javac 6m 46s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 1m 9s /results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt hadoop-yarn-project/hadoop-yarn: The patch generated 1 new + 38 unchanged - 0 fixed = 39 total (was 38)
+1 💚 mvnsite 1m 28s the patch passed
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+1 💚 javadoc 1m 4s the patch passed
+1 💚 spotbugs 3m 44s the patch passed
+1 💚 shadedclient 16m 42s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 0m 47s hadoop-yarn-api in the patch passed.
+1 💚 unit 77m 11s hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚 asflicense 0m 40s The patch does not generate ASF License warnings.
182m 54s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/3/artifact/out/Dockerfile
GITHUB PR #3814
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname Linux 3a56982ce4f7 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.2 / 2ef34e2
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/3/testReport/
Max. process+thread count 848 (vs. ulimit of 5500)
modules C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3814/3/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@szilard-nemeth
Copy link
Contributor

Thanks @brumi1024 for backporting this.
LGTM, committed to branch-3.2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@brumi1024 @hadoop-yetus @szilard-nemeth @ericbadger