Skip to content

[test] Use LLM API for Nemotron-H correctness test #5097

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Jun 12, 2025
Merged

[test] Use LLM API for Nemotron-H correctness test #5097

Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
c70cd1a
Add unittest for Nemotron-H using the pytroch LLM API, that currently…
tomeras91 Jun 4, 2025
1e5b107
use pytest instead of unittest in Nemotron-H correctness test
tomeras91 Jun 4, 2025
b650a85
Fix Nemotron-H LLM API test - (1) call shutdown() regardless if test …
tomeras91 Jun 4, 2025
966f3a7
Deal with warmup requests better in Mamba2Mixer forward - assign dumm…
tomeras91 Jun 4, 2025
02d13c7
Merge branch 'NVIDIA:main' into fix-nemotron-h-warmup
tomeras91 Jun 5, 2025
61c3db8
Merge branch 'main' into fix-nemotron-h-warmup
tomeras91 Jun 5, 2025
cb80c01
Merge branch 'fix-nemotron-h-warmup' of github.com:tomeras91/TensorRT…
tomeras91 Jun 5, 2025
8d3d4a9
Merge branch 'main' into fix-nemotron-h-warmup
tomeras91 Jun 8, 2025
a19bf8c
Merge branch 'main' into fix-nemotron-h-warmup
tomeras91 Jun 8, 2025
21e78c7
Merge branch 'main' into fix-nemotron-h-warmup
tomeras91 Jun 9, 2025
02f0817
clear memory between tests to avoid OOM on A30
tomeras91 Jun 9, 2025
b3bc5e1
Merge branch 'main' into fix-nemotron-h-warmup
tomeras91 Jun 9, 2025
2ffed74
Merge branch 'main' into llm-api-for-nemotron-h-correctness-test
tomeras91 Jun 10, 2025
64b1a6d
WIP: Add correctness test using the LLM API
tomeras91 Jun 10, 2025
7023357
fix: position_ids was off by one during manual decode. Fix + update r…
tomeras91 Jun 10, 2025
61c919c
update reference logprobs in llm api test as well. Now it passes
tomeras91 Jun 10, 2025
c06a384
remove old correctness test and llm api test. Now correctness test us…
tomeras91 Jun 10, 2025
54c3bd8
remove debug prints
tomeras91 Jun 10, 2025
8280bb8
Merge branch 'main' into llm-api-for-nemotron-h-correctness-test
tomeras91 Jun 10, 2025
2b92abb
clear torch cuda cache before test since it's memory intensive on A30…
tomeras91 Jun 10, 2025
924251a
Merge branch 'main' into llm-api-for-nemotron-h-correctness-test
tomeras91 Jun 11, 2025
2e4f1b3
Merge branch 'main' into llm-api-for-nemotron-h-correctness-test
tomeras91 Jun 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading