-
Notifications
You must be signed in to change notification settings - Fork 364
feat: C++ runtime on Windows #2806
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
5bcca38
feat: C++ runtime on Windows
HolyWu c6de545
Add cu124 for TRT_DOWNLOAD_LINK
HolyWu 63c7e76
Update vc_env_helper.bat
HolyWu 822265a
Add wheel-build-params
HolyWu 0ef76e3
Migrate to G5 instance in test
HolyWu 4b6f806
Use local file to actually run on G5 instance before merging
HolyWu e238acf
Migrate to G5 instance in build
HolyWu 69bdbab
Update build_wheels_windows.yml
HolyWu f239353
Fix ABI incompatibility with pybind11
HolyWu ac8a92b
Fix FileNotFoundError on Windows in test_export_serde
HolyWu ed8afa8
Specify runner for test-infra's build_wheels_windows
HolyWu bbb3dab
Use env var instead of wheel-build-params for release flag
HolyWu 9729183
Add -o option to unzip
HolyWu 786fbc9
Use fixed version of transformers in tests
HolyWu 411a223
Add test_dyn_models
HolyWu 087b5cd
Use requirements.txt in tests
HolyWu d1497e6
Keep up with the changes in main
HolyWu 339d3e9
Unify install-torch-tensorrt script for Linux and Windows
HolyWu 3fb0b1e
Simply use TensorRT cuda 12.4 for building
HolyWu 0e43822
Add trt-bindings and trt-libs to requirements
HolyWu 60500c5
Use pypi.nvidia.com instead of pypi.ngc.nvidia.com for extra-index-url
HolyWu ac07bb1
Use windows.8xlarge.nvidia.gpu.nonephemeral runner in test
HolyWu c2a80c1
Sort packages in requirements alphabetically
HolyWu 07c0ac9
Don't install NumPy 2.0 rc because it breaks CI
HolyWu d522edf
Remove redundant CONDA_RUN in tests
HolyWu aa31125
assertEquals is removed in Python 3.12
HolyWu 6e876b8
Fix potential torchvision error due to dependency conflict
HolyWu 19e2eed
Address review comment
HolyWu 6ceda35
Use tempdir in test_export_serde
HolyWu 11d0293
Add back CUDA version specialization for bindings and libs
HolyWu 403ebc9
Move trt_bindings and trt_libs to pyproject.toml
HolyWu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,18 @@ | ||
| #!/usr/bin/env bash | ||
| set -eou pipefail | ||
| # Source conda so it's available to the script environment | ||
| source ${BUILD_ENV_FILE} | ||
| export EXTRA_INDEX_URL="https://download.pytorch.org/whl/nightly/${CU_VERSION}" | ||
|
|
||
| TORCH_TORCHVISION=$(grep "^torch" ${PWD}/py/requirements.txt) | ||
| INDEX_URL=https://download.pytorch.org/whl/${CHANNEL}/${CU_VERSION} | ||
| PLATFORM=$(python -c "import sys; print(sys.platform)") | ||
|
|
||
| # Install all the dependencies required for Torch-TensorRT | ||
| ${CONDA_RUN} pip install --pre -r ${PWD}/tests/py/requirements.txt --use-deprecated=legacy-resolver --extra-index-url=${EXTRA_INDEX_URL} | ||
| pip install --pre ${TORCH_TORCHVISION} --index-url ${INDEX_URL} | ||
| pip install --pre -r ${PWD}/tests/py/requirements.txt --use-deprecated legacy-resolver | ||
|
|
||
| # Install Torch-TensorRT | ||
| ${CONDA_RUN} pip install /opt/torch-tensorrt-builds/torch_tensorrt*.whl | ||
| if [[ ${PLATFORM} == win32 ]]; then | ||
| pip install ${RUNNER_ARTIFACT_DIR}/torch_tensorrt*.whl | ||
| else | ||
| pip install /opt/torch-tensorrt-builds/torch_tensorrt*.whl | ||
| fi | ||
|
|
||
| echo -e "Running test script"; | ||
| echo -e "Running test script"; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,12 +1,18 @@ | ||
| python -m pip install -U numpy packaging pyyaml setuptools wheel | ||
| set -eou pipefail | ||
|
|
||
| # Install TRT from PyPi | ||
| TRT_VERSION=$(${CONDA_RUN} python -c "import yaml; print(yaml.safe_load(open('dev_dep_versions.yml', 'r'))['__tensorrt_version__'])") | ||
| pip install -U numpy packaging pyyaml setuptools wheel | ||
|
|
||
| python -m pip install tensorrt==${TRT_VERSION} tensorrt-${CU_VERSION::4}==${TRT_VERSION} tensorrt-${CU_VERSION::4}-bindings==${TRT_VERSION} tensorrt-${CU_VERSION::4}-libs==${TRT_VERSION} --extra-index-url https://pypi.nvidia.com | ||
| # Install TRT from PyPI | ||
| TRT_VERSION=$(python -c "import yaml; print(yaml.safe_load(open('dev_dep_versions.yml', 'r'))['__tensorrt_version__'])") | ||
| pip install tensorrt==${TRT_VERSION} tensorrt-${CU_VERSION::4}-bindings==${TRT_VERSION} tensorrt-${CU_VERSION::4}-libs==${TRT_VERSION} --extra-index-url https://pypi.nvidia.com | ||
|
|
||
| choco install bazelisk -y | ||
|
|
||
| curl -Lo TensorRT.zip https://developer.download.nvidia.com/compute/machine-learning/tensorrt/10.0.1/zip/TensorRT-10.0.1.6.Windows10.win10.cuda-12.4.zip | ||
| unzip -o TensorRT.zip -d C:/ | ||
|
|
||
| export CUDA_HOME="$(echo ${CUDA_PATH} | sed -e 's#\\#\/#g')" | ||
|
|
||
| cat toolchains/ci_workspaces/WORKSPACE.win.release.tmpl | envsubst > WORKSPACE | ||
|
|
||
| echo "RELEASE=1" >> ${GITHUB_ENV} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.