Add smoke test for binary package #442

guyang3532 · 2020-05-11T03:18:38Z

No description provided.

peterjc123

Generally lgtm, just some nits.

peterjc123 · 2020-05-12T03:37:24Z

windows/internal/smoke_test.bat

+set LIB=%LIB%;%install_root%/lib
+set PATH=%PATH%;%install_root%/lib
+
+>example-app.cpp (


nit: Consider creating a new file?

And the name of the executable is not clear.

peterjc123 · 2020-05-12T03:38:59Z

windows/internal/smoke_test.bat

+.\example-app.exe
+if ERRORLEVEL 1 exit /b 1
+
+echo Checking that MKL is available


peterjc123 · 2020-05-12T03:39:22Z

windows/internal/smoke_test.bat

+
+del example-app.cpp
+>example-app.cpp (
+echo #include ^<torch/torch.h^>


peterjc123 · 2020-05-12T03:41:42Z

windows/internal/smoke_test.bat

+conda create -qyn testenv python=%DESIRED_PYTHON%
+source activate testenv >/dev/null
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *.tar.bz2') do conda install -y "%%i" --offline
+conda install -yq future numpy protobuf six


nit: Check the exitcode here?

peterjc123 · 2020-05-12T03:41:48Z

windows/internal/smoke_test.bat

+set CUDA_VER_MAJOR=%CUDA_VERSION:~0,-1%
+set CUDA_VER_MINOR=%CUDA_VERSION:~-1,1%
+set CUDA_VERSION_STR=%CUDA_VER_MAJOR%.%CUDA_VER_MINOR%
+conda install -yq -c pytorch "cudatoolkit=%CUDA_VERSION_STR%"


nit: Check the exitcode here?

peterjc123 · 2020-05-12T03:44:26Z

windows/internal/smoke_test.bat

+powershell internal/vs_install.ps1
+if ERRORLEVEL 1 exit /b 1
+
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *-latest.zip') do unzip "%%i" -d tmp


nit: Consider using 7z instead. unzip is only available in Bash. The equivalent line using 7z is 7z x "%%i" -otmp.

peterjc123 · 2020-05-12T03:46:34Z

windows/internal/smoke_test.bat

+set "PATH=%CD%\Python%PYTHON_VERSION%\Scripts;%CD%\Python%PYTHON_VERSION%;%PATH%"
+
+for /F "delims=" %%i in ('where /R "%PYTORCH_FINAL_PACKAGE_DIR:/=\%" *.whl') do pip install "%%i"
+pip install -q future numpy protobuf six "mkl>=2019"


nit: Check exitcode here？

peterjc123 · 2020-05-12T03:48:27Z

windows/internal/smoke_test.bat

+set "PATH=%CONDA_HOME%;%CONDA_HOME%\scripts;%CONDA_HOME%\Library\bin;%PATH%"
+
+conda create -qyn testenv python=%DESIRED_PYTHON%
+source activate testenv >/dev/null


Again this is Bash only, please use call %CONDA_HOME%\condabin\activate.bat testenv > NUL.

peterjc123 · 2020-05-15T05:47:12Z

windows/internal/update_driver.bat

@@ -0,0 +1,4 @@
+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe


nit:

check error code

add --retry option

peterjc123 · 2020-05-15T05:47:27Z

windows/internal/update_driver.bat

+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
+
+start /wait 442.50-tesla-desktop-winserver-2019-2016-international.exe -s -noreboot


nit: check exit code

peterjc123 · 2020-05-15T05:50:11Z

windows/internal/update_driver.bat

@@ -0,0 +1,4 @@
+set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
+curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe


Suggested change

curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

curl --retry 3 -kL "%DRIVER_DOWNLOAD_LINK%" --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

if errorlevel 1 exit /b 1

peterjc123 · 2020-05-16T15:12:28Z

check_binary.sh

@@ -256,8 +256,8 @@ build_and_run_example_cpp () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
-  ./example-app
+  g++ ${BUILDER_ROOT}/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


Why is $1 used here?

Why is $1 used here?
The code file name is passed as argument from outside.

peterjc123 · 2020-05-16T15:13:37Z

windows/internal/driver_update.bat

+if errorlevel 1 exit /b 1
+
+del 442.50-tesla-desktop-winserver-2019-2016-international.exe || ver > NUL
+


nit: empty line?

peterjc123 · 2020-05-16T15:15:01Z

Could you please take some time to review this PR, @seemethere?

peterjc123 · 2020-05-18T03:10:14Z

check_binary.sh

@@ -256,8 +256,8 @@ build_and_run_example_cpp () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
-  ./example-app
+  g++ /builder/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


Wow, just noticed that this is an absolute path.

peterjc123 · 2020-05-18T03:10:24Z

check_binary.sh

@@ -287,7 +287,7 @@ build_example_cpp_with_incorrect_abi () {
  if [ -f "${install_root}/lib/libtorch_cuda.so" ] || [ -f "${install_root}/lib/libtorch_cuda.dylib" ]; then
    TORCH_CUDA_LINK_FLAGS="-ltorch_cuda"
  fi
-  g++ example-app.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o example-app
+  g++ /builder/test_example_code/$1.cpp -I${install_root}/include -I${install_root}/include/torch/csrc/api/include -D_GLIBCXX_USE_CXX11_ABI=$GLIBCXX_USE_CXX11_ABI -std=gnu++14 -L${install_root}/lib ${REF_LIB} ${ADDITIONAL_LINKER_FLAGS} -ltorch $TORCH_CPU_LINK_FLAGS $TORCH_CUDA_LINK_FLAGS $C10_LINK_FLAGS -o $1


guyang3532 force-pushed the binary_test branch 4 times, most recently from 687b13a to 68407ea Compare May 12, 2020 02:55

peterjc123 reviewed May 12, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch 3 times, most recently from cb42869 to 7318044 Compare May 15, 2020 02:39

peterjc123 reviewed May 15, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch 6 times, most recently from 6492ccc to e4a05e3 Compare May 16, 2020 12:18

peterjc123 reviewed May 16, 2020

View reviewed changes

guyang3532 force-pushed the binary_test branch from e4a05e3 to 6c371b8 Compare May 17, 2020 09:10

Add smoke test for binary package

3187392

guyang3532 force-pushed the binary_test branch from 6c371b8 to 3187392 Compare May 17, 2020 13:49

peterjc123 mentioned this pull request May 17, 2020

Add ci binary test for windows pytorch/pytorch#38297

Closed

peterjc123 approved these changes May 17, 2020

View reviewed changes

soumith merged commit 8e5b65f into pytorch:master May 18, 2020

peterjc123 reviewed May 18, 2020

View reviewed changes

peterjc123 mentioned this pull request May 18, 2020

Using relative path to test example code #443

Merged

guyang3532 mentioned this pull request Jul 28, 2020

Add Windows tests for multiple configurations pytorch/pytorch#18319

Closed

malfet mentioned this pull request Jan 18, 2022

locally installed PyTorch upgraded or superseded on windows/macos conda tests pytorch/pytorch#71340

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add smoke test for binary package #442

Add smoke test for binary package #442

guyang3532 commented May 11, 2020

peterjc123 left a comment

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 12, 2020

peterjc123 May 15, 2020

peterjc123 May 15, 2020

peterjc123 May 15, 2020

peterjc123 May 16, 2020

guyang3532 May 17, 2020

peterjc123 May 16, 2020

peterjc123 commented May 16, 2020

peterjc123 May 18, 2020

peterjc123 May 18, 2020

		@@ -0,0 +1,4 @@
		set "DRIVER_DOWNLOAD_LINK=https://s3.amazonaws.com/ossci-windows/442.50-tesla-desktop-winserver-2019-2016-international.exe"
		curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe

	curl -k -L DRIVER_DOWNLOAD_LINK --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
	curl --retry 3 -kL "%DRIVER_DOWNLOAD_LINK%" --output 442.50-tesla-desktop-winserver-2019-2016-international.exe
	if errorlevel 1 exit /b 1

		if errorlevel 1 exit /b 1

		del 442.50-tesla-desktop-winserver-2019-2016-international.exe \|\| ver > NUL

Add smoke test for binary package #442

Add smoke test for binary package #442

Conversation

guyang3532 commented May 11, 2020

peterjc123 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

peterjc123 commented May 16, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment