conda: add _GLIBCXX_USE_CXX11_ABI=1 support for linux cpu wheel

[zhuhong61]

The target of this commit is to add the support for a new linux
cpu conda wheel file built with _GLIBCXX_USE_CXX11_ABI=1.

we add a Dockerfile (conda/Dockerfile_cxx11-abi) with CentOS8 as base image to
support CXX11_ABI=1.

To build the new docker image with CXX11_ABI support, run:
CUDA_VERSION=cpu-cxx11-abi conda/build_docker.sh
or
conda/build_all_docker.sh

To build a linux cpu conda wheel with CXX11_ABI within this image, run:
// the below settings are special for this image
export DESIRED_CUDA=cpu-cxx11-abi # change from cpu for wheel name
export GPU_ARCH_TYPE=cpu-cxx11-abi # change from cpu for build.sh
export DOCKER_IMAGE=pytorch/conda-builder:cpu-cxx11-abi
export DESIRED_DEVTOOLSET=cxx11-abi

// the below settings are as usual
export BINARY_ENV_FILE=/tmp/env
export BUILDER_ROOT=/builder
export DESIRED_PYTHON=3.7 # or 3.8, 3.9, etc.
export IS_GHA=1
export PACKAGE_TYPE=conda
export PYTORCH_FINAL_PACKAGE_DIR=/artifacts
export PYTORCH_ROOT=/pytorch
export GITHUB_WORKSPACE=/your_path_to_workspace

// the '-e DESIRED_DEVTOOLSET' below is newly added for this container,
// others are as usual
set -x
mkdir -p artifacts/
container_name=$(docker run
-e BINARY_ENV_FILE
-e BUILDER_ROOT
-e DESIRED_CUDA
-e DESIRED_PYTHON
-e GPU_ARCH_TYPE
-e IS_GHA
-e PACKAGE_TYPE
-e PYTORCH_FINAL_PACKAGE_DIR
-e PYTORCH_ROOT
-e DOCKER_IMAGE
-e DESIRED_DEVTOOLSET
--tty
--detach
-v "${GITHUB_WORKSPACE}/pytorch:/pytorch"
-v "${GITHUB_WORKSPACE}/builder:/builder"
-v "${RUNNER_TEMP}/artifacts:/artifacts"
-w /
"${DOCKER_IMAGE}"
)

// build conda wheel as usual,
// and the built wheel file name looks like: pytorch-1.13.0.dev20220616-py3.7_cpu_0.tar.bz2
docker exec -t -w "${PYTORCH_ROOT}" "${container_name}" bash -c "bash .circleci/scripts/binary_populate_env.sh"
docker exec -t "${container_name}" bash -c "source ${BINARY_ENV_FILE} && bash /builder/conda/build.sh"

// to verify the built wheel file, we'll see 'True'
$ conda install pytorch-1.13.0.dev20220616-py3.7_cpu_0.tar.bz2
$ python -c 'import torch; print(torch._C._GLIBCXX_USE_CXX11_ABI)'
True

Co-authored-by: Zhu Hong [email protected]
Co-authored-by: Guo Yejun [email protected]

[gujinghui]

Hi @malfet
Could you help review? Thanks.

[powderluv]

Can we also please add a manywheel package with similar CXX11 abi ?

[zhuhong61]

Can we also please add a manywheel package with similar CXX11 abi ?

Yes, we have added the manywheel package for CXX11 abi. Please see #990 and #1023, this two PRs have been merged.

[powderluv]

@zhuhong61 thanks thanks. Are the wheels being published ?

[zhuhong61]

@zhuhong61 thanks thanks. Are the wheels being published ?

@powderluv Sorry. I don't really understand what does the "publish" mean by. To my understandings, the manywheel package has not been pulished on pytorch offical website. Now we need to add new checks in CI system for github/pytorch/pytorch to verify the manywheel package with cxx11-abi (#990 (comment)), and we are waiting for the review of the PR: pytorch/pytorch#79409.

[atalman]

cc @malfet @zhuhong61 looks like this commit breaks mac nightly conda builds https://hud.pytorch.org/hud/pytorch/pytorch/nightly

I see this error when inspecting the cmake command

-- Building version 1.13.0.dev20220715
cmake -DBUILD_PREFIX=$BUILD_PREFIX -DBUILD_PYTHON=True -DBUILD_TEST=False -DCMAKE_ARGS= -DCMAKE_BUILD_TYPE=Release -DCMAKE_GENERATOR=Unix Makefiles -DCMAKE_INSTALL_PREFIX=$SRC_DIR/torch -DCMAKE_LIBRARY_PATH=$PREFIX/lib:$PREFIX/include: -DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_PREFIX_PATH=$PREFIX/lib/python3.10/site-packages;$PREFIX -DGLIBCXX_USE_CXX11_ABI=0 -DINSTALL_TEST=0 -DNUMPY_INCLUDE_DIR=$PREFIX/lib/python3.10/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=$PREFIX/bin/python -DPYTHON_INCLUDE_DIR=$PREFIX/include/python3.10 -DPYTHON_LIBRARY=$PREFIX/lib/libpython3.10.a -DTORCH_BUILD_VERSION=1.13.0.dev20220715 -DUSE_COREML_DELEGATE=1 -DUSE_CUDA=0 -DUSE_DISTRIBUTED=1 -DUSE_FBGEMM=1 -DUSE_GLOO_WITH_OPENSSL=ON -DUSE_LLVM=$PREFIX -DUSE_MKLDNN=OFF -DUSE_NINJA=OFF -DUSE_NUMPY=True -DUSE_PYTORCH_METAL_EXPORT=1 -DUSE_QNNPACK=OFF $SRC_DIR
Error: Process completed with exit code 1.

This is working command from before

2022-07-13T10:24:17.0608810Z cmake -DBUILD_PREFIX=$BUILD_PREFIX -DBUILD_PYTHON=True -DBUILD_TEST=True -DCMAKE_ARGS= -DCMAKE_BUILD_TYPE=Release -DCMAKE_GENERATOR=Unix Makefiles -DCMAKE_INSTALL_PREFIX=$SRC_DIR/torch -DCMAKE_LIBRARY_PATH=$PREFIX/lib:$PREFIX/include: -DCMAKE_PREFIX_PATH=$PREFIX/lib/python3.9/site-packages;$PREFIX -DINSTALL_TEST=0 -DNUMPY_INCLUDE_DIR=$PREFIX/lib/python3.9/site-packages/numpy/core/include -DPYTHON_EXECUTABLE=$PREFIX/bin/python -DPYTHON_INCLUDE_DIR=$PREFIX/include/python3.9 -DPYTHON_LIBRARY=$PREFIX/lib/libpython3.9.a -DTORCH_BUILD_VERSION=1.13.0.dev20220713 -DUSE_COREML_DELEGATE=1 -DUSE_CUDA=0 -DUSE_DISTRIBUTED=1 -DUSE_FBGEMM=1 -DUSE_GLOO_WITH_OPENSSL=ON -DUSE_LLVM=$PREFIX -DUSE_NINJA=OFF -DUSE_NUMPY=True -DUSE_PYTORCH_METAL_EXPORT=1 $SRC_DIR
2022-07-13T10:24:17.0638060Z cmake --build . --target install --config Release -- -j 3

[malfet]

Thank you for the investigation @atalman
Reverted

[zhuhong61]

Thanks @atalman @malfet

1. I'm sorry I can not reproduce this issue from my side. Could you please provide more details for reproducing?
   this is my cmake command:
   
2. I this maybe the difference comes from -DUSE_LLVM? My flag is -DUSE_LLVM=/opt/llvm_no_cxx11_abi, and your command is -DUSE_LLVM=$PREFIX, could you please provide the version llvm used in you environment?

Attached:
Here is the steps I used:
Docker build:
CUDA_VERSION=cpu conda/build_docker.sh

ENV set:

export BINARY_ENV_FILE=/tmp/env
export BUILDER_ROOT=/builder
export DESIRED_CUDA=cpu
export DESIRED_PYTHON=3.10
export GPU_ARCH_TYPE=cpu
export IS_GHA=1
export PACKAGE_TYPE=conda
export PYTORCH_FINAL_PACKAGE_DIR=/artifacts
export PYTORCH_ROOT=/pytorch
export DOCKER_IMAGE=pytorch/conda-builder:cpu

create a container:

set -x
mkdir -p artifacts/
container_name=$(docker run \
  -e BINARY_ENV_FILE \
  -e BUILDER_ROOT \
  -e DESIRED_CUDA \
  -e DESIRED_DEVTOOLSET \
  -e DESIRED_PYTHON \
  -e GPU_ARCH_TYPE \
  -e IS_GHA \
  -e PACKAGE_TYPE \
  -e PYTORCH_FINAL_PACKAGE_DIR \
  -e DOCKER_IMAGE \
  -e PYTORCH_ROOT \
   --tty \
   --detach \
   -v "$(pwd)/pytorch:/pytorch" \
   -v "$(pwd)/builder:/builder" \
   -v "$(pwd)/artifacts:/artifacts" \
   -w / \
   "${DOCKER_IMAGE}"
      )

Run building command:

docker exec -t -w "${PYTORCH_ROOT}" "${container_name}" bash -c "bash .circleci/scripts/binary_populate_env.sh"
docker exec -t "${container_name}" bash -c "source ${BINARY_ENV_FILE} && bash /builder/conda/build.sh"

[zhuhong61]

Hi @atalman

1. I found two build tasks in https://hud.pytorch.org/pytorch/pytorch/commit/888ece62d5b626455b123ff905a019811e46ea74
   The first one failed:
   
   And the second one succeeded:
   
   These two tasks have the same name: conda-py3_10-cpu-build, what's the differences between them? Could you help provide more information?
2. I checked the build log in failed job:
   
   In line 989 and 990, could you please help provide these two logs $SRC_DIR/build/CMakeFiles/CMakeOutput.log and $SRC_DIR/build/CMakeFiles/CMakeError.log for us to check the issue?

Thanks!