Skip to content

C++ tests on FFmpeg 6 are broken #518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
NicolasHug opened this issue Feb 24, 2025 · 6 comments · Fixed by #658
Closed

C++ tests on FFmpeg 6 are broken #518

NicolasHug opened this issue Feb 24, 2025 · 6 comments · Fixed by #658
Labels
bug Something isn't working

Comments

@NicolasHug
Copy link
Member

And they've been broken for a few weeks now. For example: https://github.com/pytorch/torchcodec/actions/runs/13496887560/job/37706069174?pr=517

[ 64%] Linking CXX executable VideoDecoderTest
cd /home/runner/work/torchcodec/torchcodec/build_tests/test/decoders && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/VideoDecoderTest.dir/link.txt --verbose=1
/usr/bin/c++  -D_GLIBCXX_USE_CXX11_ABI=1 -g -Wl,--dependency-file=CMakeFiles/VideoDecoderTest.dir/link.d CMakeFiles/VideoDecoderTest.dir/VideoDecoderTest.cpp.o -o VideoDecoderTest   -L/lib/intel64  -L/lib/intel64_win  -L/lib/win-x64  -Wl,-rpath,/lib/intel64:/lib/intel64_win:/lib/win-x64:/home/runner/work/torchcodec/torchcodec/build_tests/src/torchcodec/decoders/_core:/usr/share/miniconda3/envs/test/lib:/usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib ../../src/torchcodec/decoders/_core/libtorchcodec6.so ../../lib/libgtest_main.a /usr/share/miniconda3/envs/test/lib/libavdevice.so /usr/share/miniconda3/envs/test/lib/libavfilter.so /usr/share/miniconda3/envs/test/lib/libavformat.so /usr/share/miniconda3/envs/test/lib/libavcodec.so /usr/share/miniconda3/envs/test/lib/libavutil.so /usr/share/miniconda3/envs/test/lib/libswscale.so /usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib/libtorch.so -Wl,--no-as-needed,"/usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib/libtorch_cpu.so" -Wl,--as-needed /usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib/libc10.so -Wl,--no-as-needed,"/usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib/libtorch.so" -Wl,--as-needed /usr/share/miniconda3/envs/test/lib/python3.12/site-packages/torch/lib/libc10.so /usr/share/miniconda3/envs/test/lib/libpython3.12.so ../../lib/libgtest.a
cd /home/runner/work/torchcodec/torchcodec/build_tests/test/decoders && /usr/local/bin/cmake -D TEST_TARGET=VideoDecoderTest -D TEST_EXECUTABLE=/home/runner/work/torchcodec/torchcodec/build_tests/test/decoders/VideoDecoderTest -D TEST_EXECUTOR= -D TEST_WORKING_DIR=/home/runner/work/torchcodec/torchcodec/build_tests/test/decoders -D TEST_EXTRA_ARGS= -D TEST_PROPERTIES= -D TEST_PREFIX= -D TEST_SUFFIX= -D TEST_FILTER= -D NO_PRETTY_TYPES=FALSE -D NO_PRETTY_VALUES=FALSE -D TEST_LIST=VideoDecoderTest_TESTS -D CTEST_FILE=/home/runner/work/torchcodec/torchcodec/build_tests/test/decoders/VideoDecoderTest[1]_tests.cmake -D TEST_DISCOVERY_TIMEOUT=5 -D TEST_DISCOVERY_EXTRA_ARGS= -D TEST_XML_OUTPUT_DIR= -P /usr/local/share/cmake-3.31/Modules/GoogleTestAddTests.cmake
/home/runner/work/torchcodec/torchcodec/build_tests/test/decoders/VideoDecoderTest: /usr/share/miniconda3/envs/test/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/runner/work/torchcodec/torchcodec/build_tests/test/decoders/VideoDecoderTest)
/home/runner/work/torchcodec/torchcodec/build_tests/test/decoders/VideoDecoderTest: /usr/share/miniconda3/envs/test/lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found (required by /home/runner/work/torchcodec/torchcodec/build_tests/src/torchcodec/decoders/_core/libtorchcodec6.so)
CMake Error at /usr/local/share/cmake-3.31/Modules/GoogleTestAddTests.cmake:132 (message):
  Error running test executable.
@NicolasHug NicolasHug mentioned this issue Feb 24, 2025
Merged
@scotts scotts added the bug Something isn't working label Mar 8, 2025
@traversaro
Copy link
Contributor

Just a curiosity, there is any specific reason why you mix packages installed via defaults/main channel and ffmpeg's installed by conda-forge ? Since a few years mixing defaults/main and conda-forge is not supported (see the docs in https://conda-forge.org/docs/user/transitioning_from_defaults/), as the packages shipped in the two channels can have different ABIs.

Using miniforge in place of miniconda should make sure that you only install packages from conda-forge, avoiding ABI problems like the one tracked in this issue.

@NicolasHug
Copy link
Member Author

Thanks for the info @traversaro , we weren't aware that conda-forge wasn't ABI compatible with the default channels.

I assumed we were installing everything from conda-forge, since we always use conda install -c conda-forge. Is this not working as we thought?

@traversaro
Copy link
Contributor

I assumed we were installing everything from conda-forge, since we always use conda install -c conda-forge. Is this not working as we thought?

Unfortunately not (even if I see why that is confusing). By looking at your CI trace, it seems that you are creating an environment with the defaults channel in the setup conda env step:

2025-02-24T11:24:28.6204484Z [command]/usr/share/miniconda3/condabin/conda create --name test python=3.12
2025-02-24T11:24:29.3632226Z Channels:
2025-02-24T11:24:29.3632763Z  - defaults
2025-02-24T11:24:29.3633490Z Platform: linux-64
2025-02-24T11:24:30.2082953Z Collecting package metadata (repodata.json): ...working... done
2025-02-24T11:24:30.2363477Z Solving environment: ...working... done
2025-02-24T11:24:30.2499283Z 
2025-02-24T11:24:30.2500014Z ## Package Plan ##
2025-02-24T11:24:30.2501021Z 
2025-02-24T11:24:30.2501520Z   environment location: /usr/share/miniconda3/envs/test
2025-02-24T11:24:30.2501975Z 
2025-02-24T11:24:30.2502163Z   added / updated specs:
2025-02-24T11:24:30.2502584Z     - python=3.12
2025-02-24T11:24:30.2502976Z 
2025-02-24T11:24:30.2503498Z 
2025-02-24T11:24:30.2503737Z The following packages will be downloaded:
2025-02-24T11:24:30.2504585Z 
2025-02-24T11:24:30.2504760Z     package                    |            build
2025-02-24T11:24:30.2505282Z     ---------------------------|-----------------
2025-02-24T11:24:30.2505803Z     _libgcc_mutex-0.1          |             main           3 KB
2025-02-24T11:24:30.2506379Z     _openmp_mutex-5.1          |            1_gnu          21 KB
2025-02-24T11:24:30.2506733Z     bzip2-1.0.8                |       h5eee18b_6         262 KB
2025-02-24T11:24:30.2507120Z     ca-certificates-2024.12.31 |       h06a4308_0         128 KB
2025-02-24T11:24:30.2507446Z     expat-2.6.4                |       h6a678d5_0         180 KB
2025-02-24T11:24:30.2507768Z     ld_impl_linux-64-2.40      |       h12ee557_0         710 KB
2025-02-24T11:24:30.2508209Z     libffi-3.4.4               |       h6a678d5_1         141 KB
2025-02-24T11:24:30.2508601Z     libgcc-ng-11.2.0           |       h1234567_1         5.3 MB
2025-02-24T11:24:30.2508912Z     libgomp-11.2.0             |       h1234567_1         474 KB
2025-02-24T11:24:30.2509242Z     libstdcxx-ng-11.2.0        |       h1234567_1         4.7 MB
2025-02-24T11:24:30.2509559Z     libuuid-1.41.5             |       h5eee18b_0          27 KB
2025-02-24T11:24:30.2509863Z     ncurses-6.4                |       h6a678d5_0         914 KB
2025-02-24T11:24:30.2510170Z     openssl-3.0.15             |       h5eee18b_0         5.2 MB
2025-02-24T11:24:30.2510754Z     pip-25.0                   |  py312h06a4308_0         2.8 MB
2025-02-24T11:24:30.2511830Z     python-3.12.9              |       h5148396_0        34.7 MB
2025-02-24T11:24:30.2512426Z     readline-8.2               |       h5eee18b_0         357 KB
2025-02-24T11:24:30.2513080Z     setuptools-75.8.0          |  py312h06a4308_0         2.2 MB
2025-02-24T11:24:30.2514110Z     sqlite-3.45.3              |       h5eee18b_0         1.2 MB
2025-02-24T11:24:30.2514645Z     tk-8.6.14                  |       h39e8969_0         3.4 MB
2025-02-24T11:24:30.2515168Z     tzdata-2025a               |       h04d1e81_0         117 KB
2025-02-24T11:24:30.2515517Z     wheel-0.45.1               |  py312h06a4308_0         147 KB
2025-02-24T11:24:30.2515807Z     xz-5.6.4                   |       h5eee18b_1         567 KB
2025-02-24T11:24:30.2516089Z     zlib-1.2.13                |       h5eee18b_1         111 KB
2025-02-24T11:24:30.2516380Z     ------------------------------------------------------------
2025-02-24T11:24:30.2516651Z                                            Total:        63.6 MB
2025-02-24T11:24:30.2516829Z

then, you install some additional dependencies with the -c conda-forge option, but that just adds the conda-forge channel on top of the channels already configure, so some packages are installed from defaults and some other from conda-forge (see the string after the package size):

2025-02-24T11:24:59.4361388Z ##[group]Run conda install "ffmpeg=6.1.1" pkg-config -c conda-forge
2025-02-24T11:24:59.4361873Z �[36;1mconda install "ffmpeg=6.1.1" pkg-config -c conda-forge�[0m
2025-02-24T11:24:59.4362196Z �[36;1mffmpeg -version�[0m
2025-02-24T11:24:59.4418486Z shell: /usr/bin/bash -l -eo pipefail {0}
2025-02-24T11:24:59.4418797Z env:
2025-02-24T11:24:59.4418993Z   INPUT_RUN_POST: true
2025-02-24T11:24:59.4419229Z   CONDA: /usr/share/miniconda3
2025-02-24T11:24:59.4419508Z   CONDA_PKGS_DIR: /home/runner/conda_pkgs_dir
2025-02-24T11:24:59.4419799Z ##[endgroup]
2025-02-24T11:25:02.6139477Z Channels:
2025-02-24T11:25:02.6139873Z  - conda-forge
2025-02-24T11:25:02.6140196Z  - defaults
2025-02-24T11:25:02.6140483Z Platform: linux-64
2025-02-24T11:25:10.5711613Z Collecting package metadata (repodata.json): ...working... done
2025-02-24T11:25:12.0048794Z Solving environment: ...working... done
2025-02-24T11:25:12.0319010Z 
2025-02-24T11:25:12.0319641Z ## Package Plan ##
2025-02-24T11:25:12.0319988Z 
2025-02-24T11:25:12.0320418Z   environment location: /usr/share/miniconda3/envs/test
2025-02-24T11:25:12.0320905Z 
2025-02-24T11:25:12.0321080Z   added / updated specs:
2025-02-24T11:25:12.0321520Z     - ffmpeg=6.1.1
2025-02-24T11:25:12.0321944Z     - pkg-config
2025-02-24T11:25:12.0322206Z 
2025-02-24T11:25:12.0322212Z 
2025-02-24T11:25:12.0322442Z The following packages will be downloaded:
2025-02-24T11:25:12.0322869Z 
2025-02-24T11:25:12.0323098Z     package                    |            build
2025-02-24T11:25:12.0323898Z     ---------------------------|-----------------
2025-02-24T11:25:12.0324501Z     aom-3.6.0                  |       h6a678d5_0         2.8 MB
2025-02-24T11:25:12.0325277Z     ca-certificates-2025.1.31  |       hbcca054_0         154 KB  conda-forge
2025-02-24T11:25:12.0326148Z     cairo-1.16.0               |       hb05425b_5         1.2 MB
2025-02-24T11:25:12.0326840Z     dav1d-1.2.1                |       h5eee18b_0         823 KB
2025-02-24T11:25:12.0327497Z     ffmpeg-6.1.1               |       h2a67f75_2         9.6 MB
2025-02-24T11:25:12.0328188Z     fontconfig-2.14.1          |       h55d465d_3         281 KB
2025-02-24T11:25:12.0328854Z     freetype-2.10.4            |       h0708190_1         890 KB  conda-forge
2025-02-24T11:25:12.0329501Z     giflib-5.2.1               |       h36c2ea0_2          77 KB  conda-forge
2025-02-24T11:25:12.0330095Z     glib-2.78.4                |       h6a678d5_0         508 KB
2025-02-24T11:25:12.0330559Z     glib-tools-2.78.4          |       h6a678d5_0         115 KB
2025-02-24T11:25:12.0330945Z     graphite2-1.3.14           |       h295c915_1          97 KB
2025-02-24T11:25:12.0331357Z     harfbuzz-10.2.0            |       hf296adc_0         2.4 MB
2025-02-24T11:25:12.0331704Z     icu-73.1                   |       h6a678d5_0        25.9 MB
2025-02-24T11:25:12.0332053Z     jbig-2.1                   |    h7f98852_2003          43 KB  conda-forge
2025-02-24T11:25:12.0332412Z     jpeg-9e                    |       h166bdaf_1         268 KB  conda-forge
2025-02-24T11:25:12.0332774Z     lame-3.100                 |    h7f98852_1001         496 KB  conda-forge
2025-02-24T11:25:12.0333162Z     leptonica-1.82.0           |       h950d820_0         2.6 MB  conda-forge
2025-02-24T11:25:12.0334137Z     lerc-2.2.1                 |       h9c3ff4c_0         213 KB  conda-forge
2025-02-24T11:25:12.0334759Z     libarchive-3.7.7           |       hfab0078_0         936 KB
2025-02-24T11:25:12.0335421Z     libdeflate-1.7             |       h7f98852_5          67 KB  conda-forge
2025-02-24T11:25:12.0336101Z     libglib-2.78.4             |       hdc74915_0         1.5 MB
2025-02-24T11:25:12.0336810Z     libiconv-1.17              |       h166bdaf_0         1.4 MB  conda-forge
2025-02-24T11:25:12.0337443Z     libogg-1.3.4               |       h7f98852_1         206 KB  conda-forge
2025-02-24T11:25:12.0338306Z     libopus-1.3.1              |       h7f98852_1         255 KB  conda-forge
2025-02-24T11:25:12.0338935Z     libpng-1.6.39              |       h5eee18b_0         304 KB
2025-02-24T11:25:12.0339298Z     libtheora-1.1.1            |    h7f98852_1005         651 KB  conda-forge
2025-02-24T11:25:12.0339646Z     libtiff-4.3.0              |       hf544144_1         668 KB  conda-forge
2025-02-24T11:25:12.0339994Z     libvorbis-1.3.7            |       h9c3ff4c_0         280 KB  conda-forge
2025-02-24T11:25:12.0340331Z     libvpx-1.13.1              |       h6a678d5_0         1.0 MB
2025-02-24T11:25:12.0340745Z     libwebp-1.2.2              |       h3452ae3_0          85 KB  conda-forge
2025-02-24T11:25:12.0341097Z     libwebp-base-1.2.2         |       h7f98852_1         824 KB  conda-forge
2025-02-24T11:25:12.0341425Z     libxcb-1.15                |       h7f8727e_0         505 KB
2025-02-24T11:25:12.0341737Z     libxml2-2.13.5             |       hfdd30dd_0         738 KB
2025-02-24T11:25:12.0342106Z     lz4-c-1.9.4                |       h6a678d5_1         156 KB
2025-02-24T11:25:12.0342461Z     openh264-2.1.1             |       h780b84a_0         1.5 MB  conda-forge
2025-02-24T11:25:12.0342888Z     openjpeg-2.4.0             |       hb52868f_1         444 KB  conda-forge
2025-02-24T11:25:12.0343566Z     pcre2-10.42                |       hebb0a14_1         1.3 MB
2025-02-24T11:25:12.0343894Z     pixman-0.40.0              |       h36c2ea0_0         627 KB  conda-forge
2025-02-24T11:25:12.0344243Z     pkg-config-0.29.2          |    h36c2ea0_1008         120 KB  conda-forge
2025-02-24T11:25:12.0344592Z     tesseract-5.2.0            |       h6a678d5_2       173.0 MB
2025-02-24T11:25:12.0345008Z     zstd-1.5.6                 |       hc292b87_0         664 KB
2025-02-24T11:25:12.0345429Z     ------------------------------------------------------------
2025-02-24T11:25:12.0345780Z                                            Total:       235.4 MB

@traversaro traversaro mentioned this issue Apr 28, 2025
Merged
@traversaro
Copy link
Contributor

Out of curiosity, I created #658 to understand if that was indeed the reason for the failure.

@traversaro
Copy link
Contributor

Out of curiosity, I created #658 to understand if that was indeed the reason for the failure.

It seems that only using conda-forge packages instead of mixing defaults and conda-forge fixes the issue: https://github.com/traversaro/torchcodec/actions/runs/14704627410/job/41261801072 (or the issue was fixed in the meanwhile by some other unrelated change).

@traversaro traversaro mentioned this issue Apr 28, 2025
Merged
@NicolasHug
Copy link
Member Author

Unfortunately not (even if I see why that is confusing). By looking at your CI trace, it seems that you are creating an environment with the defaults channel in the setup conda env step

Wow, I had no idea. Thank you for the info!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cpp_tests: Switch to only use conda-forge and add back ffmpeg 6 traversaro/torchcodec
3 participants
@scotts @NicolasHug @traversaro