Skip to content

Numcodecs 0.7.0 fails to install, missing blosc.c #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ronpandolfi opened this issue Sep 10, 2020 · 14 comments · Fixed by #244
Closed

Numcodecs 0.7.0 fails to install, missing blosc.c #242

ronpandolfi opened this issue Sep 10, 2020 · 14 comments · Fixed by #244

Comments

@ronpandolfi
Copy link

Minimal, reproducible code sample, a copy-pastable example if possible

(venv) C:\Users\LBL\PycharmProjects\merged-repo\Xi-cam.NCEM>pip install --upgrade numcodecs
Collecting numcodecs
  Downloading numcodecs-0.7.0.tar.gz (3.0 MB)
     |████████████████████████████████| 3.0 MB 1.6 MB/s
Requirement already satisfied, skipping upgrade: numpy>=1.7 in c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages (from numcodecs) (1.19.0)
Building wheels for collected packages: numcodecs
  Building wheel for numcodecs (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: 'c:\users\lbl\pycharmprojects\merged-repo\venv\scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"'; __file__='"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"
', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\LBL\AppData\Local\Temp\pip-wheel-mrbp79n3'
       cwd: C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\
  Complete output (131 lines):
  [numcodecs] setting up Blosc extension
  [numcodecs] compiling Blosc extension with SSE2 support
  [numcodecs] compiling Blosc extension with AVX2 support
  [numcodecs] setting up Zstandard extension
  [numcodecs] setting up LZ4 extension
  [numcodecs] setting up compat extension
  [numcodecs] setting up vlen extension
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build\lib.win-amd64-3.7
  creating build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\abc.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\astype.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\base64.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\bz2.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\categorize.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\checksum32.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\compat.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\delta.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\fixedscaleoffset.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\gzip.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\json.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\lzma.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\msgpacks.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\packbits.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\pickles.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\quantize.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\registry.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\version.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\zlib.py -> build\lib.win-amd64-3.7\numcodecs
  copying .\numcodecs\__init__.py -> build\lib.win-amd64-3.7\numcodecs
  creating build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\common.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_astype.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_base64.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_blosc.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_bz2.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_categorize.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_checksum32.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_compat.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_delta.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_fixedscaleoffset.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_gzip.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_json.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_lz4.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_lzma.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_msgpacks.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_packbits.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_pickles.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_quantize.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_registry.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_vlen_array.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_vlen_bytes.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_vlen_utf8.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_zlib.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\test_zstd.py -> build\lib.win-amd64-3.7\numcodecs\tests
  copying .\numcodecs\tests\__init__.py -> build\lib.win-amd64-3.7\numcodecs\tests
  running build_ext
  building 'numcodecs.blosc' extension
  creating build\temp.win-amd64-3.7
  creating build\temp.win-amd64-3.7\Release
  creating build\temp.win-amd64-3.7\Release\numcodecs
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -D__SSE2__=1 -D__AVX2__=1 -Ic-blosc\blosc -Ic:\users\lbl\pycharmprojects\merged-repo\venv\include "-IC:\Program Files\Python37\include" "-IC:\Program F
iles\Python37\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.183
62.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcnumcodecs/blosc.c /Fobuild\temp.win-amd64-3.7\Release\numcodecs/blosc.obj -DSHUFFLE_SSE2_ENABLED -DSHUFFLE_AVX2_ENABLED
  blosc.c
  numcodecs/blosc.c(705): fatal error C1083: Cannot open include file: 'blosc.h': No such file or directory
  [numcodecs] command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
  Traceback (most recent call last):
    File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 396, in compile
      self.spawn(args)
    File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 502, in spawn
      return super().spawn(cmd)
    File "C:\Program Files\Python37\lib\distutils\ccompiler.py", line 910, in spawn
      spawn(cmd, dry_run=self.dry_run)
    File "C:\Program Files\Python37\lib\distutils\spawn.py", line 38, in spawn
      _spawn_nt(cmd, search_path, dry_run=dry_run)
    File "C:\Program Files\Python37\lib\distutils\spawn.py", line 81, in _spawn_nt
      "command %r failed with exit status %d" % (cmd, rc))
  distutils.errors.DistutilsExecError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 276, in build_extension
      build_ext.build_extension(self, ext)
    File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 534, in build_extension
      depends=ext.depends)
    File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 398, in compile
      raise CompileError(msg)
  distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
  
  During handling of the above exception, another exception occurred:
  
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 350, in <module>
      run_setup(with_extensions)
    File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 343, in run_setup
      license='MIT',
    File "c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\setuptools\__init__.py", line 145, in setup
      return distutils.core.setup(**attrs)
    File "C:\Program Files\Python37\lib\distutils\core.py", line 148, in setup
      dist.run_commands()
    File "C:\Program Files\Python37\lib\distutils\dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\wheel\bdist_wheel.py", line 223, in run
      self.run_command('build')
    File "C:\Program Files\Python37\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Program Files\Python37\lib\distutils\command\build.py", line 135, in run
      self.run_command(cmd_name)
    File "C:\Program Files\Python37\lib\distutils\cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
      cmd_obj.run()
    File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 269, in run
      build_ext.run(self)
    File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 340, in run
      self.build_extensions()
    File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 449, in build_extensions
      self._build_extensions_serial()
    File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial
      self.build_extension(ext)
    File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 279, in build_extension
      raise BuildFailed()
  __main__.BuildFailed
  ----------------------------------------
  ERROR: Failed building wheel for numcodecs
  Running setup.py clean for numcodecs
Failed to build numcodecs
DEPRECATION: Could not build wheels for numcodecs which do not use PEP 517. pip will fall back to legacy 'setup.py install' for these. pip 21.0 will remove support for this functionality. A possible replacement is to fix the wheel build issue reported above. You can find discussion regarding this at https://github.com/pypa/pip/issues/8368.
Installing collected packages: numcodecs
  Attempting uninstall: numcodecs
    Found existing installation: numcodecs 0.6.4
    Uninstalling numcodecs-0.6.4:
      Successfully uninstalled numcodecs-0.6.4
    Running setup.py install for numcodecs ... error
    ERROR: Command errored out with exit status 1:
     command: 'c:\users\lbl\pycharmprojects\merged-repo\venv\scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"'; __file__='"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"
'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\LBL\AppData\Local\Temp\pip-record-i468c2_6\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\lbl\pycharmprojects\merged-repo\venv\include\site\python3.7\num
codecs'
         cwd: C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\
    Complete output (133 lines):
    [numcodecs] setting up Blosc extension
    [numcodecs] compiling Blosc extension with SSE2 support
    [numcodecs] compiling Blosc extension with AVX2 support
    [numcodecs] setting up Zstandard extension
    [numcodecs] setting up LZ4 extension
    [numcodecs] setting up compat extension
    [numcodecs] setting up vlen extension
    running install
    running build
    running build_py
    creating build
    creating build\lib.win-amd64-3.7
    creating build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\abc.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\astype.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\base64.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\bz2.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\categorize.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\checksum32.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\compat.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\delta.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\fixedscaleoffset.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\gzip.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\json.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\lzma.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\msgpacks.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\packbits.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\pickles.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\quantize.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\registry.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\version.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\zlib.py -> build\lib.win-amd64-3.7\numcodecs
    copying .\numcodecs\__init__.py -> build\lib.win-amd64-3.7\numcodecs
    creating build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\common.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_astype.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_base64.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_blosc.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_bz2.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_categorize.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_checksum32.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_compat.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_delta.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_fixedscaleoffset.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_gzip.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_json.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_lz4.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_lzma.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_msgpacks.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_packbits.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_pickles.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_quantize.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_registry.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_vlen_array.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_vlen_bytes.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_vlen_utf8.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_zlib.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\test_zstd.py -> build\lib.win-amd64-3.7\numcodecs\tests
    copying .\numcodecs\tests\__init__.py -> build\lib.win-amd64-3.7\numcodecs\tests
    running build_ext
    building 'numcodecs.blosc' extension
    creating build\temp.win-amd64-3.7
    creating build\temp.win-amd64-3.7\Release
    creating build\temp.win-amd64-3.7\Release\numcodecs
    C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\bin\HostX86\x64\cl.exe /c /nologo /Ox /W3 /GL /DNDEBUG /MD -DHAVE_LZ4=1 -DHAVE_SNAPPY=1 -DHAVE_ZLIB=1 -DHAVE_ZSTD=1 -D__SSE2__=1 -D__AVX2__=1 -Ic-blosc\blosc -Ic:\users\lbl\pycharmprojects\merged-repo\venv\include "-IC:\Program Files\Python37\include" "-IC:\Program
 Files\Python37\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\ATLMFC\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.27.29110\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.1
8362.0\shared" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\um" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\winrt" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.18362.0\cppwinrt" /Tcnumcodecs/blosc.c /Fobuild\temp.win-amd64-3.7\Release\numcodecs/blosc.obj -DSHUFFLE_SSE2_ENABLED -DSHUFFLE_AVX2_ENABLED
    blosc.c
    numcodecs/blosc.c(705): fatal error C1083: Cannot open include file: 'blosc.h': No such file or directory
    [numcodecs] command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
    Traceback (most recent call last):
      File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 396, in compile
        self.spawn(args)
      File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 502, in spawn
        return super().spawn(cmd)
      File "C:\Program Files\Python37\lib\distutils\ccompiler.py", line 910, in spawn
        spawn(cmd, dry_run=self.dry_run)
      File "C:\Program Files\Python37\lib\distutils\spawn.py", line 38, in spawn
        _spawn_nt(cmd, search_path, dry_run=dry_run)
      File "C:\Program Files\Python37\lib\distutils\spawn.py", line 81, in _spawn_nt
        "command %r failed with exit status %d" % (cmd, rc))
    distutils.errors.DistutilsExecError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 276, in build_extension
        build_ext.build_extension(self, ext)
      File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 534, in build_extension
        depends=ext.depends)
      File "C:\Program Files\Python37\lib\distutils\_msvccompiler.py", line 398, in compile
        raise CompileError(msg)
    distutils.errors.CompileError: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\VC\\Tools\\MSVC\\14.27.29110\\bin\\HostX86\\x64\\cl.exe' failed with exit status 2
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 350, in <module>
        run_setup(with_extensions)
      File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 343, in run_setup
        license='MIT',
      File "c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\setuptools\__init__.py", line 145, in setup
        return distutils.core.setup(**attrs)
      File "C:\Program Files\Python37\lib\distutils\core.py", line 148, in setup
        dist.run_commands()
      File "C:\Program Files\Python37\lib\distutils\dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\setuptools\command\install.py", line 61, in run
        return orig.install.run(self)
      File "C:\Program Files\Python37\lib\distutils\command\install.py", line 545, in run
        self.run_command('build')
      File "C:\Program Files\Python37\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "C:\Program Files\Python37\lib\distutils\command\build.py", line 135, in run
        self.run_command(cmd_name)
      File "C:\Program Files\Python37\lib\distutils\cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "C:\Program Files\Python37\lib\distutils\dist.py", line 985, in run_command
        cmd_obj.run()
      File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 269, in run
        build_ext.run(self)
      File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 340, in run
        self.build_extensions()
      File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 449, in build_extensions
        self._build_extensions_serial()
      File "C:\Program Files\Python37\lib\distutils\command\build_ext.py", line 474, in _build_extensions_serial
        self.build_extension(ext)
      File "C:\Users\LBL\AppData\Local\Temp\pip-install-_34gyqz2\numcodecs\setup.py", line 279, in build_extension
        raise BuildFailed()
    __main__.BuildFailed
    ----------------------------------------
  Rolling back uninstall of numcodecs
  Moving to c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\numcodecs-0.6.4-py3.7.egg-info
   from c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\~umcodecs-0.6.4-py3.7.egg-info
  Moving to c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\numcodecs\
   from c:\users\lbl\pycharmprojects\merged-repo\venv\lib\site-packages\~umcodecs
ERROR: Command errored out with exit status 1: 'c:\users\lbl\pycharmprojects\merged-repo\venv\scripts\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"'; __file__='"'"'C:\\Users\\LBL\\AppData\\Local\\Temp\\pip-install-_34gyqz2\\numcodecs\\setup.py'"'"
';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\LBL\AppData\Local\Temp\pip-record-i468c2_6\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\lbl\pycharmprojects\merged-rep
o\venv\include\site\python3.7\numcodecs' Check the logs for full command output.
WARNING: You are using pip version 20.2.1; however, version 20.2.3 is available.
You should consider upgrading via the 'c:\users\lbl\pycharmprojects\merged-repo\venv\scripts\python.exe -m pip install --upgrade pip' command.

Problem description

It seems that some components are missing in the 0.7.0 distribution.

Version and installation information

  • Value of numcodecs.__version__: failed to install, but 0.7.0
  • Version of Python interpreter: 3.7.7
  • Operating system (Linux/Windows/Mac): Windows
  • How NumCodecs was installed (e.g., "using pip into virtual environment", or "using conda"): venv with pip
@Carreau
Copy link
Contributor

Carreau commented Sep 10, 2020 via email

@ronpandolfi
Copy link
Author

ronpandolfi commented Sep 10, 2020 •
edited
Loading

Locally I have 41.2.0, on travis I had 50.3.0; same result.

Here's a travis log if it helps:
https://travis-ci.org/github/Xi-CAM/Xi-cam/jobs/725765698

@jakirkham
Copy link
Member

FWIW we have the same issue in conda-forge ( conda-forge/numcodecs-feedstock#51 ) using setuptools 49.6.0.

Maybe the git submodule was not included in the sdist?

Though I would have thought wheels would have been published with PR ( #241 ) now in. Maybe we are still missing something on that front as well?

@ronpandolfi ronpandolfi mentioned this issue Sep 10, 2020
Closed
@jakirkham
Copy link
Member

I've gone ahead and yanked the 0.7.0 and 0.7.0rc1 packages from PyPI until we have a chance to sort this out. Should allow people to continue in the interim.

@jakirkham
Copy link
Member

cc @joshmoore (for vis)

@jakirkham
Copy link
Member

Also cc-ing @Czaki in case you have thoughts on how we can get the wheels published to PyPI.

@jakirkham jakirkham mentioned this issue Sep 10, 2020
Merged
9 tasks
@ronpandolfi ronpandolfi mentioned this issue Sep 10, 2020
Merged
@Czaki
Copy link
Contributor

Czaki commented Sep 10, 2020

Also cc-ing @Czaki in case you have thoughts on how we can get the wheels published to PyPI.

I personally download wheels files from CI and use twine to upload wheel (I think same code which you use to upload source tar ball)
But there is full example how to build and upload wheels on github actions in fully automated version https://github.com/joerick/cibuildwheel/blob/master/examples/github-deploy.yml

As J fast check there is no problem with blosc.c file but with c-blosc directory. I also suggest to exclude c files generated from pyx from sdist.

@jakirkham To solve this issue You need to change actions/checkout@v2 to actions/checkout@v1 in build_sdist or set proper options because, by default actions/checkout@v2 does not pull subrepositories and c-blosc is subrepository.

@Czaki Czaki mentioned this issue Sep 10, 2020
Merged
7 tasks
@jakirkham
Copy link
Member

Thanks Grzegorz! Bot for the clarification and the fix.

Should we maybe keep the C files generated by Cython in case a user doesn't have Cython included?

@Czaki
Copy link
Contributor

Czaki commented Sep 10, 2020

The problem which I already meet it that new releases of python may need changed c files. And cython does not recompile pyx file if c file exists.

kiyo-masui/bitshuffle#76

@jakirkham
Copy link
Member

Gotcha. Yeah it may be possible to bypass this sort of issue using the --force flag (though I haven't checked). Worst case one deletes the .c files manually, which we've done in some conda-forge releases before.

@joshmoore
Copy link
Member

joshmoore commented Sep 10, 2020 •
edited
Loading

Sorry if you guys would have liked to have the results of this discussion included in 0.7.1, but for what it's worth, it's looking functional. Please do give it a try.

@jakirkham
Copy link
Member

AFAICT the .c files are in the sdist. So I think we are ok.

@Czaki
Copy link
Contributor

Czaki commented Sep 10, 2020

the source of problem are c files in sdist. If there are binaries in wheel there is no compilation step.

Problem is produced by abi changes in python which needs new c file, but cython does not create new file because one exists in sdist.

@jakirkham
Copy link
Member

Right like when a new Python version comes out (like 3.9). Personally I would argue this is a feature as opposed to a bug since the developers (in this case us) haven't tested and shipped with something that supports the new Python version yet. When that happens users are welcome to bug us on the issue tracker :) Alternatively users can remove .c files by using --force (just tested this locally) to regenerate the C files, but by doing so are acknowledging that they are building against a currently unsupported Python version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix build Czaki/numcodecs
5 participants
@joshmoore @Carreau @jakirkham @Czaki @ronpandolfi