SystemError: initialization of _heapq did not return an extension module

[OliverEvans96]

I'm experiencing the same error mentioned by @sirgogo in #88 .

oliver@oliver-arch:~ % ipython
Python 3.5.5 |Anaconda, Inc.| (default, Mar 12 2018, 23:12:44) 
Type 'copyright', 'credits' or 'license' for more information
IPython 6.2.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import julia

In [2]: j = julia.Julia()
Error processing line 1 of /home/oliver/local/miniconda3/lib/python3.5/site-packages/PyUtilib-5.4.1-py2.7-nspkg.pth:

Failed to import the site module
Traceback (most recent call last):
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 167, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "/home/oliver/local/miniconda3/lib/python3.5/types.py", line 166, in <module>
    import functools as _functools
  File "/home/oliver/local/miniconda3/lib/python3.5/functools.py", line 21, in <module>
    from collections import namedtuple
  File "/home/oliver/local/miniconda3/lib/python3.5/collections/__init__.py", line 29, in <module>
    import heapq as _heapq
  File "/home/oliver/local/miniconda3/lib/python3.5/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 559, in <module>
    main()
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 541, in main
    known_paths = addsitepackages(known_paths)
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 318, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 206, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/home/oliver/local/miniconda3/lib/python3.5/site.py", line 177, in addpackage
    import traceback
  File "/home/oliver/local/miniconda3/lib/python3.5/traceback.py", line 3, in <module>
    import collections
  File "/home/oliver/local/miniconda3/lib/python3.5/collections/__init__.py", line 29, in <module>
    import heapq as _heapq
  File "/home/oliver/local/miniconda3/lib/python3.5/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

I'm running Arch Linux and using the following versions:

Package	Version
python	3.5.5 :: Anaconda, Inc.
julia	0.6.2
PyJulia	0.1.5
PyCall	1.15.0

Any advice is appreciated!

Thanks,
Oliver

[SHTILL]

Have exactly the same error:
JULIA_HOME = /home/vasya/dist/julia/bin, libjulia_path = /home/vasya/dist/julia/bin/../lib/libjulia.so.0.6
calling jl_init_with_image(b'/home/vasya/anaconda3/lib/python3.6/site-packages/julia/fake-julia', /home/vasya/dist/julia/lib/julia/sys.so)
seems to work...
exception occured? None
exception occured? None
exception occured? None
exception occured? None
Error processing line 1 of /home/vasya/anaconda3/lib/python3.6/site-packages/matplotlib-2.1.2-py3.6-nspkg.pth

[matteoacrossi]

I have a similar error but it happens in matplotlib

SystemError: initialization of _heapq did not return an extension module

Python 3.5.5 |Anaconda custom (64-bit)| (default, Mar 12 2018, 16:25:05) 
[GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import julia
>>> j = julia.Julia()
Error processing line 1 of /Users/matteo/anaconda/lib/python3.5/site-packages/matplotlib-2.2.2-py3.5-nspkg.pth:

Failed to import the site module
Traceback (most recent call last):
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 167, in addpackage
    exec(line)
  File "<string>", line 1, in <module>
  File "/Users/matteo/anaconda/lib/python3.5/types.py", line 166, in <module>
    import functools as _functools
  File "/Users/matteo/anaconda/lib/python3.5/functools.py", line 21, in <module>
    from collections import namedtuple
  File "/Users/matteo/anaconda/lib/python3.5/collections/__init__.py", line 29, in <module>
    import heapq as _heapq
  File "/Users/matteo/anaconda/lib/python3.5/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 559, in <module>
    main()
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 541, in main
    known_paths = addsitepackages(known_paths)
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 318, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 206, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/Users/matteo/anaconda/lib/python3.5/site.py", line 177, in addpackage
    import traceback
  File "/Users/matteo/anaconda/lib/python3.5/traceback.py", line 3, in <module>
    import collections
  File "/Users/matteo/anaconda/lib/python3.5/collections/__init__.py", line 29, in <module>
    import heapq as _heapq
  File "/Users/matteo/anaconda/lib/python3.5/heapq.py", line 587, in <module>
    from _heapq import *
SystemError: initialization of _heapq did not return an extension module

[carmagnole]

I have the same problem:

julia.Julia(debug = True)
JULIA_HOME = /Applications/Julia-0.6.app/Contents/Resources/julia/bin, libjulia_path = /Applications/Julia-0.6.app/Contents/Resources/julia/lib/libjulia.0.6.2.dylib
calling jl_init_with_image(b'/Applications/Julia-0.6.app/Contents/Resources/julia/bin', /Applications/Julia-0.6.app/Contents/Resources/julia/lib/julia/sys.dylib)
seems to work...
exception occured? None
Failed to import the site module
Traceback (most recent call last):
File "/anaconda3/envs/Neuroscience/lib/python3.6/site.py", line 541, in
main()
File "/anaconda3/envs/Neuroscience/lib/python3.6/site.py", line 522, in main
known_paths = addusersitepackages(known_paths)
File "/anaconda3/envs/Neuroscience/lib/python3.6/site.py", line 282, in addusersitepackages
user_site = getusersitepackages()
File "/anaconda3/envs/Neuroscience/lib/python3.6/site.py", line 258, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/anaconda3/envs/Neuroscience/lib/python3.6/site.py", line 248, in getuserbase
USER_BASE = get_config_var('userbase')
File "/anaconda3/envs/Neuroscience/lib/python3.6/sysconfig.py", line 608, in get_config_var
return get_config_vars().get(name)
File "/anaconda3/envs/Neuroscience/lib/python3.6/sysconfig.py", line 587, in get_config_vars
import _osx_support
File "/anaconda3/envs/Neuroscience/lib/python3.6/_osx_support.py", line 4, in
import re
File "/anaconda3/envs/Neuroscience/lib/python3.6/re.py", line 122, in
import enum
File "/anaconda3/envs/Neuroscience/lib/python3.6/enum.py", line 2, in
from types import MappingProxyType, DynamicClassAttribute
File "/anaconda3/envs/Neuroscience/lib/python3.6/types.py", line 171, in
import functools as _functools
File "/anaconda3/envs/Neuroscience/lib/python3.6/functools.py", line 21, in
from collections import namedtuple
File "/anaconda3/envs/Neuroscience/lib/python3.6/collections/init.py", line 29, in
import heapq as _heapq
File "/anaconda3/envs/Neuroscience/lib/python3.6/heapq.py", line 587, in
from _heapq import *
SystemError: initialization of _heapq did not return an extension module

[randy3k]

I have a similar issue on my conda python, but not the homebrew python.

It may be because JuliaCall fails to detect if (anaconda/miniconda) python has been initialized.

I got the following from my miniconda python

In [1]: import ctypes

In [2]: libpython = ctypes.PyDLL("/Users/Randy/miniconda3/lib/libpython3.6m.dylib")

In [3]: libpython.Py_IsInitialized()
Out[3]: 0

I got the following from my homebrew python

In [1]: import ctypes

In [2]: libpython = ctypes.PyDLL("/usr/local/Cellar/python3/3.6.5/Frameworks/Python.framework/Versions/3.6/lib/libpython3.6.dylib")

In [3]: libpython.Py_IsInitialized()
Out[4]: 1

UPDATE:

The following code works in miniconda python

In [1]: import ctypes

In [2]: libpython = ctypes.CDLL(None)

In [3]: libpython.Py_IsInitialized()
Out[3]: 1

And it seems that PyCall already loads in process symbols JuliaPy/PyCall.jl#293
Not sure the reason now.

[bstellato]

I am having the same issue with miniconda. I am also getting the same outputs as in #175 (comment) Any ideas why it cannot detect if python has been initialized?

[tkf]

Hmm... I thought I fixed #175 by this commit. Does it happen with PyJulia master as well?

I remember it happened with Julia 0.6 when the precompilation failed.

[bstellato]

Yes I am using the master and Julia 1.0. I deleted the ~/.julia/compiled/v1.0/PyCall/xxx.ji file before but it does not seem to help. The ctypes code here in #150 (comment) gives the same answer for me.

[tkf]

Julia 1.0 with miniconda is not supported ATM so it is supposed to print some informative error message. Are you using Linux?

If you use miniconda in Linux, python-jl is probably the best way to make it work: https://github.com/tkf/pyjulia/blob/readme/README.md#your-python-interpreter-is-statically-linked-to-libpython

[bstellato]

I am using Mac but it actually works with python-jl and Julia 1.0. Thanks a lot! However, it would be great to have the anaconda python working without that hack.

[tkf]

Good that it works!

BTW, in your comment you are mentioning you tried ldd, right? Meaning that

ldd PATH/TO/YOUR/python

doesn't show libpython? Do you actually have ldd in macOS? I thought it's different command.

Additionally, can you run the following in miniconda python (not python-jl) you were using?

from julia.find_libpython import find_libpython, linked_libpython
linked_libpython()  # what does it return?
find_libpython()    # what does it return?

I thought macOS didn't need python-jl. But if linked_libpython() above returns None it means our assumption was not correct...

it would be great to have the anaconda python working without that hack

Yeah I know... But I can't do anything now. 😿

[bstellato]

Sorry, I meant otool -L! Unfortunately it does not list anything related to libpython.

I have tried your commands.

• linked_libpython() returns None.
• find_libpython() returns the right library /Users/xxx/miniconda3/envs/python36/lib/libpython3.6m.dylib

Thanks for the quick responses by the way :)

[tkf]

You are welcome :) And thanks for trying the commands. Good to figure out that linked_libpython() can return None in macOS (I mean, it's bad, but better than not knowing it...).