TypeError: environment can only contain strings

[ggsf13]

Summary

I'm working on an image processing task with nipype, and I got the following problem

Actual behavior

Expected behavior

How to replicate the behavior

Script/Workflow details

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\study\project\3D cnn\3DUnetCNN-master\brats\preprocess.py", line 155, in convert_brats_data
    no_bias_correction_modalities=no_bias_correction_modalities)
  File "D:\study\project\3D cnn\3DUnetCNN-master\brats\preprocess.py", line 124, in convert_brats_folder
    normalize_image(image_file, out_file, bias_correction=perform_bias_correction)
  File "D:\study\project\3D cnn\3DUnetCNN-master\brats\preprocess.py", line 112, in normalize_image
    correct_bias(in_file, out_file)
  File "D:\study\project\3D cnn\3DUnetCNN-master\brats\preprocess.py", line 70, in correct_bias
    done = correct.run()
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\base\core.py", line 498, in run
    version=self.version)
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\ants\base.py", line 125, in version
    return Info.version()
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\base\core.py", line 1283, in version
    terminal_output='allatonce').run()
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\base\core.py", line 516, in run
    runtime = self._run_interface(runtime)
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\base\core.py", line 1020, in _run_interface
    runtime = run_command(runtime, output=self.terminal_output)
  File "D:\softwares\Annoconda3\lib\site-packages\nipype\interfaces\base\core.py", line 750, in run_command
    close_fds=False,
  File "D:\softwares\Annoconda3\lib\subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "D:\softwares\Annoconda3\lib\subprocess.py", line 997, in _execute_child
    startupinfo)
TypeError: environment can only contain strings

here are the codes that produced the error:

        def _execute_child(self, args, executable, preexec_fn, close_fds,
                           pass_fds, cwd, env,
                           startupinfo, creationflags, shell,
                           p2cread, p2cwrite,
                           c2pread, c2pwrite,
                           errread, errwrite,
                           unused_restore_signals, unused_start_new_session):
            """Execute program (MS Windows version)"""

            assert not pass_fds, "pass_fds not supported on Windows."

            if not isinstance(args, str):
                args = list2cmdline(args)

            # Process startup details
            if startupinfo is None:
                startupinfo = STARTUPINFO()
            if -1 not in (p2cread, c2pwrite, errwrite):
                startupinfo.dwFlags |= _winapi.STARTF_USESTDHANDLES
                startupinfo.hStdInput = p2cread
                startupinfo.hStdOutput = c2pwrite
                startupinfo.hStdError = errwrite

            if shell:
                startupinfo.dwFlags |= _winapi.STARTF_USESHOWWINDOW
                startupinfo.wShowWindow = _winapi.SW_HIDE
                comspec = os.environ.get("COMSPEC", "cmd.exe")
                args = '{} /c "{}"'.format (comspec, args)

            # Start the process
            try:
                hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
                                         # no special security
                                         None, None,
                                         int(not close_fds),
                                         creationflags,
                                         env,
                                         os.fspath(cwd) if cwd is not None else None,
                                         startupinfo)

Platform details:

{'pkg_path': 'D:\\softwares\\Annoconda3\\lib\\site-packages\\nipype',
 'commit_source': 'installation',
 'commit_hash': 'bded0d02',
 'nipype_version': '1.0.1',
 'sys_version': '3.6.3 |Anaconda custom (64-bit)| (default, Oct 15 2017, 03:27:45) [MSC v.1900 64 bit (AMD64)]',
 'sys_executable': 'D:\\softwares\\Annoconda3\\python.exe',
 'sys_platform': 'win32',
 'numpy_version': '1.13.3',
 'scipy_version': '0.19.1',
 'networkx_version': '2.0',
 'nibabel_version': '2.2.1',
 'traits_version': '4.6.0'}
1.0.1

Execution environment

can anyone help me with this? any suggestion is appreciated.

[effigies]

So it looks like this may be an incompatibility of 3DUnetCNN with more recent versions of nipype. Do they provide docker images that pre-package known compatible versions? If not, this may be worth tracking down on their side, and then seeing if there's something we can do here to resolve the issue.

[glioi]

I have exactly the same issue! Can anyone help with this?

[yunfeiluo]

I also have the same issue when using SPM interface. Is there any solutions? I'm using win10, and have spm running correctly in matlab.

[effigies]

Can you share the code producing the issue?

[yunfeiluo]

I think maybe this is the issue with popen under win10 environment. There are no such error when I switch to run my code in docker.

[AugustKRZhu]

Hi, I also face this problem when I just used the example code.

from nipype.interfaces.dcm2nii import Dcm2niix

converter = Dcm2niix()
converter.inputs.source_dir = 'xxx'
converter.inputs.compression = 5
converter.inputs.output_dir = 'yyy'
converter.cmdline
'dcm2niix -b y -z y -5 -x n -t n -m n -o ds005 -s n -v n dicomdir'
converter.run()

can anybody solve this?

Thanks

[effigies]

It looks like there are some related but not identical issues. The original issue is just a too-old nipype. That bug was fixed in #2085.

This latest is unclear exactly why it's happening, since it's occurring in version_from_command, which should be unused. Submitting a patch to fix it until we can remove it entirely.

[effigies]

Oh, actually, it seems to be the opposite. Windows has gone from only permitting bytes to only permitting strings.

[effigies]

Can people try installing my branch into your environment and testing?

pip install git+https://github.com/effigies/nipype.git@fix/canonicalize_env

[effigies]

Anybody up for a test? I don't have a Windows machine to test on.

[axiezai]

I set up the environment and have the following platform details:

{'commit_hash': '7d5639e78',
 'commit_source': 'installation',
 'networkx_version': '2.5',
 'nibabel_version': '3.2.0',
 'nipype_version': '1.6.0-dev',
 'numpy_version': '1.19.4',
 'pkg_path': 'C:\\Users\\Xihe\\anaconda3\\envs\\nipype_test\\lib\\site-packages\\nipype',
 'scipy_version': '1.5.4',
 'sys_executable': 'C:\\Users\\Xihe\\anaconda3\\envs\\nipype_test\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit '
                '(AMD64)]',
 'traits_version': '6.1.1'}

With the dicom2niix code snippet I got TypeError: 'OSError' object is not subscriptable error message instead... could not get the original poster's code to work, so couldn't replicate the error at all... maybe someone that originally reported the error message can give this a shot?

[benjo-90]

Hi there,

after I tried to run the nipype.interfaces.petpvc module on my windows 10 machine (Python 3.8, nipype 1.5.1), I got the following error:

Traceback (most recent call last):
  File "C:/Users/Benjo/PycharmProjects/generate_simulation_input/cluster_healthy_brains_pvc.py", line 60, in <module>
    outs = pvc.run()
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 419, in run
    runtime = self._run_interface(runtime)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 812, in _run_interface
    runtime = run_command(runtime, output=self.terminal_output)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 103, in run_command
    proc = Popen(
  File "c:\users\benjo\appdata\local\programs\python\python38\lib\subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "c:\users\benjo\appdata\local\programs\python\python38\lib\subprocess.py", line 1307, in _execute_child
    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
TypeError: environment can only contain strings

So I tried the solution from @effigies

Can people try installing my branch into your environment and testing?

pip install git+https://github.com/effigies/nipype.git@fix/canonicalize_env

and now after trying to run PETPVC again I am getting the same error as @axiezai:

Return code : 0
201111-10:22:04,182 nipype.interface INFO:
	 [WinError 10038] Ein Vorgang bezog sich auf ein Objekt, das kein Socket ist
Traceback (most recent call last):
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 124, in _process
    res = select.select(streams, [], [], timeout)
OSError: [WinError 10038] Ein Vorgang bezog sich auf ein Objekt, das kein Socket ist

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/Benjo/PycharmProjects/generate_simulation_input/cluster_healthy_brains_pvc.py", line 60, in <module>
    outs = pvc.run()
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 419, in run
    runtime = self._run_interface(runtime)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 812, in _run_interface
    runtime = run_command(runtime, output=self.terminal_output)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 137, in run_command
    _process()
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 127, in _process
    if e[0] == errno.EINTR:
TypeError: 'OSError' object is not subscriptable

As you can see, the first part is in german, unfortunately :-)

@axiezai By the way: With which command did you get this dictionary with platform details?

[axiezai]

@axiezai By the way: With which command did you get this dictionary with platform details?

python -c "import nipype; from pprint import pprint; pprint(nipype.get_info())

Taken from the new issues template :)

[effigies]

@benjo-90 The OSError issue should now be fixed, if you could reinstall my branch:

pip install git+https://github.com/effigies/nipype.git@fix/canonicalize_env

[effigies]

@benjo-90 @axiezai Have either of you had a chance to retry?

[axiezai]

I just pointed the dcm2niix example to an empty directory, with the updated installation you suggested:

{'commit_hash': '0dbcedda8',
 'commit_source': 'installation',
 'networkx_version': '2.5',
 'nibabel_version': '3.2.0',
 'nipype_version': '1.6.0-dev',
 'numpy_version': '1.19.4',
 'pkg_path': 'C:\\Users\\Xihe\\anaconda3\\envs\\nipype_test\\lib\\site-packages\\nipype',
 'scipy_version': '1.5.4',
 'sys_executable': 'C:\\Users\\Xihe\\anaconda3\\envs\\nipype_test\\python.exe',
 'sys_platform': 'win32',
 'sys_version': '3.8.5 (default, Sep  3 2020, 21:29:08) [MSC v.1916 64 bit '
                '(AMD64)]',
 'traits_version': '6.1.1'}

Here's the code I used:

from nipype.interfaces.dcm2nii import Dcm2niix

converter = Dcm2niix()
converter.inputs.source_dir = 'C:/Users/Xihe/Documents/nipype_testing/'
converter.inputs.compression = 5
converter.inputs.output_dir = 'C:/Users/Xihe/Documents/nipype_testing/'
converter.run()

The directory has no dicom images so I expect the following msg:

Error: Unable to find any DICOM images in /mnt/c/Users/Xihe/Downloads (or subfolders 5 deep) 
Return code: 2 

Which is what happens on my ubuntu subsystem on the same windows machine I'm testing on.

But in Windows Anaconda installation I am now getting:

---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
<ipython-input-8-e1d471e9c2ab> in <module>
----> 1 converter.run()

~\anaconda3\envs\nipype_test\lib\site-packages\nipype\interfaces\base\core.py in run(self, cwd, ignore_exception, **inputs)
    432         try:
    433             runtime = self._pre_run_hook(runtime)
--> 434             runtime = self._run_interface(runtime)
    435             runtime = self._post_run_hook(runtime)
    436             outputs = self.aggregate_outputs(runtime)

~\anaconda3\envs\nipype_test\lib\site-packages\nipype\interfaces\dcm2nii.py in _run_interface(self, runtime)
    442     def _run_interface(self, runtime):
    443         # may use return code 1 despite conversion
--> 444         runtime = super(Dcm2niix, self)._run_interface(
    445             runtime, correct_return_codes=(0, 1,)
    446         )

~\anaconda3\envs\nipype_test\lib\site-packages\nipype\interfaces\base\core.py in _run_interface(self, runtime, correct_return_codes)
    825             else "<skipped>"
    826         )
--> 827         runtime = run_command(runtime, output=self.terminal_output)
    828         if runtime.returncode is None or runtime.returncode not in correct_return_codes:
    829             self.raise_exception(runtime)

~\anaconda3\envs\nipype_test\lib\site-packages\nipype\utils\subprocess.py in run_command(runtime, output, timeout)
    135         while proc.returncode is None:
    136             proc.poll()
--> 137             _process()
    138 
    139         _process(drain=1)

~\anaconda3\envs\nipype_test\lib\site-packages\nipype\utils\subprocess.py in _process(drain)
    122         def _process(drain=0):
    123             try:
--> 124                 res = select.select(streams, [], [], timeout)
    125             except select.error as e:
    126                 iflogger.info(e)

OSError: [WinError 10038] An operation was attempted on something that is not a socket

Not sure if this is a separate issue or something weird going on with my old laptop

[effigies]

Thanks. I think the good news is that to get there you should have been able to spawn a new process...

[effigies]

So the issue is that select doesn't work for polling filehandles in Windows. I think the cleanest solution will be to use asyncio, which uses different mechanisms for Unix and Windows, but I don't understand the module well enough to do this quickly.

I do believe that the environment issue is resolved, so I'm merging #3267. Will open a new issue for the select error.

[benjo-90]

@benjo-90 @axiezai Have either of you had a chance to retry?

I was till now not able to test it, sorry.

I can confirm getting the new OSError after trying to run the nipype.interfaces.petpvc module on my windows 10 machine (Python 3.8, nipype 1.6.0):

C:\Users\Benjo\virtualEnvs\simpleITK\Scripts\python.exe C:/Users/Benjo/PycharmProjects/generate_simulation_input/test3.py
201128-22:44:48,787 nipype.interface INFO:
	 [WinError 10038] Ein Vorgang bezog sich auf ein Objekt, das kein Socket ist
Traceback (most recent call last):
  File "C:/Users/Benjo/PycharmProjects/generate_simulation_input/test3.py", line 13, in <module>
    outs = pvc.run()
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 434, in run
    runtime = self._run_interface(runtime)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\interfaces\base\core.py", line 827, in _run_interface
    runtime = run_command(runtime, output=self.terminal_output)
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 137, in run_command
    _process()
  File "C:\Users\Benjo\virtualEnvs\simpleITK\lib\site-packages\nipype\utils\subprocess.py", line 124, in _process
    res = select.select(streams, [], [], timeout)
OSError: [WinError 10038] Ein Vorgang bezog sich auf ein Objekt, das kein Socket ist

Process finished with exit code 1

[effigies]

I believe if you run with Cygwin, this will be resolved. Possibly also the Windows Subsystem for Linux (WSL).

Opened #3277 for further discussion.