permanentFail when running a workflow with optional parameters between steps

[tom-tan]

I put a set of workflow codes in https://github.com/tom-tan/cwl-poc.

A workflow (workflow.cwl) consists of three steps: br, path1 and path2.
A step br outputs output1.txt as output1 if an input of workflow is 1 or outputs output2.txt as output2 otherwise.
The type for output1 and output2 in br is File?.
A step path1 (or path2 respectively) receives the output of br and print Path1! output1.txt if br outputs output1.txt or print Path1! otherwise.
The input type of path1 and path2 is File?.

How to reproduce

$ git clone https://github.com/tom-tan/cwl-poc
$ cd cwl-poc
$ cwltool workflow.cwl --inp 1

Expected Behavior

The following commands are executed.

• python branch.py 1 (for br)
• echo Path1! output1.txt (for path1)
• echo Path2! (for path2)

Actual Behavior

A workflow resulted in permanentFailure because path1 resulted in permanentFailure.
Note that path2 finished successfully.

Outputs

$ cwltool workflow.cwl --inp 1
(skip)
...
[step path1] start
[step path1] start
unrecognized extension field `http://commonwl.org/cwltool#generation`.  Did you include a $schemas section?
[job path1] /var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp1kc5n8lv$ docker \
    run \
    -i \
    --volume=/private/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp1kc5n8lv:/private/var/spool/cwl:rw \
    --volume=/private/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp7l588b6_:/tmp:rw \
    --volume=/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmplav7asjk/output1.txt:/private/var/lib/cwl/stg70994074-fc0f-4db4-b667-c4cd498fe422/output1.txt:ro \
    --workdir=/private/var/spool/cwl \
    --read-only=true \
    --user=501:20 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/private/var/spool/cwl \
    alpine \
    echo \
    Path1! \
    output1.txt
[job path1] /var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp1kc5n8lv$ docker \
    run \
    -i \
    --volume=/private/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp1kc5n8lv:/private/var/spool/cwl:rw \
    --volume=/private/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmp7l588b6_:/tmp:rw \
    --volume=/var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmplav7asjk/output1.txt:/private/var/lib/cwl/stg70994074-fc0f-4db4-b667-c4cd498fe422/output1.txt:ro \
    --workdir=/private/var/spool/cwl \
    --read-only=true \
    --user=501:20 \
    --rm \
    --env=TMPDIR=/tmp \
    --env=HOME=/private/var/spool/cwl \
    alpine \
    echo \
    Path1! \
    output1.txt
docker: Error response from daemon: Mounts denied:
The path /var/folders/gk/zyzswjbs6m59ywz69zhrtmlh0000gn/T/tmplav7asjk/output1.txt
is not shared from OS X and is not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.
.
ERRO[0000] error waiting for container: context canceled
[job path1] completed permanentFail
[job path1] completed permanentFail
[step path1] completed permanentFail
[step path1] completed permanentFail
[workflow workflow.cwl] completed permanentFail
[workflow workflow.cwl] completed permanentFail
{}
Final process status is permanentFail
Final process status is permanentFail

I also run the above command with --leave-outputs option and find that output1.txt is successfully generated by br.

Your Environment

• cwltool version:
  I confirmed this issue with cwltool installed via pip.

$ cwltool --version
/Users/tanjo/.pyenv/versions/3.6.2/bin/cwltool 1.0.20180211183944

• OS version: macOS High Sierra (10.13.3)
• Docker version: Docker Community Edition (version 17.12.0-ce-ac49 (21995))

[anton-khodak]

This solution worked for me: #328 (comment). I think we should set this programmaticaly on Mac OS.