Skip to content

Commit 744b19f

Browse files
kinowkinow
committed
Use runcmd
1 parent 4c80fb1 commit 744b19f

File tree

4 files changed

+70
-104
lines changed

4 files changed

+70
-104
lines changed

‎src/_includes/cwl/troubleshooting/troubleshooting-wf1-stepb-fixed.cwl

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
cwlVersion: v1.2
2+
class: Workflow
3+
4+
inputs: []
5+
outputs: []
6+
7+
steps:
8+
step_a:
9+
run:
10+
class: CommandLineTool
11+
inputs: []
12+
outputs:
13+
step_a_file:
14+
type: File
15+
outputBinding:
16+
glob: 'step_a.txt'
17+
arguments: ['touch', 'step_a.txt']
18+
in: []
19+
out: [step_a_file]
20+
step_b:
21+
run:
22+
class: CommandLineTool
23+
inputs: []
24+
outputs: []
25+
arguments: ['touch', 'step_b.txt']
26+
# To force step_b to wait for step_a
27+
in:
28+
step_a_file:
29+
source: step_a/step_a_file
30+
out: []

‎src/_includes/cwl/troubleshooting/troubleshooting-wf1.cwl

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
cwlVersion: v1.2
2+
class: Workflow
3+
4+
inputs: []
5+
outputs: []
6+
7+
steps:
8+
step_a:
9+
run:
10+
class: CommandLineTool
11+
inputs: []
12+
outputs:
13+
step_a_file:
14+
type: File
15+
outputBinding:
16+
glob: 'step_a.txt'
17+
arguments: ['touch', 'step_a.txt']
18+
in: []
19+
out: [step_a_file]
20+
step_b:
21+
run:
22+
class: CommandLineTool
23+
inputs: []
24+
outputs: []
25+
arguments: ['ouch', 'step_b.txt']
26+
# To force step_b to wait for step_a
27+
in:
28+
step_a_file:
29+
source: step_a/step_a_file
30+
out: []

‎src/topics/inputs.md

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -33,16 +33,8 @@ Create a file called `inp-job.yml`:
3333
You can use `cwltool` to create a template input object. That saves you from having
3434
to type all the input parameters in a input object file:
3535
36-
```bash
37-
$ cwltool --make-template inp.cwl
38-
INFO /home/kinow/Development/python/workspace/cwltool/venv/bin/cwltool 3.1.20220621073108
39-
INFO Resolved 'inp.cwl' to 'file:///tmp/inp.cwl'
40-
example_string: a_string # type "string"
41-
example_int: 0 # type "int"
42-
example_flag: false # type "boolean"
43-
example_file: # type "File" (optional)
44-
class: File
45-
path: a/file/path
36+
```{runcmd} cwltool --make-template inp.cwl
37+
:working-directory: src/_includes/cwl/inputs
4638
```
4739
4840
You can redirect the output to a file, i.e. `cwltool --make-template inp.cwl > inp-job.yml`,

‎src/topics/troubleshooting.md

Lines changed: 8 additions & 94 deletions
Original file line numberDiff line numberDiff line change
@@ -17,39 +17,11 @@ These two steps are executed in order (we enforce it, see note below). The first
1717
where instead of also executing the `touch` command it tries to execute `ouch`, which
1818
fails.
1919

20-
```{code-block} cwl
20+
```{literalinclude} /_includes/cwl/troubleshooting/troubleshooting-wf1.cwl
21+
:language: cwl
2122
:name: "`troubleshooting-wf1.cwl`"
2223
:caption: "`troubleshooting-wf1.cwl`"
23-
cwlVersion: v1.2
24-
class: Workflow
25-
26-
inputs: []
27-
outputs: []
28-
29-
steps:
30-
step_a:
31-
run:
32-
class: CommandLineTool
33-
inputs: []
34-
outputs:
35-
step_a_file:
36-
type: File
37-
outputBinding:
38-
glob: 'step_a.txt'
39-
arguments: ['touch', 'step_a.txt']
40-
in: []
41-
out: [step_a_file]
42-
step_b:
43-
run:
44-
class: CommandLineTool
45-
inputs: []
46-
outputs: []
47-
arguments: ['ouch', 'step_b.txt']
48-
# To force step_b to wait for step_a
49-
in:
50-
step_a_file:
51-
source: step_a/step_a_file
52-
out: []
24+
:emphasize-lines: 25
5325
```
5426

5527
```{note}
@@ -62,54 +34,18 @@ of `step_b`.
6234
Let's execute this workflow with `/tmp/cachedir/` as the `--cachedir` value (`cwltool` will
6335
create the directory for you if it does not exist already):
6436

65-
```{code-block} console
37+
```{runcmd} cwltool --cachedir /tmp/cachedir/ troubleshooting-wf1.cwl
38+
:working-directory: src/_includes/cwl/troubleshooting
6639
:emphasize-lines: 12-14, 19-21
67-
$ cwltool --cachedir /tmp/cachedir/ troubleshoot_wf1.cwl
68-
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220830195442
69-
INFO Resolved 'troubleshoot_wf1.cwl' to 'file:///tmp/troubleshoot_wf1.cwl'
70-
WARNING Workflow checker warning:
71-
troubleshoot_wf1.cwl:28:7: 'step_a_file' is not an input parameter of ordereddict([('class',
72-
'CommandLineTool'), ('inputs', []), ('outputs', []), ('arguments',
73-
['ouch', 'step_b.txt']), ('id',
74-
'_:af6cdc76-ead7-4438-94a2-f9f96b3d70c5')]), expected
75-
INFO [workflow ] start
76-
INFO [workflow ] starting step step_a
77-
INFO [step step_a] start
78-
INFO [job step_a] Output of job will be cached in /tmp/cachedir/5504f8afaebc04b48f07e6e5f2b5237b
79-
INFO [job step_a] /tmp/cachedir/5504f8afaebc04b48f07e6e5f2b5237b$ touch \
80-
step_a.txt
81-
INFO [job step_a] completed success
82-
INFO [step step_a] completed success
83-
INFO [workflow ] starting step step_b
84-
INFO [step step_b] start
85-
INFO [job step_b] Output of job will be cached in /tmp/cachedir/feec39505ecca29dce1a210f75b12283
86-
INFO [job step_b] /tmp/cachedir/feec39505ecca29dce1a210f75b12283$ ouch \
87-
step_b.txt
88-
ERROR 'ouch' not found: [Errno 2] No such file or directory: 'ouch'
89-
WARNING [job step_b] completed permanentFail
90-
WARNING [step step_b] completed permanentFail
91-
INFO [workflow ] completed permanentFail
92-
{}
93-
WARNING Final process status is permanentFail
9440
```
9541

9642
The workflow is in the `permanentFail` status due to `step_b` failing to execute the
9743
non-existent `ouch` command. The `step_a` was executed successfully and its output
9844
has been cached in your `cachedir` location. You can inspect the intermediate files
9945
created:
10046

101-
```{code-block} console
47+
```{runcmd} tree /tmp/cachedir
10248
:emphasize-lines: 4
103-
$ tree /tmp/cachedir
104-
/tmp/cachedir
105-
├── 5504f8afaebc04b48f07e6e5f2b5237b
106-
│ └── step_a.txt
107-
├── 5504f8afaebc04b48f07e6e5f2b5237b.status
108-
├── abz3v9aq
109-
├── feec39505ecca29dce1a210f75b12283
110-
└── feec39505ecca29dce1a210f75b12283.status
111-
112-
3 directories, 3 files
11349
```
11450

11551
Each workflow step has received a unique ID (the long value that looks like a hash).
@@ -122,31 +58,9 @@ as the previous time, note that now `cwltool` output contains information about
12258
pre-cached outputs for `step_a`, and about a new cache entry for the output of `step_b`.
12359
Also note that the status of `step_b` is now of success.
12460

125-
```{code-block} console
61+
```{runcmd} cwltool --cachedir /tmp/cachedir/ troubleshooting-wf1-stepb-fixed.cwl
62+
:working-directory: src/_includes/cwl/troubleshooting
12663
:emphasize-lines: 12, 16-18
127-
$ cwltool --cachedir /tmp/cachedir/ troubleshoot_wf1.cwl
128-
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220830195442
129-
INFO Resolved 'troubleshoot_wf1.cwl' to 'file:///tmp/troubleshoot_wf1.cwl'
130-
WARNING Workflow checker warning:
131-
troubleshoot_wf1.cwl:28:7: 'step_a_file' is not an input parameter of ordereddict([('class',
132-
'CommandLineTool'), ('inputs', []), ('outputs', []), ('arguments',
133-
['touch', 'step_b.txt']), ('id',
134-
'_:50e379f8-dce8-4794-9142-c53dc4e0e30d')]), expected
135-
INFO [workflow ] start
136-
INFO [workflow ] starting step step_a
137-
INFO [step step_a] start
138-
INFO [job step_a] Using cached output in /tmp/cachedir/5504f8afaebc04b48f07e6e5f2b5237b
139-
INFO [step step_a] completed success
140-
INFO [workflow ] starting step step_b
141-
INFO [step step_b] start
142-
INFO [job step_b] Output of job will be cached in /tmp/cachedir/822d8caf8894683f434ed8eb8be1b10d
143-
INFO [job step_b] /tmp/cachedir/822d8caf8894683f434ed8eb8be1b10d$ touch \
144-
step_b.txt
145-
INFO [job step_b] completed success
146-
INFO [step step_b] completed success
147-
INFO [workflow ] completed success
148-
{}
149-
INFO Final process status is success
15064
```
15165

15266
In this example the workflow step `step_a` was not re-evaluated as it had been cached, and

0 commit comments

Comments
 (0)