Where did those `/tmp` paths come from?
- -The CWL reference runner (cwltool) and other runners create temporary -directories with symbolic ("soft") links to your input files to ensure that -the tools aren't accidentally accessing files that were not explicitly -specified -``` - -The field `inputBinding` is optional and indicates whether and how the -input parameter should appear on the tool's command line. If -`inputBinding` is missing, the parameter does not appear on the command -line. Let's look at each example in detail. - -```cwl -example_flag: - type: boolean - inputBinding: - position: 1 - prefix: -f -``` - -Boolean types are treated as a flag. If the input parameter -"example_flag" is "true", then `prefix` will be added to the -command line. If false, no flag is added. - -```cwl -example_string: - type: string - inputBinding: - position: 3 - prefix: --example-string -``` - -String types appear on the command line as literal values. The `prefix` -is optional, if provided, it appears as a separate argument on the -command line before the parameter . In the example above, this is -rendered as `--example-string hello`. - -```cwl -example_int: - type: int - inputBinding: - position: 2 - prefix: -i - separate: false -``` - -Integer (and floating point) types appear on the command line with -decimal text representation. When the option `separate` is false (the -default value is true), the prefix and value are combined into a single -argument. In the example above, this is rendered as `-i42`. - - -```cwl -example_file: - type: File? - inputBinding: - prefix: --file= - separate: false - position: 4 -``` - -File types appear on the command line as the path to the file. When the -parameter type ends with a question mark `?` it indicates that the -parameter is optional. In the example above, this is rendered as -`--file=/tmp/random/path/whale.txt`. However, if the "example_file" -parameter were not provided in the input, nothing would appear on the -command line. - -Input files are read-only. If you wish to update an input file, you must -[first copy it to the output directory](/15-staging/index.md). - -The value of `position` is used to determine where parameter should -appear on the command line. Positions are relative to one another, not -absolute. As a result, positions do not have to be sequential, three -parameters with positions 1, 3, 5 will result in the same command -line as 1, 2, 3. More than one parameter can have the same position -(ties are broken using the parameter name), and the position field itself -is optional. The default position is 0. - -The `baseCommand` field will always appear in the final command line before the parameters. - -[touch]: http://www.linfo.org/touch.html diff --git a/04-output/index.md b/04-output/index.md deleted file mode 100644 index 3da13821..00000000 --- a/04-output/index.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "Where does the output of a command go?" -- "How can I save the output of a command?" -objectives: -- "Learn how to describe and handle outputs from a tool." -keypoints: -- "Outputs are described in the `outputs` section of a CWL description." -- "The field `outputBinding` describes how to set the value of each -output parameter." -- "Wildcards are allowed in the `glob` field." ---- - -# Returning Output Files - -The `outputs` of a tool is a list of output parameters that should be -returned after running the tool. Each parameter has an `id` for the name -of parameter, and `type` describing what types of values are valid for -that parameter. - -When a tool runs under CWL, the starting working directory is the -designated output directory. The underlying tool or script must record -its results in the form of files created in the output directory. The -output parameters returned by the CWL tool are either the output files -themselves, or come from examining the content of those files. - -The following example demonstrates how to return a file that has been extracted from a tar file. - -```{tip} -Passing mandatory arguments to the `baseCommand` - -In previous examples, the `baseCommand` was just a string, with any arguments passed as CWL inputs. -Instead of a single string we can use an _array of strings_. The first element is the command to run, and -any subsequent elements are mandatory command line arguments -``` - -*tar.cwl* - -```{literalinclude} /_includes/cwl/04-output/tar.cwl -:language: cwl -``` - -*tar-job.yml* - -```{literalinclude} /_includes/cwl/04-output/tar-job.yml -:language: yaml -``` - -Next, create a tar file for the example and invoke `cwl-runner` with the tool -wrapper and the input object on the command line: - -```bash -$ touch hello.txt && tar -cvf hello.tar hello.txt -$ cwl-runner tar.cwl tar-job.yml -[job tar.cwl] /tmp/tmpqOeawQ$ tar \ - --extract --file \ - /tmp/tmpGDk8Y1/stg80bbad20-494d-47af-8075-dffc32df03a3/hello.tar -[job tar.cwl] completed success -{ - "example_out": { - "location": "file:///home/me/cwl/user_guide/hello.txt", - "basename": "hello.txt", - "class": "File", - "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709", - "size": 0, - "path": "/home/me/cwl/user_guide/hello.txt" - } -} -Final process status is success -``` - -The field `outputBinding` describes how to set the value of each -output parameter. - -```cwl -outputs: - example_out: - type: File - outputBinding: - glob: hello.txt -``` - -The `glob` field consists of the name of a file in the output directory. -If you don't know name of the file in advance, you can use a wildcard pattern like `glob: '*.txt'`. diff --git a/05-stdout/index.md b/05-stdout/index.md deleted file mode 100644 index 50a896dd..00000000 --- a/05-stdout/index.md +++ /dev/null @@ -1,50 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "How do I capture the standard output stream of a command?" -objectives: -- "Learn how to capture streamed output from a tool." -keypoints: -- "Use the `stdout` field to specify a filename to capture streamed output." -- "The corresponding output parameter must have `type: stdout`." ---- - -# Capturing Standard Output - -To capture a tool's standard output stream, add the `stdout` field with -the name of the file where the output stream should go. Then add `type: -stdout` on the corresponding output parameter. - -*stdout.cwl* - -```{literalinclude} /_includes/cwl/05-stdout/stdout.cwl -:language: cwl -``` - -*echo-job.yml* - -```{literalinclude} /_includes/cwl/05-stdout/echo-job.yml -:language: yaml -``` - -Now invoke `cwl-runner` providing the tool wrapper and the input object -on the command line: - -```bash -$ cwl-runner stdout.cwl echo-job.yml -[job stdout.cwl] /tmp/tmpE0gTz7$ echo \ - 'Hello world!' > /tmp/tmpE0gTz7/output.txt -[job stdout.cwl] completed success -{ - "example_out": { - "location": "file:///home/me/cwl/user_guide/output.txt", - "basename": "output.txt", - "class": "File", - "checksum": "sha1$47a013e660d408619d894b20806b1d5086aab03b", - "size": 13, - "path": "/home/me/cwl/user_guide/output.txt" - } -} -Final process status is success -``` diff --git a/09-array-inputs/index.md b/09-array-inputs/index.md deleted file mode 100644 index 20627f09..00000000 --- a/09-array-inputs/index.md +++ /dev/null @@ -1,80 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "How do I provide multiple values for a single argument?" -objectives: -- "Learn how to provide parameter arrays as input to a tool." -- "Learn how to control the organization of array parameters on the command -line." -keypoints: -- "Array parameter definitions are nested under the `type` field with -`type: array`." -- "The appearance of array parameters on the command line differs depending on -with the `inputBinding` field is provided in the description." -- "Use the `itemSeparator` field to control concatenatation of array -parameters." ---- - -# Array Inputs - -It is easy to add arrays of input parameters represented to the command -line. There are two ways to specify an array parameter. First is to provide -`type` field with `type: array` and `items` defining the valid data types -that may appear in the array. Alternatively, brackets `[]` may be added after -the type name to indicate that input parameter is array of that type. - -*array-inputs.cwl* - -```{literalinclude} /_includes/cwl/09-array-inputs/array-inputs.cwl -:language: cwl -``` - -*array-inputs-job.yml* - -```{literalinclude} /_includes/cwl/09-array-inputs/array-inputs-job.yml -:language: yaml -``` - -Now invoke `cwl-runner` providing the tool wrapper and the input object -on the command line: - -```bash -$ cwl-runner array-inputs.cwl array-inputs-job.yml -[job array-inputs.cwl] /home/examples$ echo \ - -A \ - one \ - two \ - three \ - -B=four \ - -B=five \ - -B=six \ - -C=seven,eight,nine > /home/examples/output.txt -[job array-inputs.cwl] completed success -{ - "example_out": { - "location": "file:///home/examples/output.txt", - "basename": "output.txt", - "class": "File", - "checksum": "sha1$91038e29452bc77dcd21edef90a15075f3071540", - "size": 60, - "path": "/home/examples/output.txt" - } -} -Final process status is success -$ cat output.txt --A one two three -B=four -B=five -B=six -C=seven,eight,nine -``` - -The `inputBinding` can appear either on the outer array parameter definition -or the inner array element definition, and these produce different behavior when -constructing the command line, as shown above. -In addition, the `itemSeparator` field, if provided, specifies that array -values should be concatenated into a single argument separated by the item -separator string. - -Note that the arrays of inputs are specified inside square brackets `[]` in `array-inputs-job.yml`. Arrays can also be expressed over multiple lines, where -array values that are not defined with an associated key are marked by a leading `-`. -This will be demonstrated in the next lesson -and is discussed in more detail in the [YAML Guide](../yaml/index.md#arrays). -You can specify arrays of arrays, arrays of records, and other complex types. diff --git a/10-array-outputs/index.md b/10-array-outputs/index.md deleted file mode 100644 index c1928c63..00000000 --- a/10-array-outputs/index.md +++ /dev/null @@ -1,59 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "What do I do when a tool produces output in more than one file?" -- "How do I specify which output files should be kept?" -objectives: -- "Learn how to create arrays of output files." -keypoints: -- "You can capture multiple output files into an array of files using `glob`." -- "Use wildcards and filenames to specify the output files that will be returned -after tool execution." ---- - -# Array Outputs - -You can also capture multiple output files into an array of files using `glob`. - -*array-outputs.cwl* - -```{literalinclude} /_includes/cwl/10-array-outputs/array-outputs.cwl -:language: cwl -``` - -*array-outputs-job.yml* - -```{literalinclude} /_includes/cwl/10-array-outputs/array-outputs-job.yml -:language: yaml -``` - -Now invoke `cwl-runner` providing the tool wrapper and the input object -on the command line: - -```bash -$ cwl-runner array-outputs.cwl array-outputs-job.yml -[job 140190876078160] /home/example$ touch foo.txt bar.dat baz.txt -Final process status is success -{ - "output": [ - { - "size": 0, - "location": "foo.txt", - "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709", - "class": "File" - }, - { - "size": 0, - "location": "baz.txt", - "checksum": "sha1$da39a3ee5e6b4b0d3255bfef95601890afd80709", - "class": "File" - } - ] -} -``` - -As described in the [YAML Guide](../yaml/index.md#arrays), -the array of expected outputs is specified in `array-outputs-job.yml` with each -entry marked by a leading `-`. This format can also be used in CWL descriptions -to mark entries in arrays, as demonstrated in several of the upcoming sections. diff --git a/11-records/index.md b/11-records/index.md deleted file mode 100644 index 3a10ab8e..00000000 --- a/11-records/index.md +++ /dev/null @@ -1,110 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "How do I describe which parameters must and must not be used in combination?" -objectives: -- "Learn how to use records to describe the relationships between inputs." -keypoints: -- "Use the `record` field to group parameters together." -- "Multiple `record`s within the same parameter description are treated as -exclusive." ---- - -# Advanced Inputs - -Sometimes an underlying tool has several arguments that must be provided -together (they are dependent) or several arguments that cannot be provided -together (they are exclusive). You can use records and type unions to group -parameters together to describe these two conditions. - -*record.cwl* - -```{literalinclude} /_includes/cwl/11-records/record.cwl -:language: cwl -``` - -*record-job1.yml* - -```{literalinclude} /_includes/cwl/11-records/record-job1.yml -:language: yaml -``` - -```bash -$ cwl-runner record.cwl record-job1.yml -Workflow error, try again with --debug for more information: -Invalid job input record: -record-job1.yml:1:1: the `dependent_parameters` field is not valid because - missing required field `itemB` -``` - -In the first example, you can't provide `itemA` without also providing `itemB`. - -*record-job2.yml* - -```{literalinclude} /_includes/cwl/11-records/record-job2.yml -:language: yaml -``` - -```cwl -$ cwl-runner record.cwl record-job2.yml -record-job2.yml:6:3: invalid field `itemD`, expected one of: 'itemC' -[job record.cwl] /home/example$ echo \ - -A \ - one \ - -B \ - two \ - -C \ - three > /home/example/output.txt -[job record.cwl] completed success -{ - "example_out": { - "location": "file:///home/example/11-records/output.txt", - "basename": "output.txt", - "class": "File", - "checksum": "sha1$329fe3b598fed0dfd40f511522eaf386edb2d077", - "size": 23, - "path": "/home/example/output.txt" - } -} -Final process status is success -$ cat output.txt --A one -B two -C three -``` - -In the second example, `itemC` and `itemD` are exclusive, so only `itemC` -is added to the command line and `itemD` is ignored. - -*record-job3.yml* - -```{literalinclude} /_includes/cwl/11-records/record-job3.yml -:language: yaml -``` - -```bash -$ cwl-runner record.cwl record-job3.yml -[job record.cwl] /home/example$ echo \ - -A \ - one \ - -B \ - two \ - -D \ - four > /home/example/output.txt -[job record.cwl] completed success -{ - "example_out": { - "location": "file:///home/example/output.txt", - "basename": "output.txt", - "class": "File", - "checksum": "sha1$77f572b28e441240a5e30eb14f1d300bcc13a3b4", - "size": 22, - "path": "/home/example/output.txt" - } -} -Final process status is success -$ cat output.txt --A one -B two -D four -``` - -In the third example, only `itemD` is provided, so it appears on the -command line. diff --git a/21-1st-workflow/index.md b/21-1st-workflow/index.md deleted file mode 100644 index df8b7723..00000000 --- a/21-1st-workflow/index.md +++ /dev/null @@ -1,139 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "How do I connect tools together into a workflow?" -objectives: -- "Learn how to construct workflows from multiple CWL tool descriptions." -keypoints: -- "Each step in a workflow must have its own CWL description." -- "Top level inputs and outputs of the workflow are described in the `inputs` -and `outputs` fields respectively." -- "The steps are specified under `steps`." -- "Execution order is determined by the connections between steps." ---- - -# Writing Workflows - -This workflow extracts a java source file from a tar file and then -compiles it. - -*1st-workflow.cwl* - -```{literalinclude} /_includes/cwl/21-1st-workflow/1st-workflow.cwl -:language: cwl -``` - -```{admonition} Visualization of 1st-workflow.cwl -[](https://view.commonwl.org/workflows/github.com/common-workflow-language/user_guide/blob/main/_includes/cwl/21-1st-workflow/1st-workflow.cwl) -``` - -Use a YAML or a JSON object in a separate file to describe the input of a run: - -*1st-workflow-job.yml* - -```{literalinclude} /_includes/cwl/21-1st-workflow/1st-workflow-job.yml -:language: yaml -``` - -Now invoke `cwl-runner` with the tool wrapper and the input object on the -command line: - -```bash -$ echo "public class Hello {}" > Hello.java && tar -cvf hello.tar Hello.java -$ cwl-runner 1st-workflow.cwl 1st-workflow-job.yml -[job untar] /tmp/tmp94qFiM$ tar --create --file /home/example/hello.tar Hello.java -[step untar] completion status is success -[job compile] /tmp/tmpu1iaKL$ docker run -i --volume=/tmp/tmp94qFiM/Hello.java:/var/lib/cwl/job301600808_tmp94qFiM/Hello.java:ro --volume=/tmp/tmpu1iaKL:/var/spool/cwl:rw --volume=/tmp/tmpfZnNdR:/tmp:rw --workdir=/var/spool/cwl --read-only=true --net=none --user=1001 --rm --env=TMPDIR=/tmp java:7 javac -d /var/spool/cwl /var/lib/cwl/job301600808_tmp94qFiM/Hello.java -[step compile] completion status is success -[workflow 1st-workflow.cwl] outdir is /home/example -Final process status is success -{ - "compiled_class": { - "location": "/home/example/Hello.class", - "checksum": "sha1$e68df795c0686e9aa1a1195536bd900f5f417b18", - "class": "File", - "size": 416 - } -} -``` - -What's going on here? Let's break it down: - -```cwl -cwlVersion: v1.0 -class: Workflow -``` - -The `cwlVersion` field indicates the version of the CWL spec used by the -document. The `class` field indicates this document describes a workflow. - -```cwl -inputs: - tarball: File - name_of_file_to_extract: string -``` - -The `inputs` section describes the inputs of the workflow. This is a -list of input parameters where each parameter consists of an identifier -and a data type. These parameters can be used as sources for input to -specific workflows steps. - -```cwl -outputs: - compiled_class: - type: File - outputSource: compile/classfile -``` - -The `outputs` section describes the outputs of the workflow. This is a -list of output parameters where each parameter consists of an identifier -and a data type. The `outputSource` connects the output parameter `classfile` -of the `compile` step to the workflow output parameter `compiled_class`. - -```cwl -steps: - untar: - run: tar-param.cwl - in: - tarfile: tarball - extractfile: name_of_file_to_extract - out: [extracted_file] -``` - -The `steps` section describes the actual steps of the workflow. In this -example, the first step extracts a file from a tar file, and the second -step compiles the file from the first step using the java compiler. -Workflow steps are not necessarily run in the order they are listed, -instead the order is determined by the dependencies between steps (using -`source`). In addition, workflow steps which do not depend on one -another may run in parallel. - -The first step, `untar` runs `tar-param.cwl` (described previously in -[Parameter References](/06-params/index.md)). -This tool has two input parameters, `tarfile` and `extractfile` and one output -parameter `extracted_file`. - -The ``in`` section of the workflow step connects these two input parameters to -the inputs of the workflow, `tarball` and `name_of_file_to_extract` using -`source`. This means that when the workflow step is executed, the values -assigned to `tarball` and `name_of_file_to_extract` will be used for the -parameters `tarfile` and `extractfile` in order to run the tool. - -The `out` section of the workflow step lists the output parameters that are -expected from the tool. - -```cwl - compile: - run: arguments.cwl - in: - src: untar/extracted_file - out: [classfile] -``` - -The second step `compile` depends on the results from the first step by -connecting the input parameter `src` to the output parameter of `untar` using -`untar/extracted_file`. It runs `arguments.cwl` (described previously in -[Additional Arguments and Parameters](/08-arguments/index.md)). -The output of this step `classfile` is connected to the -`outputs` section for the Workflow, described above. diff --git a/22-nested-workflows/index.md b/22-nested-workflows/index.md deleted file mode 100644 index a70cbbd6..00000000 --- a/22-nested-workflows/index.md +++ /dev/null @@ -1,130 +0,0 @@ ---- -teaching: 10 -exercises: 0 -questions: -- "How do I connect multiple workflows together?" -objectives: -- "Learn how to construct nested workflows from multiple CWL workflow -descriptions." -keypoints: -- "A workflow can be used as a step in another workflow, if the workflow engine -supports the `SubworkflowFeatureRequirement`." -- "The workflows are specified under `steps`, with the worklow's description -file provided as the value to the `run` field." -- "Use `default` to specify a default value for a field, which can be -overwritten by a value in the input object." -- "Use `>` to ignore newlines in long commands split over multiple lines." ---- - -# Nested Workflows - -Workflows are ways to combine multiple tools to perform a larger operations. -We can also think of a workflow as being a tool itself; a CWL workflow can be -used as a step in another CWL workflow, if the workflow engine supports the -`SubworkflowFeatureRequirement`: - -```cwl -requirements: - SubworkflowFeatureRequirement: {} -``` - -Here's an example workflow that uses our `1st-workflow.cwl` as a nested -workflow: - -*nestedworkflows.cwl* - -```{literalinclude} /_includes/cwl/22-nested-workflows/nestedworkflows.cwl -:language: cwl -``` - -```{note} -Visualization of the workflow and the inner workflow from its `compile` step
-
-This two-step workflow starts with the `create-tar` step which is connected to
-the `compile` step in orange; `compile` is another workflow, diagrammed on the
-right. In purple we see the fixed string `"Hello.java"` being supplied as the
-`name_of_file_to_extract`.
-
-
-
-```
-
-A CWL `Workflow` can be used as a `step` just like a `CommandLineTool`, its CWL
-file is included with `run`. The workflow inputs (`tarball` and `name_of_file_to_extract`) and outputs
-(`compiled_class`) then can be mapped to become the step's input/outputs.
-
-```cwl
- compile:
- run: 1st-workflow.cwl
- in:
- tarball: create-tar/tar_compressed_java_file
- name_of_file_to_extract:
- default: "Hello.java"
- out: [compiled_class]
-```
-
-Our `1st-workflow.cwl` was parameterized with workflow inputs, so when running
-it we had to provide a job file to denote the tar file and `*.java` filename.
-This is generally best-practice, as it means it can be reused in multiple parent
-workflows, or even in multiple steps within the same workflow.
-
-Here we use `default:` to hard-code `"Hello.java"` as the `name_of_file_to_extract`
-input, however our workflow also requires a tar file at `tarball`, which we will
-prepare in the `create-tar` step. At this point it is probably a good idea to refactor
-`1st-workflow.cwl` to have more specific input/output names, as those also
-appear in its usage as a tool.
-
-It is also possible to do a less generic approach and avoid external
-dependencies in the job file. So in this workflow we can generate a hard-coded
-`Hello.java` file using the previously mentioned `InitialWorkDirRequirement`
-requirement, before adding it to a tar file.
-
-```cwl
- create-tar:
- requirements:
- InitialWorkDirRequirement:
- listing:
- - entryname: Hello.java
- entry: |
- public class Hello {
- public static void main(String[] argv) {
- System.out.println("Hello from Java");
- }
- }
-```
-
-In this case our step can assume `Hello.java` rather than be parameterized, so
-we can use hardcoded values `hello.tar` and `Hello.java` in a `baseCommand` and
-the resulting `outputs`:
-
-```cwl
- run:
- class: CommandLineTool
- inputs: []
- baseCommand: [tar, --create, --file=hello.tar, Hello.java]
- outputs:
- tar_compressed_java_file:
- type: File
- streamable: true
- outputBinding:
- glob: "hello.tar"
-```
-
-Did you notice that we didn't split out the `tar --create` tool to a separate file,
-but rather embedded it within the CWL Workflow file? This is generally not best
-practice, as the tool then can't be reused. The reason for doing it in this case
-is because the command line is hard-coded with filenames that only make sense
-within this workflow.
-
-In this example we had to prepare a tar file outside, but only because our inner
-workflow was designed to take that as an input. A better refactoring of the
-inner workflow would be to take a list of Java files to compile, which would
-simplify its usage as a tool step in other workflows.
-
-Nested workflows can be a powerful feature to generate higher-level functional
-and reusable workflow units - but just like for creating a CWL Tool description,
-care must be taken to improve its usability in multiple workflows.
diff --git a/23-scatter-workflow/index.md b/23-scatter-workflow/index.md
deleted file mode 100644
index 8576dc96..00000000
--- a/23-scatter-workflow/index.md
+++ /dev/null
@@ -1,163 +0,0 @@
----
-teaching: 10
-exercises: 0
-questions:
-- "How do I run tools or workflows in parallel?"
-objectives:
-- "Learn how to create workflows that can run a step over a list of inputs."
-keypoints:
-- "A workflow can scatter over an input array in a step of a workflow, if the workflow engine
-supports the `ScatterFeatureRequirement`."
-- The `scatter` field is specified for each step you want to scatter
-- The `scatter` field references the step level inputs, not the workflow inputs
-- Scatter runs on each step specified independently
----
-
-# Scattering Workflows
-
-Now that we know how to write workflows, we can start utilizing the `ScatterFeatureRequirement`.
-This feature tells the runner that you wish to run a tool or workflow multiple times over a list
-of inputs. The workflow then takes the input(s) as an array and will run the specified step(s)
-on each element of the array as if it were a single input. This allows you to run the same workflow
-on multiple inputs without having to generate many different commands or input yaml files.
-
-```cwl
-requirements:
- ScatterFeatureRequirement: {}
-```
-
-The most common reason a new user might want to use scatter is to perform the same analysis on
-different samples. Let's start with a simple workflow that calls our first example and takes
-an array of strings as input to the workflow:
-
-*scatter-workflow.cwl*
-
-```{literalinclude} /_includes/cwl/23-scatter-workflow/scatter-workflow.cwl
-:language: cwl
-```
-
-Aside from the `requirements` section including `ScatterFeatureRequirement`, what is
-going on here?
-
-```cwl
-inputs:
- message_array: string[]
-```
-
-First of all, notice that the main workflow level input here requires an array of strings.
-
-```cwl
-steps:
- echo:
- run: 1st-tool.cwl
- scatter: message
- in:
- message: message_array
- out: []
-```
-
-Here we've added a new field to the step `echo` called `scatter`. This field tells the
-runner that we'd like to scatter over this input for this particular step. Note that
-the input name listed after scatter is the one of the step's input, not a workflow level input.
-
-For our first scatter, it's as simple as that! Since our tool doesn't collect any outputs, we
-still use `outputs: []` in our workflow, but if you expect that the final output of your
-workflow will now have multiple outputs to collect, be sure to update that to an array type
-as well!
-
-Using the following input file:
-
-*scatter-job.yml*
-
-```{literalinclude} /_includes/cwl/23-scatter-workflow/scatter-job.yml
-:language: yaml
-```
-
-As a reminder, `1st-tool.cwl` (described previously in
-[First Example](/02-1st-example/index.md)) simply calls the command `echo` on a message. If we invoke
-`cwl-runner scatter-workflow.cwl scatter-job.yml` on the command line:
-
-```bash
-$ cwl-runner scatter-workflow.cwl scatter-job.yml
-[workflow scatter-workflow.cwl] start
-[step echo] start
-[job echo] /tmp/tmp0hqmg400$ echo \
- 'Hello world!'
-Hello world!
-[job echo] completed success
-[step echo] start
-[job echo_2] /tmp/tmpu65_m1zw$ echo \
- 'Hola mundo!'
-Hola mundo!
-[job echo_2] completed success
-[step echo] start
-[job echo_3] /tmp/tmp5cs7a2wh$ echo \
- 'Bonjour le monde!'
-Bonjour le monde!
-[job echo_3] completed success
-[step echo] start
-[job echo_4] /tmp/tmp301wo7p8$ echo \
- 'Hallo welt!'
-Hallo welt!
-[job echo_4] completed success
-[step echo] completed success
-[workflow scatter-workflow.cwl] completed success
-{}
-Final process status is success
-```
-
-You can see that the workflow calls echo multiple times on each element of our
-`message_array`. Ok, so how about if we want to scatter over two steps in a workflow?
-
-Let's perform a simple echo like above, but capturing `stdout` by adding the following
-lines instead of `outputs: []`
-
-*1st-tool-mod.cwl*
-
-```cwl
-outputs:
- echo_out:
- type: stdout
-```
-
-And add a second step that uses `wc` to count the characters in each file. See the tool
-below:
-
-*wc-tool.cwl*
-
-```{literalinclude} /_includes/cwl/23-scatter-workflow/wc-tool.cwl
-:language: cwl
-```
-
-Now, how do we incorporate scatter? Remember the scatter field is under each step:
-
-*scatter-two-steps.cwl*
-
-```{literalinclude} /_includes/cwl/23-scatter-workflow/scatter-two-steps.cwl
-:language: cwl
-```
-
-Here we have placed the scatter field under each step. This is fine for this example since
-it runs quickly, but if you're running many samples for a more complex workflow, you may
-wish to consider an alternative. Here we are running scatter on each step independently, but
-since the second step is not dependent on the first step completing all languages, we aren't
-using the scatter functionality efficiently. The second step expects an array as input from
-the first step, so it will wait until everything in step one is finished before doing anything.
-Pretend that `echo Hello World!` takes 1 minute to perform, `wc -c` on the output takes 3 minutes
-and that `echo Hallo welt!` takes 5 minutes to perform, and `wc` on that output takes 3 minutes.
-Even though `echo Hello World!` could finish in 4 minutes, it will actually finish in 8 minutes
-because the first step must wait on `echo Hallo welt!`. You can see how this might not scale
-well.
-
-Ok, so how do we scatter on steps that can proceed independent of other samples? Remember from
-[chapter 21](/22-nested-workflows/index.md), that we can make an entire workflow a single step in another workflow! Convert our
-two step workflow to a single step subworkflow:
-
-*scatter-nested-workflow.cwl*
-
-```{literalinclude} /_includes/cwl/23-scatter-workflow/scatter-nested-workflow.cwl
-:language: cwl
-```
-
-Now the scatter acts on a single step, but that step consists of two steps so each step is performed
-in parallel.
diff --git a/24_conditional-workflow/index.md b/24_conditional-workflow/index.md
deleted file mode 100644
index 4229d369..00000000
--- a/24_conditional-workflow/index.md
+++ /dev/null
@@ -1,132 +0,0 @@
----
-teaching: 24
-exercises: 0
-questions:
-- "How do you write workflows with conditions?"
-objectives:
-- "Learn how to construct workflows containing conditional steps."
-keypoints:
-- ""
----
-
-# Conditional workflows
-
-This workflow contains a conditional step and is executed based on the input.
-This allows workflows to skip additional steps based on input parameters given at the start of the program or by previous steps.
-
-*conditional-workflow.cwl*
-
-```cwl
-class: Workflow
-cwlVersion: v1.2
-inputs:
- val: int
-
-steps:
-
- step1:
- in:
- in1: val
- a_new_var: val
- run: foo.cwl
- when: $(inputs.in1 < 1)
- out: [out1]
-
- step2:
- in:
- in1: val
- a_new_var: val
- run: foo.cwl
- when: $(inputs.a_new_var > 2)
- out: [out1]
-
-outputs:
- out1:
- type: string
- outputSource:
- - step1/out1
- - step2/out1
- pickValue: first_non_null
-
-requirements:
- InlineJavascriptRequirement: {}
- MultipleInputFeatureRequirement: {}
-```
-
-The first thing you'll notice is that this workflow is only compatible for version 1.2 or greater of the CWL standards.
-
-```cwl
-class: Workflow
-cwlVersion: v1.2
-```
-
-The first step of the worklfow (step1) contains two input properties and will execute foo.cwl when the conditions are met. The new property `when` is where the condition validation takes place. In this case only when `in1` from the workflow contains a value `< 1` this step will be executed.
-
-```cwl
-steps:
-
- step1:
- in:
- in1: val
- a_new_var: val
- run: foo.cwl
- when: $(inputs.in1 < 1)
- out: [out1]
-```
-
-Using the following command `cwltool cond-wf-003.1.cwl --val 0` the value will pass the first conditional step and will therefor be executed and is shown in the log by `INFO [step step1] start` whereas the second step is skipped as indicated by `INFO [step step2] will be skipped`.
-
-```bash
-INFO [workflow ] start
-INFO [workflow ] starting step step1
-INFO [step step1] start
-INFO [job step1] /private/tmp/docker_tmpdcyoto2d$ echo
-
-INFO [job step1] completed success
-INFO [step step1] completed success
-INFO [workflow ] starting step step2
-INFO [step step2] will be skipped
-INFO [step step2] completed skipped
-INFO [workflow ] completed success
-{
- "out1": "foo 0"
-}
-INFO Final process status is success
-```
-
-When a value of 3 is given the first conditional step will not be executed but the second step will `cwltool cond-wf-003.1.cwl --val 3`.
-
-```bash
-INFO [workflow ] start
-INFO [workflow ] starting step step1
-INFO [step step1] will be skipped
-INFO [step step1] completed skipped
-INFO [workflow ] starting step step2
-INFO [step step2] start
-INFO [job step2] /private/tmp/docker_tmpqwr93mxx$ echo
-
-INFO [job step2] completed success
-INFO [step step2] completed success
-INFO [workflow ] completed success
-{
- "out1": "foo 3"
-}
-INFO Final process status is success
-```
-
-If no conditions are met for example when using `--val 2` the workflow will raise a permanentFail.
-
-```bash
-cwltool cond-wf-003.1.cwl --val 2
-
-INFO [workflow ] start
-INFO [workflow ] starting step step1
-INFO [step step1] will be skipped
-INFO [step step1] completed skipped
-INFO [workflow ] starting step step2
-INFO [step step2] will be skipped
-INFO [step step2] completed skipped
-ERROR [workflow ] Cannot collect workflow output: All sources for 'out1' are null
-INFO [workflow ] completed permanentFail
-WARNING Final process status is permanentFail
-```
diff --git a/AUTHORS b/AUTHORS.md
similarity index 100%
rename from AUTHORS
rename to AUTHORS.md
diff --git a/CITATION b/CITATION.md
similarity index 100%
rename from CITATION
rename to CITATION.md
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index bc40b2fe..70acd090 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -26,23 +26,28 @@ or a factual error.
This is a good way to introduce yourself
and to meet some of our community members.
-1. If you do not have a [GitHub][github] account,
+1. If you do not have a [GitHub][github] account,
you can [send us comments by email][discuss-list].
However,
we will be able to respond more quickly if you use one of the other methods described below.
-2. If you have a [GitHub][github] account,
+2. If you have a [GitHub][github] account,
or are willing to [create one][github-join],
but do not know how to use Git,
you can report problems or suggest improvements by [creating an issue][issues].
This allows us to assign the item to someone
and to respond to it in a threaded discussion.
-3. If you are comfortable with Git,
+3. If you are comfortable with Git,
and would like to add or change material,
you can submit a pull request (PR).
Instructions for doing this are [included below][#using-github].
+4. To build and run the user guide locally, see [building][#building].
+
+Pull requests include an automatic preview provided by
+[ReadTheDocs](https://readthedocs.org/projects/common-workflow-languageuser-guide/).
+
## What to Contribute
There are many ways to contribute,
@@ -85,6 +90,27 @@ your clone of the originating `main` branch.
Lastly, published copies of all the sections are available in the `main` branch of the originating
repository for reference while revising.
+## Building
+
+The user guide uses [Sphinx](https://www.sphinx-doc.org/), a Python documentation
+tool. You must have a recent version of Python 3.6+ installed to build the project
+locally. It is also recommended having `make` (otherwise look at the commands used
+in `Makefile`).
+
+```bash
+# Create and activate a virtual environment
+python -m venv venv
+source venv/bin/activate
+# Install the dependencies in your virtual environment
+(venv) pip install -e .[all]
+# Create the HTML to visualize locally
+(venv) make html
+(venv) firefox _build/index.html
+# Or you can start a serve that watches for local file changes
+(venv) make watch
+# Open
| - {{ episode.title }} - | -
-
|
-
- The Carpentries comprises - Software Carpentry, Data Carpentry, and Library Carpentry communities of Instructors, Trainers, - Maintainers, helpers, and supporters who share a mission to teach - foundational coding and data science skills to researchers and people - working in library- and information-related roles. In January, - 2018, The Carpentries was formed by the merger of Software Carpentry and - Data Carpentry. Library Carpentry became an official Carpentries Lesson Program in November 2018.
- -While individual lessons and workshops continue to be run under each - lesson project, The Carpentries provide overall staffing and governance, as - well as support for assessment, instructor training and mentoring. - Memberships are joint, and the Carpentries project maintains a shared Code - of Conduct. The Carpentries is a fiscally sponsored project of Community - Initiatives, a registered 501(c)3 non-profit based in California, USA.
-
- Since 1998, Software Carpentry has - been teaching researchers across all disciplines the foundational coding - skills they need to get more done in less time and with less pain. Its - volunteer instructors have run hundreds of events for thousands of learners - around the world. Now that all research involves some degree of - computational work, whether with big data, cloud computing, or simple task - automation, these skills are needed more than ever.
-
- Data Carpentry develops and teaches - workshops on the fundamental data skills needed to conduct research. Its - target audience is researchers who have little to no prior computational - experience, and its lessons are domain specific, building on learners' - existing knowledge to enable them to quickly apply skills learned to their - own research. Data Carpentry workshops take researchers through the entire - data life cycle.
-
- Library Carpentry develops lessons and - teaches workshops for and with people working in library- and - information-related roles. Its goal is to create an on-ramp to empower this - community to use software and data in their own work, as well as be - advocates for and train others in efficient, effective and reproducible data - and software practices.
-- Data Carpentry - aims to help researchers get their work done - in less time and with less pain - by teaching them basic research computing skills. - This hands-on workshop will cover basic concepts and tools, - including program design, version control, data management, - and task automation. - Participants will be encouraged to help one another - and to apply what they have learned to their own research problems. -
-- - For more information on what we teach and why, - please see our paper - "Best Practices for Scientific Computing". - -
diff --git a/_includes/dc/schedule.html b/_includes/dc/schedule.html deleted file mode 100644 index 6894a19e..00000000 --- a/_includes/dc/schedule.html +++ /dev/null @@ -1,24 +0,0 @@ -| 09:00 | Automating tasks with the Unix shell |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | Building programs with Python |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
| 09:00 | Version control with Git |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | Managing data with SQL |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
add, commit, ...status, diff, ...clone, pull, push, ...- Who: - The course is aimed at graduate students and other researchers. - - You don't need to have any previous knowledge of the tools - that will be presented at the workshop. - -
diff --git a/_includes/episode_break.html b/_includes/episode_break.html deleted file mode 100644 index 3b51c46d..00000000 --- a/_includes/episode_break.html +++ /dev/null @@ -1,13 +0,0 @@ -% Display a break's timings in a box similar to a learning episode's. --diff --git a/_includes/episode_keypoints.html b/_includes/episode_keypoints.html deleted file mode 100644 index 5c20348c..00000000 --- a/_includes/episode_keypoints.html +++ /dev/null @@ -1,9 +0,0 @@ -% Display key points for an episode. -Overview
- --- -- Break: {{ page.break }} min ----
-diff --git a/_includes/episode_navbar.html b/_includes/episode_navbar.html deleted file mode 100644 index 1bdf6107..00000000 --- a/_includes/episode_navbar.html +++ /dev/null @@ -1,38 +0,0 @@ -% For some reason, the relative_root_path seems out of scope in this file, so we -% need to re-assign it here - -```{include} /_includes/base_path.html -``` - -% Navigation bar for an episode. - -```{include} /_includes/manual_episode_order.html -``` -% 'previous_episode' and 'next_episodes' are defined in 'manual_episode_order.html'. -% These replace 'page.previous' and 'page.next' objects, correspondingly. - -Key Points
-- {% for keypoint in page.keypoints %} -
-- {{ keypoint|markdownify }}
- {% endfor %} -
-diff --git a/_includes/episode_title.html b/_includes/episode_title.html deleted file mode 100644 index d0abc654..00000000 --- a/_includes/episode_title.html +++ /dev/null @@ -1,9 +0,0 @@ -Overview
- --- -- Teaching: {{ page.teaching }} min --
- Exercises: {{ page.exercises }} min -- Questions --- {% for question in page.questions %} -
-- {{ question|markdownify }}
- {% endfor %} --- ---- Objectives --- {% for objective in page.objectives %} -
-- {{ objective|markdownify }}
- {% endfor %} -
- Library Carpentry - is made by people working in library- and information-related roles to help you: -
-- - Library Carpentry introduces you to the fundamentals of computing - and provides you with a platform for further self-directed learning. - For more information on what we teach and why, please see our paper - "Library Carpentry: software skills training for library professionals". - -
diff --git a/_includes/lc/schedule.html b/_includes/lc/schedule.html deleted file mode 100644 index cc2b5920..00000000 --- a/_includes/lc/schedule.html +++ /dev/null @@ -1,24 +0,0 @@ -| 09:00 | Data Intro for Librarians |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | Shell Lessons for Libraries |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
| 09:00 | Git Intro for Librarians |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | OpenRefine for Librarians |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
add, commit, ...statusclone, pull, push, ...- Who: - The course is for people working in library- and information-related roles. - - You don't need to have any previous knowledge of the tools that - will be presented at the workshop. - -
diff --git a/_includes/lesson_footer.html b/_includes/lesson_footer.html deleted file mode 100644 index 52b8d97a..00000000 --- a/_includes/lesson_footer.html +++ /dev/null @@ -1,53 +0,0 @@ -% Footer for lesson pages. - -```{include} /_includes/gh_variables.html -``` - - diff --git a/_includes/life_cycle.html b/_includes/life_cycle.html deleted file mode 100644 index 222eb147..00000000 --- a/_includes/life_cycle.html +++ /dev/null @@ -1,32 +0,0 @@ - -{% if site.life_cycle == "pre-alpha" %} - -- Software Carpentry - aims to help researchers get their work done - in less time and with less pain - by teaching them basic research computing skills. - This hands-on workshop will cover basic concepts and tools, - including program design, version control, data management, - and task automation. - Participants will be encouraged to help one another - and to apply what they have learned to their own research problems. -
-- - For more information on what we teach and why, - please see our paper - "Best Practices for Scientific Computing". - -
diff --git a/_includes/sc/schedule.html b/_includes/sc/schedule.html deleted file mode 100644 index 6894a19e..00000000 --- a/_includes/sc/schedule.html +++ /dev/null @@ -1,24 +0,0 @@ -| 09:00 | Automating tasks with the Unix shell |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | Building programs with Python |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
| 09:00 | Version control with Git |
| 10:30 | Coffee |
| 12:00 | Lunch break |
| 13:00 | Managing data with SQL |
| 14:30 | Coffee |
| 16:00 | Wrap-up |
add, commit, ...status, diff, ...clone, pull, push, ...- Who: - The course is aimed at graduate students and other researchers. - - You don't need to have any previous knowledge of the tools - that will be presented at the workshop. - -
diff --git a/_includes/syllabus.html b/_includes/syllabus.html deleted file mode 100644 index 5d6ef79c..00000000 --- a/_includes/syllabus.html +++ /dev/null @@ -1,93 +0,0 @@ -```{include} /_includes/base_path.html -``` - -% Display syllabus in tabular form. -% Days are displayed if at least one episode has 'start = true'. - -```{include} /_includes/manual_episode_order.html -``` - -| {% endif %} - | - | Setup | -Download files required for the lesson | -
| {% endif %} - | {% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }} | -Finish | -- |
| {% if episode.start %}Day {{ day }}{% endif %} | {% endif %} -{% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }} | -- {% assign lesson_number = lesson_number | plus: 1 %} - {{ lesson_number }}. {{ episode.title }} - | -
- {% if episode.break %}
- Break
- {% else %}
- {% if episode.questions %}
- {% for question in episode.questions %}
- {{question|markdownify|strip_html}}
- {% unless forloop.last %}
- - {% endunless %} - {% endfor %} - {% endif %} - {% endif %} - |
-
| {% endif %} - | {% if hours < 10 %}0{% endif %}{{ hours }}:{% if minutes < 10 %}0{% endif %}{{ minutes }} | -Finish | -- |
- The actual schedule may vary slightly depending on the topics and exercises chosen by the instructor. -
- -{{page.humandate}}
-{% if page.humantime %}{{page.humantime}}{% endif %}
-- Instructors: - {% if page.instructor %} - {{page.instructor | join: ', ' %}} - {% else %} - to be announced. - {% endif %} -
- {% if page.helper %} -- Helpers: - {{page.helper | join: ', ' %}} -
- {% endif %} -Contributions and Feedback are Welcome!
- -This document is a work in progress. Not all features are covered, yet. -If you find that something is missing from this guide, -or if you'd like to provide other feedback, -we would be delighted if you would file an Issue on the -[project repository for this guide][repo]. -You can also suggest changes directly -by clicking the "Edit on GitHub" button at the top-right -of the relevant page. -``` - -```{admonition} Prerequisites -* A text editor - -* A CWL runner. It is recommended to start with the [reference implementation][cwltool-install]. The full list of CWL runners is on [the project homepage][cwl-runners-list]. -``` - -You also may be interested in: -1. [A quick tutorial for the subset of YAML used in CWL](/yaml/index.md) -2. [CWL Recommended Practices](/rec-practices/index.md) -3. [Miscellaneous CWL tips](/misc/index.md) -4. [CWL patterns](https://github.com/common-workflow-library/cwl-patterns) - -[cwl-runners-list]: https://www.commonwl.org/#Implementations -[cwltool-install]: https://github.com/common-workflow-language/cwltool#install - -## Schedule - -```{toctree} -:maxdepth: 1 -:hidden: - -CODE_OF_CONDUCT.md -setup.md -``` - -```{toctree} -:maxdepth: 2 - -episodes.md -``` - -```{toctree} -:maxdepth: 1 -:hidden: - -extras.md -LICENSE.md -``` - -[repo]: https://github.com/common-workflow-language/user_guide/issues diff --git a/_episodes_rmd/.gitkeep b/pyproject.toml similarity index 100% rename from _episodes_rmd/.gitkeep rename to pyproject.toml diff --git a/rec-practices/index.md b/rec-practices/index.md deleted file mode 100644 index fe51a385..00000000 --- a/rec-practices/index.md +++ /dev/null @@ -1,100 +0,0 @@ -# Recommended Practices - -Below are a set of recommended good practices to keep in mind when writing a -Common Workflow Language description for a tool or workflow. These guidelines -are presented for consideration on a scale of usefulness: more is better, not -all are required. - -☐ No `type: string` parameters for names of input or reference -files/directories; use `type: File` or `type: Directory` as appropriate. - -☐ Include a license that allows for re-use by anyone, e.g. -[Apache 2.0][apache-license]. If possible, the license should be specified with -its corresponding [SPDX identifier][spdx]. Construct the metadata field for the -licence by providing a URL of the form `https://spdx.org/licenses/[SPDX-ID]` -where `SPDX-ID` is the taken from the list of identifiers linked above. See the -example snippet below for guidance. For non-standard licenses without an SPDX -identifier, provide a URL to the license. - -_Example of metadata field for license with SPDX identifier:_ - -```cwl -$namespaces: - s: https://schema.org/ -s:license: https://spdx.org/licenses/Apache-2.0 -# other s: declarations -``` - -For more examples of providing metadata within CWL descriptions, see the -[Metadata and Authorship section](/17-metadata/index.md) -of this User Guide. - -☐ Include [attribution information][license-example] for the author(s) of -the CWL tool or workflow description. Use unambiguous identifiers like -[ORCID][orcid]. - -☐ In tool descriptions, list dependencies using short name(s) under -`SoftwareRequirement`. - -☐ Include [SciCrunch][scicrunch] identifiers for dependencies in -`https://identifiers.org/rrid/RRID:SCR_NNNNNN` format. - -☐ All `input` and `output` identifiers should reflect their conceptual -identity. Use informative names like `unaligned_sequences`, `reference_genome`, -`phylogeny`, or `aligned_sequences` instead of `foo_input`, `foo_file`, -`result`, `input`, `output`, and so forth. - -☐ In tool descriptions, include a list of version(s) of the tool that are -known to work with this description under `SoftwareRequirement`. - -☐ `format` should be specified for all input and output `File`s. -Bioinformatics tools should use format identifiers from [EDAM][edam-example]. -See also `iana:text/plain`, `iana:text/tab-separated-values` with -`$namespaces: { iana: "https://www.iana.org/assignments/media-types/" }`. -[Full IANA media type list][iana-types] (also known as MIME types). For -non-bioinformatics tools use or build an appropriate ontology/controlled -vocabulary in the same way. Please edit this page to let us know about it. - -☐ Mark all input and output `File`s that are read from or written to in a -streaming compatible way (only once, no random-access), as `streamable: true`. - -☐ Each `CommandLineTool` description should focus on a single operation -only, even if the (sub)command is capable of more. Don't overcomplicate your -tool descriptions with options that you don't need/use. - -☐ Custom types should be defined with one external YAML per type -definition for re-use. - -☐ Include a top level short `label` summarising the tool/workflow. - -☐ If useful, include a top level `doc` as well. This should provide a -longer, more detailed description than was provided in the top level `label` -(see above). - -☐ Use `type: enum` instead of `type: string` for elements with a fixed -list of valid values. - -☐ Evaluate all use of JavaScript for possible elimination or replacement. -One common example: manipulating `File` names and paths? Consider whether one -of the [built in `File` properties][file-prop] like `basename`, `nameroot`, -`nameext`, etc, could be used instead. - -☐ Give the tool description to a colleague (preferably at a different -institution) to test and provide feedback. - -☐ Complex workflows with individual components which can be abstracted -should utilise the [`SubworkflowFeatureRequirement`][subworkflow] to make their -workflow modular and allow sections of them to be easily reused. - -☐ Software containers should be made to be conformant to the ["Recommendations for the packaging and containerizing of bioinformatics software"][containers] (also useful to other disciplines). - -[containers]: https://doi.org/10.12688/f1000research.15140.1 -[apache-license]: https://spdx.org/licenses/Apache-2.0.html -[license-example]: https://github.com/ProteinsWebTeam/ebi-metagenomics-cwl/blob/master/workflows/emg-assembly.cwl#L200 -[scicrunch]: https://scicrunch.org -[edam-example]: http://edamontology.org/format_1915 -[iana-types]: https://www.iana.org/assignments/media-types/media-types.xhtml -[file-prop]: https://www.commonwl.org/v1.0/CommandLineTool.html#File -[orcid]: https://orcid.org -[subworkflow]: https://www.commonwl.org/v1.0/Workflow.html#SubworkflowFeatureRequirement -[spdx]: https://spdx.org/licenses/ diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 72893bff..00000000 --- a/requirements.txt +++ /dev/null @@ -1,4 +0,0 @@ -myst-parser==0.17.* -pydata-sphinx-theme==0.8.* -sphinx==4.5.* -sphinx-autobuild==2021.3.14 diff --git a/setup.cfg b/setup.cfg new file mode 100644 index 00000000..2566cfb8 --- /dev/null +++ b/setup.cfg @@ -0,0 +1,52 @@ +[metadata] +name=cwl-user-guide +version=attr: cwl.doc.__version__ +url=https://www.commonwl.org/user_guide/ +description=CWL User Guide +long_description=file: README.md +long_description_content_type=text/markdown +project_urls= + Documentation=https://www.commonwl.org/user_guide/ + Source=https://github.com/common-workflow-language/user_guide + Tracker=https://github.com/common-workflow-language/user_guide/issues +license=GPL +platforms=any +classifiers= + Environment :: Console + Environment :: Web Environment + Intended Audience :: Developers + Intended Audience :: System Administrators + Intended Audience :: Science/Research + License :: OSI Approved :: GNU General Public License v3 (GPLv3) + Operating System :: POSIX :: Linux + Programming Language :: Python + Programming Language :: Python :: 3.6 + Programming Language :: Python :: 3.7 + Programming Language :: Python :: 3.8 + Programming Language :: Python :: 3.9 + Programming Language :: Python :: 3 :: Only + Programming Language :: Python :: Implementation :: CPython + +[options] +packages = find_namespace: +include_package_data = True +python_requires = >=3.6 +install_requires = + myst-parser==0.* + pydata-sphinx-theme==0.* + sphinx==5.* + sphinx-reredirects==0.1.* + cwl-utils==0.* + +[options.packages.find] +include = cwl* + +[options.extras_require] +test = + cwltest==2.* + cwltool==3.1.* +watch = + sphinx-autobuild==2021.3.* +all = + %(test)s + %(watch)s diff --git a/setup.md b/setup.md deleted file mode 100644 index 39c945e8..00000000 --- a/setup.md +++ /dev/null @@ -1,19 +0,0 @@ -# Setup - -To follow the Common Workflow Language user guide, you will need an up-to-date -Linux or Mac OS X system, with the following software installed: - -- A text editor with YAML support, for example [Visual Studio Code with the Benten (CWL) plugin](https://github.com/rabix/benten#install-vs-code-extension). If you want to use a different text editor then review [this list of plugins that add CWL support to code editors](https://www.commonwl.org/#Editors_and_viewers) -- A CWL implementation to run the described tools and workflows. When first -getting started with CWL, we recommend the reference implementation, -[`cwltool`][ref-imp]. You can find a full list on -[the project homepage][commonwl]. -- Most of the software tools used in the user guide are standard UNIX commands -(`cat`, `echo`, `env`, `tar`, `touch`, `wc`), but to run all examples you will -also need: - - node.js - - Java compiler - - Docker (optional) - -[ref-imp]: https://github.com/common-workflow-language/cwltool#install -[commonwl]: https://www.commonwl.org/ diff --git a/src/LICENSE.md b/src/LICENSE.md new file mode 100644 index 00000000..84798e58 --- /dev/null +++ b/src/LICENSE.md @@ -0,0 +1,2 @@ +```{include} ../LICENSE.md +``` diff --git a/_includes/cwl/02-1st-example/1st-tool.cwl b/src/_includes/cwl/02-1st-example/1st-tool.cwl similarity index 100% rename from _includes/cwl/02-1st-example/1st-tool.cwl rename to src/_includes/cwl/02-1st-example/1st-tool.cwl diff --git a/_includes/cwl/02-1st-example/echo-job.yml b/src/_includes/cwl/02-1st-example/echo-job.yml similarity index 100% rename from _includes/cwl/02-1st-example/echo-job.yml rename to src/_includes/cwl/02-1st-example/echo-job.yml diff --git a/_includes/cwl/03-input/inp-job.yml b/src/_includes/cwl/03-input/inp-job.yml similarity index 100% rename from _includes/cwl/03-input/inp-job.yml rename to src/_includes/cwl/03-input/inp-job.yml diff --git a/_includes/cwl/03-input/inp.cwl b/src/_includes/cwl/03-input/inp.cwl similarity index 100% rename from _includes/cwl/03-input/inp.cwl rename to src/_includes/cwl/03-input/inp.cwl diff --git a/_includes/cwl/03-input/whale.txt b/src/_includes/cwl/03-input/whale.txt similarity index 100% rename from _includes/cwl/03-input/whale.txt rename to src/_includes/cwl/03-input/whale.txt diff --git a/_includes/cwl/04-output/hello.tar b/src/_includes/cwl/04-output/hello.tar similarity index 100% rename from _includes/cwl/04-output/hello.tar rename to src/_includes/cwl/04-output/hello.tar diff --git a/_includes/cwl/04-output/tar-job.yml b/src/_includes/cwl/04-output/tar-job.yml similarity index 100% rename from _includes/cwl/04-output/tar-job.yml rename to src/_includes/cwl/04-output/tar-job.yml diff --git a/_includes/cwl/04-output/tar.cwl b/src/_includes/cwl/04-output/tar.cwl similarity index 100% rename from _includes/cwl/04-output/tar.cwl rename to src/_includes/cwl/04-output/tar.cwl diff --git a/_includes/cwl/05-stdout/echo-job.yml b/src/_includes/cwl/05-stdout/echo-job.yml similarity index 100% rename from _includes/cwl/05-stdout/echo-job.yml rename to src/_includes/cwl/05-stdout/echo-job.yml diff --git a/_includes/cwl/05-stdout/stdout.cwl b/src/_includes/cwl/05-stdout/stdout.cwl similarity index 100% rename from _includes/cwl/05-stdout/stdout.cwl rename to src/_includes/cwl/05-stdout/stdout.cwl diff --git a/_includes/cwl/06-params/hello.tar b/src/_includes/cwl/06-params/hello.tar similarity index 100% rename from _includes/cwl/06-params/hello.tar rename to src/_includes/cwl/06-params/hello.tar diff --git a/_includes/cwl/06-params/tar-param-job.yml b/src/_includes/cwl/06-params/tar-param-job.yml similarity index 100% rename from _includes/cwl/06-params/tar-param-job.yml rename to src/_includes/cwl/06-params/tar-param-job.yml diff --git a/_includes/cwl/06-params/tar-param.cwl b/src/_includes/cwl/06-params/tar-param.cwl similarity index 100% rename from _includes/cwl/06-params/tar-param.cwl rename to src/_includes/cwl/06-params/tar-param.cwl diff --git a/_includes/cwl/07-containers/docker-job.yml b/src/_includes/cwl/07-containers/docker-job.yml similarity index 100% rename from _includes/cwl/07-containers/docker-job.yml rename to src/_includes/cwl/07-containers/docker-job.yml diff --git a/_includes/cwl/07-containers/docker.cwl b/src/_includes/cwl/07-containers/docker.cwl similarity index 100% rename from _includes/cwl/07-containers/docker.cwl rename to src/_includes/cwl/07-containers/docker.cwl diff --git a/_includes/cwl/07-containers/hello.js b/src/_includes/cwl/07-containers/hello.js similarity index 100% rename from _includes/cwl/07-containers/hello.js rename to src/_includes/cwl/07-containers/hello.js diff --git a/_includes/cwl/08-arguments/Hello.java b/src/_includes/cwl/08-arguments/Hello.java similarity index 100% rename from _includes/cwl/08-arguments/Hello.java rename to src/_includes/cwl/08-arguments/Hello.java diff --git a/_includes/cwl/08-arguments/arguments-job.yml b/src/_includes/cwl/08-arguments/arguments-job.yml similarity index 100% rename from _includes/cwl/08-arguments/arguments-job.yml rename to src/_includes/cwl/08-arguments/arguments-job.yml diff --git a/_includes/cwl/08-arguments/arguments.cwl b/src/_includes/cwl/08-arguments/arguments.cwl similarity index 100% rename from _includes/cwl/08-arguments/arguments.cwl rename to src/_includes/cwl/08-arguments/arguments.cwl diff --git a/_includes/cwl/09-array-inputs/array-inputs-job.yml b/src/_includes/cwl/09-array-inputs/array-inputs-job.yml similarity index 100% rename from _includes/cwl/09-array-inputs/array-inputs-job.yml rename to src/_includes/cwl/09-array-inputs/array-inputs-job.yml diff --git a/_includes/cwl/09-array-inputs/array-inputs.cwl b/src/_includes/cwl/09-array-inputs/array-inputs.cwl similarity index 100% rename from _includes/cwl/09-array-inputs/array-inputs.cwl rename to src/_includes/cwl/09-array-inputs/array-inputs.cwl diff --git a/_includes/cwl/10-array-outputs/array-outputs-job.yml b/src/_includes/cwl/10-array-outputs/array-outputs-job.yml similarity index 100% rename from _includes/cwl/10-array-outputs/array-outputs-job.yml rename to src/_includes/cwl/10-array-outputs/array-outputs-job.yml diff --git a/_includes/cwl/10-array-outputs/array-outputs.cwl b/src/_includes/cwl/10-array-outputs/array-outputs.cwl similarity index 100% rename from _includes/cwl/10-array-outputs/array-outputs.cwl rename to src/_includes/cwl/10-array-outputs/array-outputs.cwl diff --git a/_includes/cwl/11-records/record-job1.yml b/src/_includes/cwl/11-records/record-job1.yml similarity index 100% rename from _includes/cwl/11-records/record-job1.yml rename to src/_includes/cwl/11-records/record-job1.yml diff --git a/_includes/cwl/11-records/record-job2.yml b/src/_includes/cwl/11-records/record-job2.yml similarity index 100% rename from _includes/cwl/11-records/record-job2.yml rename to src/_includes/cwl/11-records/record-job2.yml diff --git a/_includes/cwl/11-records/record-job3.yml b/src/_includes/cwl/11-records/record-job3.yml similarity index 100% rename from _includes/cwl/11-records/record-job3.yml rename to src/_includes/cwl/11-records/record-job3.yml diff --git a/_includes/cwl/11-records/record.cwl b/src/_includes/cwl/11-records/record.cwl similarity index 100% rename from _includes/cwl/11-records/record.cwl rename to src/_includes/cwl/11-records/record.cwl diff --git a/_includes/cwl/12-env/echo-job.yml b/src/_includes/cwl/12-env/echo-job.yml similarity index 100% rename from _includes/cwl/12-env/echo-job.yml rename to src/_includes/cwl/12-env/echo-job.yml diff --git a/_includes/cwl/12-env/env.cwl b/src/_includes/cwl/12-env/env.cwl similarity index 100% rename from _includes/cwl/12-env/env.cwl rename to src/_includes/cwl/12-env/env.cwl diff --git a/_includes/cwl/13-expressions/empty.yml b/src/_includes/cwl/13-expressions/empty.yml similarity index 100% rename from _includes/cwl/13-expressions/empty.yml rename to src/_includes/cwl/13-expressions/empty.yml diff --git a/_includes/cwl/13-expressions/expression.cwl b/src/_includes/cwl/13-expressions/expression.cwl similarity index 100% rename from _includes/cwl/13-expressions/expression.cwl rename to src/_includes/cwl/13-expressions/expression.cwl diff --git a/_includes/cwl/14-runtime/createfile.cwl b/src/_includes/cwl/14-runtime/createfile.cwl similarity index 100% rename from _includes/cwl/14-runtime/createfile.cwl rename to src/_includes/cwl/14-runtime/createfile.cwl diff --git a/_includes/cwl/14-runtime/echo-job.yml b/src/_includes/cwl/14-runtime/echo-job.yml similarity index 100% rename from _includes/cwl/14-runtime/echo-job.yml rename to src/_includes/cwl/14-runtime/echo-job.yml diff --git a/_includes/cwl/15-staging/Hello.java b/src/_includes/cwl/15-staging/Hello.java similarity index 100% rename from _includes/cwl/15-staging/Hello.java rename to src/_includes/cwl/15-staging/Hello.java diff --git a/_includes/cwl/15-staging/arguments-job.yml b/src/_includes/cwl/15-staging/arguments-job.yml similarity index 100% rename from _includes/cwl/15-staging/arguments-job.yml rename to src/_includes/cwl/15-staging/arguments-job.yml diff --git a/_includes/cwl/15-staging/linkfile.cwl b/src/_includes/cwl/15-staging/linkfile.cwl similarity index 100% rename from _includes/cwl/15-staging/linkfile.cwl rename to src/_includes/cwl/15-staging/linkfile.cwl diff --git a/_includes/cwl/16-file-formats/file-formats.bam b/src/_includes/cwl/16-file-formats/file-formats.bam similarity index 100% rename from _includes/cwl/16-file-formats/file-formats.bam rename to src/_includes/cwl/16-file-formats/file-formats.bam diff --git a/_includes/cwl/16-file-formats/metadata_example.cwl b/src/_includes/cwl/16-file-formats/metadata_example.cwl similarity index 100% rename from _includes/cwl/16-file-formats/metadata_example.cwl rename to src/_includes/cwl/16-file-formats/metadata_example.cwl diff --git a/_includes/cwl/16-file-formats/sample.yml b/src/_includes/cwl/16-file-formats/sample.yml similarity index 100% rename from _includes/cwl/16-file-formats/sample.yml rename to src/_includes/cwl/16-file-formats/sample.yml diff --git a/_includes/cwl/17-metadata/file-formats.bam b/src/_includes/cwl/17-metadata/file-formats.bam similarity index 100% rename from _includes/cwl/17-metadata/file-formats.bam rename to src/_includes/cwl/17-metadata/file-formats.bam diff --git a/_includes/cwl/17-metadata/metadata_example2.cwl b/src/_includes/cwl/17-metadata/metadata_example2.cwl similarity index 100% rename from _includes/cwl/17-metadata/metadata_example2.cwl rename to src/_includes/cwl/17-metadata/metadata_example2.cwl diff --git a/_includes/cwl/17-metadata/metadata_example3.cwl b/src/_includes/cwl/17-metadata/metadata_example3.cwl similarity index 100% rename from _includes/cwl/17-metadata/metadata_example3.cwl rename to src/_includes/cwl/17-metadata/metadata_example3.cwl diff --git a/_includes/cwl/17-metadata/sample.yml b/src/_includes/cwl/17-metadata/sample.yml similarity index 100% rename from _includes/cwl/17-metadata/sample.yml rename to src/_includes/cwl/17-metadata/sample.yml diff --git a/_includes/cwl/19-custom-types/InterProScan-apps.yml b/src/_includes/cwl/19-custom-types/InterProScan-apps.yml similarity index 100% rename from _includes/cwl/19-custom-types/InterProScan-apps.yml rename to src/_includes/cwl/19-custom-types/InterProScan-apps.yml diff --git a/_includes/cwl/19-custom-types/biom-convert-table.yaml b/src/_includes/cwl/19-custom-types/biom-convert-table.yaml similarity index 100% rename from _includes/cwl/19-custom-types/biom-convert-table.yaml rename to src/_includes/cwl/19-custom-types/biom-convert-table.yaml diff --git a/_includes/cwl/19-custom-types/custom-types.cwl b/src/_includes/cwl/19-custom-types/custom-types.cwl similarity index 100% rename from _includes/cwl/19-custom-types/custom-types.cwl rename to src/_includes/cwl/19-custom-types/custom-types.cwl diff --git a/_includes/cwl/19-custom-types/custom-types.yml b/src/_includes/cwl/19-custom-types/custom-types.yml similarity index 100% rename from _includes/cwl/19-custom-types/custom-types.yml rename to src/_includes/cwl/19-custom-types/custom-types.yml diff --git a/_includes/cwl/19-custom-types/rich_sparse_otu_table.biom b/src/_includes/cwl/19-custom-types/rich_sparse_otu_table.biom similarity index 100% rename from _includes/cwl/19-custom-types/rich_sparse_otu_table.biom rename to src/_includes/cwl/19-custom-types/rich_sparse_otu_table.biom diff --git a/_includes/cwl/19-custom-types/test_proteins.fasta b/src/_includes/cwl/19-custom-types/test_proteins.fasta similarity index 100% rename from _includes/cwl/19-custom-types/test_proteins.fasta rename to src/_includes/cwl/19-custom-types/test_proteins.fasta diff --git a/_includes/cwl/20-software-requirements/InterProScan-apps.yml b/src/_includes/cwl/20-software-requirements/InterProScan-apps.yml similarity index 100% rename from _includes/cwl/20-software-requirements/InterProScan-apps.yml rename to src/_includes/cwl/20-software-requirements/InterProScan-apps.yml diff --git a/_includes/cwl/20-software-requirements/custom-types.cwl b/src/_includes/cwl/20-software-requirements/custom-types.cwl similarity index 100% rename from _includes/cwl/20-software-requirements/custom-types.cwl rename to src/_includes/cwl/20-software-requirements/custom-types.cwl diff --git a/_includes/cwl/20-software-requirements/custom-types.yml b/src/_includes/cwl/20-software-requirements/custom-types.yml similarity index 100% rename from _includes/cwl/20-software-requirements/custom-types.yml rename to src/_includes/cwl/20-software-requirements/custom-types.yml diff --git a/_includes/cwl/20-software-requirements/test_proteins.fasta b/src/_includes/cwl/20-software-requirements/test_proteins.fasta similarity index 100% rename from _includes/cwl/20-software-requirements/test_proteins.fasta rename to src/_includes/cwl/20-software-requirements/test_proteins.fasta diff --git a/_includes/cwl/21-1st-workflow/1st-workflow-job.yml b/src/_includes/cwl/21-1st-workflow/1st-workflow-job.yml similarity index 100% rename from _includes/cwl/21-1st-workflow/1st-workflow-job.yml rename to src/_includes/cwl/21-1st-workflow/1st-workflow-job.yml diff --git a/_includes/cwl/21-1st-workflow/1st-workflow.cwl b/src/_includes/cwl/21-1st-workflow/1st-workflow.cwl similarity index 100% rename from _includes/cwl/21-1st-workflow/1st-workflow.cwl rename to src/_includes/cwl/21-1st-workflow/1st-workflow.cwl diff --git a/_includes/cwl/21-1st-workflow/arguments.cwl b/src/_includes/cwl/21-1st-workflow/arguments.cwl similarity index 100% rename from _includes/cwl/21-1st-workflow/arguments.cwl rename to src/_includes/cwl/21-1st-workflow/arguments.cwl diff --git a/_includes/cwl/21-1st-workflow/hello.tar b/src/_includes/cwl/21-1st-workflow/hello.tar similarity index 100% rename from _includes/cwl/21-1st-workflow/hello.tar rename to src/_includes/cwl/21-1st-workflow/hello.tar diff --git a/_includes/cwl/21-1st-workflow/tar-param.cwl b/src/_includes/cwl/21-1st-workflow/tar-param.cwl similarity index 100% rename from _includes/cwl/21-1st-workflow/tar-param.cwl rename to src/_includes/cwl/21-1st-workflow/tar-param.cwl diff --git a/_includes/cwl/22-nested-workflows/1st-workflow.cwl b/src/_includes/cwl/22-nested-workflows/1st-workflow.cwl similarity index 100% rename from _includes/cwl/22-nested-workflows/1st-workflow.cwl rename to src/_includes/cwl/22-nested-workflows/1st-workflow.cwl diff --git a/_includes/cwl/22-nested-workflows/arguments.cwl b/src/_includes/cwl/22-nested-workflows/arguments.cwl similarity index 100% rename from _includes/cwl/22-nested-workflows/arguments.cwl rename to src/_includes/cwl/22-nested-workflows/arguments.cwl diff --git a/_includes/cwl/22-nested-workflows/nestedworkflows.cwl b/src/_includes/cwl/22-nested-workflows/nestedworkflows.cwl similarity index 100% rename from _includes/cwl/22-nested-workflows/nestedworkflows.cwl rename to src/_includes/cwl/22-nested-workflows/nestedworkflows.cwl diff --git a/_includes/cwl/22-nested-workflows/tar-param.cwl b/src/_includes/cwl/22-nested-workflows/tar-param.cwl similarity index 100% rename from _includes/cwl/22-nested-workflows/tar-param.cwl rename to src/_includes/cwl/22-nested-workflows/tar-param.cwl diff --git a/_includes/cwl/23-scatter-workflow/1st-tool-mod.cwl b/src/_includes/cwl/23-scatter-workflow/1st-tool-mod.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/1st-tool-mod.cwl rename to src/_includes/cwl/23-scatter-workflow/1st-tool-mod.cwl diff --git a/_includes/cwl/23-scatter-workflow/1st-tool.cwl b/src/_includes/cwl/23-scatter-workflow/1st-tool.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/1st-tool.cwl rename to src/_includes/cwl/23-scatter-workflow/1st-tool.cwl diff --git a/_includes/cwl/23-scatter-workflow/scatter-job.yml b/src/_includes/cwl/23-scatter-workflow/scatter-job.yml similarity index 100% rename from _includes/cwl/23-scatter-workflow/scatter-job.yml rename to src/_includes/cwl/23-scatter-workflow/scatter-job.yml diff --git a/_includes/cwl/23-scatter-workflow/scatter-nested-workflow.cwl b/src/_includes/cwl/23-scatter-workflow/scatter-nested-workflow.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/scatter-nested-workflow.cwl rename to src/_includes/cwl/23-scatter-workflow/scatter-nested-workflow.cwl diff --git a/_includes/cwl/23-scatter-workflow/scatter-two-steps.cwl b/src/_includes/cwl/23-scatter-workflow/scatter-two-steps.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/scatter-two-steps.cwl rename to src/_includes/cwl/23-scatter-workflow/scatter-two-steps.cwl diff --git a/_includes/cwl/23-scatter-workflow/scatter-workflow.cwl b/src/_includes/cwl/23-scatter-workflow/scatter-workflow.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/scatter-workflow.cwl rename to src/_includes/cwl/23-scatter-workflow/scatter-workflow.cwl diff --git a/_includes/cwl/23-scatter-workflow/wc-tool.cwl b/src/_includes/cwl/23-scatter-workflow/wc-tool.cwl similarity index 100% rename from _includes/cwl/23-scatter-workflow/wc-tool.cwl rename to src/_includes/cwl/23-scatter-workflow/wc-tool.cwl diff --git a/_includes/cwl/conformance-test.yml b/src/_includes/cwl/conformance-test.yml similarity index 99% rename from _includes/cwl/conformance-test.yml rename to src/_includes/cwl/conformance-test.yml index 47340e0c..acdab96d 100644 --- a/_includes/cwl/conformance-test.yml +++ b/src/_includes/cwl/conformance-test.yml @@ -1,7 +1,7 @@ # Section 2 - doc: Test for section 2 job: 02-1st-example/echo-job.yml - tool: 02-1st-example/1st-tool.cwl + tool: hello_world.cwl should_fail: false output: {} diff --git a/src/_includes/cwl/hello_world.cwl b/src/_includes/cwl/hello_world.cwl new file mode 100644 index 00000000..3d35dbb8 --- /dev/null +++ b/src/_includes/cwl/hello_world.cwl @@ -0,0 +1,17 @@ +cwlVersion: v1.2 + +# What type of CWL process we have in this document. +class: CommandLineTool +# This CommandLineTool executes the linux "echo" command-line tool. +baseCommand: echo + +# The inputs for this process. +inputs: + message: + type: string + # A default value that can be overridden, e.g. --message "Hola mundo" + default: "Hello World" + # Bind this message value as an argument to "echo". + inputBinding: + position: 1 +outputs: [] diff --git a/src/_includes/cwl/simplest_cwl.cwl b/src/_includes/cwl/simplest_cwl.cwl new file mode 100644 index 00000000..f09c94ee --- /dev/null +++ b/src/_includes/cwl/simplest_cwl.cwl @@ -0,0 +1,6 @@ +cwlVersion: v1.2 +class: CommandLineTool +inputs: [] +outputs: [] +# `true` is a Linux command that exits with exit code `0` (success). +baseCommand: "true" diff --git a/src/_includes/cwl/simplest_cwl_shebang.cwl b/src/_includes/cwl/simplest_cwl_shebang.cwl new file mode 100644 index 00000000..09b8164c --- /dev/null +++ b/src/_includes/cwl/simplest_cwl_shebang.cwl @@ -0,0 +1,8 @@ +#!/usr/bin/env cwl-runner + +cwlVersion: v1.2 +class: CommandLineTool +inputs: [] +outputs: [] +# `true` is a Linux command that exits with exit code `0` (success). +baseCommand: "true" diff --git a/src/_includes/expression-tool.md b/src/_includes/expression-tool.md new file mode 100644 index 00000000..68094ac8 --- /dev/null +++ b/src/_includes/expression-tool.md @@ -0,0 +1,68 @@ +An expression tool is a type of Process that can be run by itself or +as a Workflow step. It executes a pure JavaScript expression. It is +meant to be used as a way to isolate complex JavaScript expressions +that need to operate on input data and produce some result as output. + +Similar to the command-line tool it requires `inputs` and `outputs`. +But instead of `baseCommand`, it requires an `expression` attribute. + +% TODO: Fix the missing link the graph below. We cannot have +% it here as this file is included in two other files. +% Sphinx prohibits it for the case where this could lead +% to duplicate anchors in a page (e.g. single-html). +% :name: expression-tool-graph + +```{graphviz} +:caption: CWL expression tool. +:align: center + +digraph G { + compound=true; + rankdir="LR"; + fontname="Verdana"; + fontsize="10"; + graph [splines=ortho]; + + node [fontname="Verdana", fontsize="10", shape=box]; + edge [fontname="Verdana", fontsize="10"]; + + subgraph cluster_0 { + expression[style="filled" label="JavaScript"]; + label="expression"; + fill=gray; + } + + inputs -> expression [lhead=cluster_0]; + expression -> outputs [ltail=cluster_0]; +} +``` + +% TODO: Fix the missing link the code below. We cannot have +% it here as this file is included in two other files. +% Sphinx prohibits it for the case where this could lead +% to duplicate anchors in a page (e.g. single-html). +% :name: uppercase.cwl + +```{code-block} cwl +:caption: "`uppercase.cwl`" +cwlVersion: v1.2 +class: ExpressionTool + +requirements: + InlineJavascriptRequirement: {} + +inputs: + message: string +outputs: + uppercase_message: string + +expression: | + ${ return {"uppercase_message": inputs.message.toUpperCase()}; } +``` + +```{note} + +We had to use an `InlineJavascriptRequirement` as our expression +contains a JavaScript call in `.toUpperCase()`. This means to tools +using the expression tool that JavaScript is a requirement. +``` diff --git a/_includes/favicons.html b/src/_includes/favicons.html similarity index 100% rename from _includes/favicons.html rename to src/_includes/favicons.html diff --git a/src/_includes/what-is-cwl.md b/src/_includes/what-is-cwl.md new file mode 100644 index 00000000..f9357b8f --- /dev/null +++ b/src/_includes/what-is-cwl.md @@ -0,0 +1,3 @@ +CWL is a way to describe command-line tools and connect them together to create workflows. +Because CWL is a specification and not a specific piece of software, tools and workflows +described using CWL are portable across a variety of platforms that support the CWL standard. diff --git a/_static/images/favicons/cwl/android-icon-144x144.png b/src/_static/images/favicons/cwl/android-icon-144x144.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-144x144.png rename to src/_static/images/favicons/cwl/android-icon-144x144.png diff --git a/_static/images/favicons/cwl/android-icon-192x192.png b/src/_static/images/favicons/cwl/android-icon-192x192.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-192x192.png rename to src/_static/images/favicons/cwl/android-icon-192x192.png diff --git a/_static/images/favicons/cwl/android-icon-36x36.png b/src/_static/images/favicons/cwl/android-icon-36x36.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-36x36.png rename to src/_static/images/favicons/cwl/android-icon-36x36.png diff --git a/_static/images/favicons/cwl/android-icon-48x48.png b/src/_static/images/favicons/cwl/android-icon-48x48.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-48x48.png rename to src/_static/images/favicons/cwl/android-icon-48x48.png diff --git a/_static/images/favicons/cwl/android-icon-72x72.png b/src/_static/images/favicons/cwl/android-icon-72x72.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-72x72.png rename to src/_static/images/favicons/cwl/android-icon-72x72.png diff --git a/_static/images/favicons/cwl/android-icon-96x96.png b/src/_static/images/favicons/cwl/android-icon-96x96.png similarity index 100% rename from _static/images/favicons/cwl/android-icon-96x96.png rename to src/_static/images/favicons/cwl/android-icon-96x96.png diff --git a/_static/images/favicons/cwl/apple-icon-180x180.png b/src/_static/images/favicons/cwl/apple-icon-180x180.png similarity index 100% rename from _static/images/favicons/cwl/apple-icon-180x180.png rename to src/_static/images/favicons/cwl/apple-icon-180x180.png diff --git a/_static/images/favicons/cwl/apple-icon-precomposed.png b/src/_static/images/favicons/cwl/apple-icon-precomposed.png similarity index 100% rename from _static/images/favicons/cwl/apple-icon-precomposed.png rename to src/_static/images/favicons/cwl/apple-icon-precomposed.png diff --git a/_static/images/favicons/cwl/apple-icon.png b/src/_static/images/favicons/cwl/apple-icon.png similarity index 100% rename from _static/images/favicons/cwl/apple-icon.png rename to src/_static/images/favicons/cwl/apple-icon.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-114x114.png b/src/_static/images/favicons/cwl/apple-touch-icon-114x114.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-114x114.png rename to src/_static/images/favicons/cwl/apple-touch-icon-114x114.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-120x120.png b/src/_static/images/favicons/cwl/apple-touch-icon-120x120.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-120x120.png rename to src/_static/images/favicons/cwl/apple-touch-icon-120x120.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-144x144.png b/src/_static/images/favicons/cwl/apple-touch-icon-144x144.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-144x144.png rename to src/_static/images/favicons/cwl/apple-touch-icon-144x144.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-152x152.png b/src/_static/images/favicons/cwl/apple-touch-icon-152x152.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-152x152.png rename to src/_static/images/favicons/cwl/apple-touch-icon-152x152.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-57x57.png b/src/_static/images/favicons/cwl/apple-touch-icon-57x57.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-57x57.png rename to src/_static/images/favicons/cwl/apple-touch-icon-57x57.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-60x60.png b/src/_static/images/favicons/cwl/apple-touch-icon-60x60.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-60x60.png rename to src/_static/images/favicons/cwl/apple-touch-icon-60x60.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-72x72.png b/src/_static/images/favicons/cwl/apple-touch-icon-72x72.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-72x72.png rename to src/_static/images/favicons/cwl/apple-touch-icon-72x72.png diff --git a/_static/images/favicons/cwl/apple-touch-icon-76x76.png b/src/_static/images/favicons/cwl/apple-touch-icon-76x76.png similarity index 100% rename from _static/images/favicons/cwl/apple-touch-icon-76x76.png rename to src/_static/images/favicons/cwl/apple-touch-icon-76x76.png diff --git a/_static/images/favicons/cwl/favicon-128.png b/src/_static/images/favicons/cwl/favicon-128.png similarity index 100% rename from _static/images/favicons/cwl/favicon-128.png rename to src/_static/images/favicons/cwl/favicon-128.png diff --git a/_static/images/favicons/cwl/favicon-16x16.png b/src/_static/images/favicons/cwl/favicon-16x16.png similarity index 100% rename from _static/images/favicons/cwl/favicon-16x16.png rename to src/_static/images/favicons/cwl/favicon-16x16.png diff --git a/_static/images/favicons/cwl/favicon-196x196.png b/src/_static/images/favicons/cwl/favicon-196x196.png similarity index 100% rename from _static/images/favicons/cwl/favicon-196x196.png rename to src/_static/images/favicons/cwl/favicon-196x196.png diff --git a/_static/images/favicons/cwl/favicon-32x32.png b/src/_static/images/favicons/cwl/favicon-32x32.png similarity index 100% rename from _static/images/favicons/cwl/favicon-32x32.png rename to src/_static/images/favicons/cwl/favicon-32x32.png diff --git a/_static/images/favicons/cwl/favicon-96x96.png b/src/_static/images/favicons/cwl/favicon-96x96.png similarity index 100% rename from _static/images/favicons/cwl/favicon-96x96.png rename to src/_static/images/favicons/cwl/favicon-96x96.png diff --git a/_static/images/favicons/cwl/favicon.ico b/src/_static/images/favicons/cwl/favicon.ico similarity index 100% rename from _static/images/favicons/cwl/favicon.ico rename to src/_static/images/favicons/cwl/favicon.ico diff --git a/_static/images/favicons/cwl/ms-icon-144x144.png b/src/_static/images/favicons/cwl/ms-icon-144x144.png similarity index 100% rename from _static/images/favicons/cwl/ms-icon-144x144.png rename to src/_static/images/favicons/cwl/ms-icon-144x144.png diff --git a/_static/images/favicons/cwl/ms-icon-150x150.png b/src/_static/images/favicons/cwl/ms-icon-150x150.png similarity index 100% rename from _static/images/favicons/cwl/ms-icon-150x150.png rename to src/_static/images/favicons/cwl/ms-icon-150x150.png diff --git a/_static/images/favicons/cwl/ms-icon-310x310.png b/src/_static/images/favicons/cwl/ms-icon-310x310.png similarity index 100% rename from _static/images/favicons/cwl/ms-icon-310x310.png rename to src/_static/images/favicons/cwl/ms-icon-310x310.png diff --git a/_static/images/favicons/cwl/ms-icon-70x70.png b/src/_static/images/favicons/cwl/ms-icon-70x70.png similarity index 100% rename from _static/images/favicons/cwl/ms-icon-70x70.png rename to src/_static/images/favicons/cwl/ms-icon-70x70.png diff --git a/_static/images/logos/cwl/CWL-Logo-HD-cropped2.png b/src/_static/images/logos/cwl/CWL-Logo-HD-cropped2.png similarity index 100% rename from _static/images/logos/cwl/CWL-Logo-HD-cropped2.png rename to src/_static/images/logos/cwl/CWL-Logo-HD-cropped2.png diff --git a/_static/images/logos/cwl/cwl-icon.png b/src/_static/images/logos/cwl/cwl-icon.png similarity index 100% rename from _static/images/logos/cwl/cwl-icon.png rename to src/_static/images/logos/cwl/cwl-icon.png diff --git a/_static/switcher.json b/src/_static/switcher.json similarity index 100% rename from _static/switcher.json rename to src/_static/switcher.json diff --git a/_templates/layout.html b/src/_templates/layout.html similarity index 79% rename from _templates/layout.html rename to src/_templates/layout.html index 3ba500b7..70133ecd 100644 --- a/_templates/layout.html +++ b/src/_templates/layout.html @@ -5,6 +5,9 @@