Adding information about Workflow, updating CWL workflows

Author: kinow

src/basic-concepts.md

@@ -35,21 +35,18 @@ in parallel across many nodes.
 
 % TODO: add a link to the Core Concepts -> Requirements section below?
 
-```{note}
+## Requirements
+
 The CWL specification allows for implementations to provide extra
 functionality through Requirements. Some CWL runners may provide
 Requirements that are not in the specification. For example, GPU
 requirements are supported in `cwltool`, but it is not part of the
 {{ cwl_version }} specification. Requirements are explained in
 detail in another section.
-```
 
-## Processing units
+% TODO: better explanation about requirements, with an example.
 
-The CWL document is a JSON or YAML file — or a combination of both
-JSON and YAML. One of the required attributes of the CWL document
-is the `class`. The `class` defines the type of processing unit used
-in the CWL document.
+## Processing units
 
 There are four types of processing units defined in the CWL specification
 {{ cwl_version }}:
@@ -62,12 +59,11 @@ There are four types of processing units defined in the CWL specification
 {{ CWL_PROCESSING_UNITS_GRAPH }}
 
 In `cwltool` you can execute a CWL document with a command-line tool,
-an expression tool, or a workflow.
+an expression tool, or a workflow. Operation is a special unit, not
+covered in this section.
 
-```{note}
-
-`class: Operation` is a special processing unit, not covered in this section.
-```
+You define the processing unit in your CWL document using the
+`class` attribute, e.g. `class: Workflow`.
 
 ### Command-line tool
 
@@ -81,20 +77,6 @@ The following example contains a minimal example of a CWL
 command-line tool for the `echo` Linux command, using inputs and
 outputs.
 
-```{code-block} cwl
-:name: echo.cwl
-:caption: "`echo.cwl`"
-cwlVersion: v1.2
-class: CommandLineTool
-
-baseCommand: echo
-
-inputs:
-  message: string
-outputs:
-  out: stdout
-```
-
 ```{graphviz}
 :name: command-line-tool-graph
 :caption: CWL command-line tool.
@@ -110,6 +92,29 @@ digraph "CWL command-line tool" {
 }
 ```
 
+```{code-block} cwl
+:name: echo.cwl
+:caption: "`echo.cwl`"
+cwlVersion: v1.2
+class: CommandLineTool
+
+baseCommand: echo
+
+stdout: output.txt
+
+inputs:
+  message:
+    type: string
+    inputBinding: {}
+outputs:
+  out:
+    type: string
+    outputBinding:
+      glob: output.txt
+      loadContents: true
+      outputEval: $(self[0].contents)
+```
+
 ```{note}
 
 The example above uses a simplified form to define inputs and outputs.
@@ -124,21 +129,54 @@ as a Workflow step. It executes a pure JavaScript expression. It is
 meant to be used as a way to isolate complex JavaScript expressions
 that need to operate on input data and produce some result as output.
 
-### Workflow
+Similar to the command-line tool it requires `inputs` and `outputs`.
+But instead of `baseCommand`, it requires an `expression` attribute.
+
+```{graphviz}
+:name: expression-tool-graph
+:caption: CWL expression tool.
+:align: center
 
-Besides having `class: Workflow`, the CWL document of a workflow processing
-unit must also have the `cwlVersion`, `inputs`, `outputs`, and `steps`.
+digraph "CWL command-line tool" {
+    rankdir="LR";
+    graph [splines=ortho];
 
-```cwl
-cwlVersion: {{ cwl_version_text }}
-class: Workflow
+    node [fontname="Verdana", fontsize="10", shape=box];
+    edge [fontname="Verdana", fontsize="10"];
+    inputs -> expression -> outputs;
+}
+```
+
+```{code-block} cwl
+:name: uppercase.cwl
+:caption: "`uppercase.cwl`"
+cwlVersion: v1.2
+class: ExpressionTool
+
+requirements:
+  InlineJavascriptRequirement: {}
+
+inputs:
+  message: string
+outputs:
+  uppercase_message: string
 
-inputs: []
-steps: []
-outputs: []
+expression: |
+  ${ return {"uppercase_message": inputs.message.toUpperCase()}; }
 ```
 
-A workflow step can call a command-line tool or expression tool.
+```{note}
+
+We had to use an `InlineJavascriptRequirement` as our expression
+contains a JavaScript call in `.toUpperCase()`. This means to tools
+using the expression tool that JavaScript is a requirement.
+```
+
+### Workflow
+
+A workflow is a CWL processing unit that executes command-line tools,
+expression tools, or workflows (sub-workflows) as steps. It must have
+`inputs`, `outputs`, and `steps` defined in the CWL document.
 
 ```{graphviz}
 :name: workflow-graph
@@ -155,6 +193,35 @@ digraph "CWL Inputs, Steps, and Outputs" {
 }
 ```
 
+```cwl
+cwlVersion: v1.2
+class: Workflow
+
+requirements:
+  InlineJavascriptRequirement: {}
+
+inputs:
+  message: string
+
+outputs:
+  out:
+    type: string
+    outputSource: uppercase/uppercase_message
+
+steps:
+  echo:
+    run: echo.cwl
+    in:
+      message: message
+    out: [out]
+  uppercase:
+    run: uppercase.cwl
+    in:
+      message:
+        source: echo/out
+    out: [uppercase_message]
+```
+
 ## FAIR workflows
 
 CWL has roots in "make" and many similar tools that determine order of