Invalid `do:switch`

[JBBianchi]

What seems off:
After #875, and because of, #884 (comment), #882 was merged, making do a single task entry. This breaks the possibility to use do:switch as there is nothing to switch to. It forces the user to use do:execute:sequentially:switch.

What you expected to be:
We might need to rollback to the original idea of #875, which is to have do as map[string, task], but not only for top level.

Anything else we need to know?:
As mentioned by @matthias-pichler-warrify in #875, making do a map[string, task], the execute.sequentially directive doesn't really make sense anymore. We might want to refactor execute at the same time (or keep that for another issue).

[fjtirado]

Can we do this for sequentiality

do:
   -task1:
   -task2: 

and this for concurrency

do:
   -task1:
   -task2: 
onParallel:
   compete: false

If we agree on the syntax, I think that will be doable schema wise (by putting do: array as required and onParallel: object as optional of a new schema construct)

[JBBianchi]

Discussion from #875 related to this issue:

@JBBianchi #875 (comment)

I think we overlooked the case of switch with the adopted solution. It's relegated to a second zone citizen, only available in execute:

do:
  execute:
    sequentially:
      - processOrder:
          switch:
          # ...

https://github.com/serverlessworkflow/specification/blob/main/dsl-reference.md#switch

Shouldn't we go back to do being a map[string, task][] instead of a single task ?

@cdavernas #875 (comment)

@JBBianchi Yes, indeed. Do as a mapping array is what we should opt for:

* It resolves consistency of the keyword `do`: it's alway a mapping array of tasks

* It allows for top-level switch statements

* It's more consistent with what a workflow is, to most people: a successions of multiple tasks, not a single top level one, which is then... a task

@matthias-pichler-warrify #875 (comment)

That is true. And do as an array is absolutely fine by me, however as @cdavernas pointed out: what's the purpose of sequentially then?

Which given the discussion around concurrently and the execute keyword in general, might mean:

* drop sequentially (not needed as we have arrays everywhere

* only keep `concurrently` (or replace with `branch` to make it a verb for consistency)

@cdavernas #875 (comment)

multitask
/ˈmʌltɪtɑːsk/
verb

1. (of a person) deal with more than one task at the same time.
   "I managed my time efficiently and multitasked"

It is a verb, though certainly not the best term. But there is only a little amount of alternatives, such as the (horrible) parallelize

The idea for branch also came from AWS StepFunctions (they run in parallel there)

Interesting. Might be worth digging.

@JBBianchi #875 (comment)

Do you mean:

do:
  execute:
    branch:
      - task1:
      - task2:
      - task3:    

or

do:
  branch:
    - task1:
    - task2:
    - task3: 

(an alternative to branch could also be fork)

In the first case, we keep the "controversial" execute but have an extra level to put config like compete: true. In the second one, we get rid of the controversial keyword but we might need an extra level of depth.

@fjtirado #875 (comment)

@JBBianchi Im trying to undestand the issue with swith. I understood that rather than writing

do:
  execute:
    sequentially:
      - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
          execute:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
          execute:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
          execute:
            sequentially:
              - logWarning: {}
              - notifyAdmin: {}

we should be able to write

do:
     - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
          execute:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
          execute:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
          execute:
            sequentially:
              - logWarning: {}
              - notifyAdmin: {}

isnt it? But it we remove execute, as option 2 in this discussion, it would be

do:
  sequentially:
     - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
           sequentially:
              - logWarning: {}
              - notifyAdmin: {}

Am I missing anything? (probably I am, my apologies in advance)

@JBBianchi #875 (comment)

@JBBianchi Im trying to undestand the issue with swith. I understood that rather than writing

do:
  execute:
    sequentially:
      - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
          execute:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
          execute:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
          execute:
            sequentially:
              - logWarning: {}
              - notifyAdmin: {}

we should be able to write

do:
     - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
          execute:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
          execute:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
          execute:
            sequentially:
              - logWarning: {}
              - notifyAdmin: {}

isnt it?

That's my concern indeed.

But it we remove execute, as option 2 in this discussion, it would be

do:
  sequentially:
     - processOrder:
          switch:
            - case1:
                when: .orderType == "electronic"
                then: processElectronicOrder
            - case2:
                when: .orderType == "physical"
                then: processPhysicalOrder
            - default:
                then: handleUnknownOrderType
      - processElectronicOrder:
          execute:
            sequentially:
              - validatePayment: {}
              - fulfillOrder: {}
          then: exit
      - processPhysicalOrder:
          execute:
            sequentially:
              - checkInventory: {}
              - packItems: {}
              - scheduleShipping: {}
          then: exit
      - handleUnknownOrderType:
          execute:
            sequentially:
              - logWarning: {}
              - notifyAdmin: {}

Am I missing anything? (probably I am, my apologies in advance)

The later example doesn't seem to match any late suggestions made in this issue. I didn't mention removing execute per se as it's another topic. But if I understand what @matthias-pichler-warrify and @cdavernas were talking about just after I raised my concern, is that sequentially is the default behavior of a do, therefore execute.sequentially doesn't really have a reason to exists. Which leaves us with concurrently, which could be replaced by branch(, multitask, parallelize or fork). What's unclear is if we keep execute.branch (or alternative) or we just get rid of execute altogether an opt for branch (or alternative) straight on - which then, rejoin the point 2 of #889 and makes the discussion obsolete. In the later case though, we'd probably need an extra level of depth for some params like racing (or threads count for instance).

e.g.:

do:
  - processOrder:
      switch:
        - case1:
            when: .orderType == "electronic"
            then: processElectronicOrder
        - case2:
            when: .orderType == "physical"
            then: processPhysicalOrder
        - default:
            then: handleUnknownOrderType
  - processElectronicOrder:
      do:
        - validatePayment: {}
        - fulfillOrder: {}
        then: exit
  - processPhysicalOrder:
      execute:
        compete: true
        branch:  # `multitask`, `parallelize` or `fork`
          - checkInventory: {}
          - packItems: {}
          - scheduleShipping: {}
            then: exit
  - handleUnknownOrderType: #...

or

do:
  - processOrder:
      switch:
        - case1:
            when: .orderType == "electronic"
            then: processElectronicOrder
        - case2:
            when: .orderType == "physical"
            then: processPhysicalOrder
        - default:
            then: handleUnknownOrderType
  - processElectronicOrder:
      do:
        - validatePayment: {}
        - fulfillOrder: {}
        then: exit
  - processPhysicalOrder:
      branch:
        compete: true
        on:  # `multitask`, `parallelize` or `fork`
          - checkInventory: {}
          - packItems: {}
          - scheduleShipping: {}
            then: exit
  - handleUnknownOrderType: #...

@cdavernas @matthias-pichler-warrify am I getting it right ?

@fjtirado #875 (comment)

@JBBianchi There was a typo in my last example, I edited it. My point was that this is kind of related with the solution for do: execute:, if we simplify how multitask is written, its not really an issue that switch is a task which only makes sense within a multitask (which is why we are proposing to make do: multitask again) Also, doing do: always a multitask (so if you want a single task you just write one named task, as far as this also applies to loops, retries and catch) and removing sequentially (because its implicit in do:), leaving concurrently as an special case of multitask will also work.

@cdavernas #875 (comment)

its not really an issue that switch is a task which only makes sense within a multitask

I strongly disagree: it is a problem, and a big one. The fact that do could have different meaning is blocking for you in terms of consistency, but not the fact that the switch task - and the switch task only - cannot be used at top level? Furthermore, as said above, a single task per workflow goes against the definition of a workflow. It also is a problem when it comes to visualization.

(which is why we are proposing to make do: multitask again)

That is not what we are proposing. We are proposing:

1. To make `do` a (sequential) array everywhere

2. To get rid of `execute/sequentially`

3. Create a new task to "parallelize" tasks

@fjtirado #875 (comment)

@cdavernas I think we are are saying the same thing, so there is not need to strongly disagree ;) When I said that we are proposing to do a multitask I mean do: being an array (which I said is fine as far as do: have the same syntax everywhere, not only in main flow) And removing execute/sequentially is part of the do:execute: redundancy discussion.

@cdavernas #875 (comment)

I think we are are saying the same thing, so there is not need to strongly disagree ;)

Hehe, sorry, seems I misunderstood you indeed 🤣

And removing execute/sequentially is part of the do:execute: redundancy discussion.

Indeed.

[matthias-pichler]

@JBBianchi with this example:

do:
  - processOrder:
      switch:
        - case1:
            when: .orderType == "electronic"
            then: processElectronicOrder
        - case2:
            when: .orderType == "physical"
            then: processPhysicalOrder
        - default:
            then: handleUnknownOrderType
  - processElectronicOrder:
      do:
        - validatePayment: {}
        - fulfillOrder: {}
        then: exit
  - processPhysicalOrder:
      branch:
        compete: true
        on:  # `multitask`, `parallelize` or `fork`
          - checkInventory: {}
          - packItems: {}
          - scheduleShipping: {}
            then: exit
  - handleUnknownOrderType: #...

I think you are mixing some proposals together because now here:

 - processElectronicOrder:
      do:
        - validatePayment: {}
        - fulfillOrder: {}
        then: exit

do appears as a task which was only discusses in the sense of renaming execute to do.

But actually it makes sense here as a substitute for the lost execute:sequentially

[matthias-pichler]

But it seems that on this one we are already very close to a consensus:

1. To make do a (sequential) array everywhere

that includes the main flow, for and catch. Is there still another way to then define sequential actions? Because it leaves us with no alternative for try and before & after in extensions. So either need to make these arrays as well or have some execute/sequentially alternative ... @JBBianchi proposed do itself:

do:
 - processOrder:
     switch:
       - case1:
           when: .orderType == "electronic"
           then: processElectronicOrder
       - case2:
           when: .orderType == "physical"
           then: processPhysicalOrder
       - default:
           then: handleUnknownOrderType
 - processElectronicOrder:
     do: # do instead of execute:sequentially
       - validatePayment: {}
       - fulfillOrder: {}
     then: exit
 - processPhysicalOrder: #...
 - handleUnknownOrderType: #...

2. To get rid of execute/sequentially

that makes absolute sense to me because in my head "doing a set of tasks in order" should be the default mode for a workflow anyway. We just need to decide on how to substitute all its different use-cases, namely in try, before, after and nested tasks.

3. Create a new task to "parallelize" tasks

that also makes sense and we already have a couple of good ideas how to structure and name it (fork, branch, multitask, parallelize) 👍

[cdavernas]

that includes the main flow, for and catch. Is there still another way to then define sequential actions? Because it leaves us with no alternative for try and before & after in extensions. So either need to make these arrays as well or have some execute/sequentially alternative

I think it should also apply to try, before and after. IMHO there's no reason a user should be not be able to use top level switch in those cases.

[matthias-pichler]

that includes the main flow, for and catch. Is there still another way to then define sequential actions? Because it leaves us with no alternative for try and before & after in extensions. So either need to make these arrays as well or have some execute/sequentially alternative

I think it should also apply to try, before and after. IMHO there's no reason a user should be not be able to use top level switch in those cases.

agreed. But what should we do about nested sequential tasks? like @JBBianchi tried to show here:

 - processElectronicOrder:
      do:
        - validatePayment: {}
        - fulfillOrder: {}
      then: exit

[cdavernas]

agreed. But what should we do about nested sequential tasks?

Exactly what @JBBianchi proposed I guess: basically use do to define a sequential composite task.

So, to summarize for sake of brievety, here's my understanding of what's proposed:

1. Make do a (sequential) task array everywhere
2. Make try, before and after a (sequential) task array
3. Create a new do task, used to define a nested (sequential) task array
4. Create a new fork/branch/multitask/parallelize task, used to define a parallel (and possibly competing) task array

[fjtirado]

@cdavernas A variant for 4) will be to use do: both for sequential and concurrency, as proposed here #894 (comment).

[cdavernas]

@cdavernas A variant for 4) will be to use do: both for sequential and concurrency, as proposed here #894 (comment).

Nice idea! I'm however not a fan of the suggestion, for multiple reasons:

• It is made out of two words
• It is not a verb, and does not respect imperative tense
• As @matthias-pichler-warrify mentionned on another issue, I would avoid having qualified properties at top level (ex: document/use/evaluate/do/onParallel <= the later feels off, and inconsistent)

[matthias-pichler]

@cdavernas A variant for 4) will be to use do: both for sequential and concurrency, as proposed here #894 (comment).

Nice idea! I'm however not a fan of the suggestion, for multiple reasons:

• It is made out of two words
• It is not a verb, and does not respect imperative tense
• As @matthias-pichler-warrify mentionned on another issue, I would avoid having qualified properties at top level (ex: document/use/evaluate/do/onParallel <= the later feels off, and inconsistent)

I am also more a fan of a dedicated task type for parallel execution since

• it's what I (and therefore potentially others) are used to from other workflow languages
  • Branch in AWS StepFunctions
  • FORK_JOIN in netflix conductor
  • parallel in GCP workflows
  • ...
• concurrency (as any programmer knows) can be tough and making it more explicit can't hurt
• it offers a nice place to add options (such as compete or threads etc) which do not make sense for sequential execution

[fjtirado]

I came from an OO world, so let me try to justfy why I feel adding a property to same object where do: is defined is a good alternative to some new keyword that reflect the concept of do concurrently.
I think we agree both do: and concurrently: (Im using the keyword for conveninece, we can use a different name) are array of task.
do: is an array of task that is executed sequentially
concurrently: is an array of task that is executed concurrently (concurrency might be configured)
Therefore it is not crazy to consider concurrently: an extension of do: and rather than redering it as a different keyword, render both as the same object with different internal state.
In Json schema, that means defining a new object that contains do: as array of task and optional properties that setups how this array of task is run. Sequenatially if there is not any extra property. And concurrently if we write the addional property.
That approach allow us, eventually, to add new properties that qualifies do: (the array of tasks) that are both shared by sequential and concurrent execution (we feel there are none, but we might be wrong)
The optional property onParallel: (we already have properties that are not vebs, like with, as...) is the place to add properties that are specific for concurrent execution
If we find out another way to execute the task (for example onReverse:) we just add the new optional property.

[fjtirado]

Anyway, I guess we are taking this approach

concurrently (or any other word):
   compete:
   #any additional property like compete
   branch (or some other word to write the array): 
      - task 1: 
     - task 2: 

That also works for me (my previous message was just for the record ;))