diff --git a/docs/diagrams/README.md b/docs/diagrams/README.md new file mode 100644 index 00000000..6c0ddd2a --- /dev/null +++ b/docs/diagrams/README.md @@ -0,0 +1,30 @@ +# Diagrams + +This folder contains diagrams about functionalities of CWL Viewer. +The source file for the diagrams is stored in a SVG file. We suggest +contributors to use Inkscape to edit the SVG file. + +## Flowcharts + +When a user submits a workflow to the CWL Viewer, the Spring controller +will handle the request delegating most of the work to Spring services. + +![flow part 1](cwlviewer_flowchat_01.png) + +The first part of the flow is responsible for validating the request and +checking if the workflow exists in the Postgres database, and if not +creating a `QueuedWorkflow` in the database. + +![flow part 2](cwlviewer_flowchat_02.png) + +An asynchronous task is then scheduled to parse the CWL workflow with +`cwltool`, use Apache Jena to create the workflow graph, and to store the +`Workflow` object (not queued anymore) in the Postgres database. + +![flow part 3](cwlviewer_flowchat_03.png) + +This final part of the flow is also an asynchronous task, responsible for +creating the Research Objects bundle file. It uses Apache Taverna API +to create a `Bundle`, and stores parts of the Git repository, graph +images generated by the `cwltool` and Jena graph data, and extra data +for provenance. The result is updated in the `Workflow` object in Postgres. diff --git a/docs/diagrams/cwlviewer.svg b/docs/diagrams/cwlviewer.svg new file mode 100755 index 00000000..0aa50f1f --- /dev/null +++ b/docs/diagrams/cwlviewer.svg @@ -0,0 +1,4431 @@ + + + +CWLToolRunnerWorkflowControllerPostgreSQLgitcwltoolWorkflowServiceUser"compile.cwl" Git fileHTTP POST /workflowValidates andparses CWL filegetWorkflow (GitDetails)findByRetrievedFrom (GitDetails)null?NoReturn WorkflowYescreateQueuedWorkflow (GitDetails) + + + + + + + + + + + + + + + + + + +gitsemaphoreGitServicegetRepository (GitDetails, semaphoreFlag)git clone...CWLServiceGit (jgit)parseWorkflowNative(file, packedWorkflowId)CWLToolgetVersion()cwltool --versionWorkflowSets Git info,retrievedOn,last commit dataCreates aQueuedWorkflowWorkflowRepositoryQueuedWorkflowRepositorysave(queuedWorkflow)createWorkflowFromQueued(queuedWorkflow) + + + + + + + + + + + + + + + + + + +gitsemaphoreReturn QueuedWorkflow + + + + + + + + + + + + + + + + + + +gitsemaphoreA Spring Component, that keeps a Mapof Threads x Git repository URL's.12Temporaryfiles createdCWLToolRunnerAsync2createWorkflowFromQueued(queuedWorkflow)Async + + + + + + + + + + + + + + + + + + +gitsemaphoreGitServicegitgit clone... Temporary files created(git semaphorecontrols it)getRepositoryCWLServiceparseWorkflowWithCwltoolWorkflowCWLToolgetRDFRDFServicecwltoolcwltool --print-rdfgraphExistsexists?NostoreModelgetLabelAndDocgetInputs stepNameFromURI labelFromNamegetOutputs stepNameFromURI labelFromNamegetSteps stepNameFromURI formatDefault strToRuntype labelFromNamegetLicensegetDockerLinkApache JenaTDB2SPARQL andJena API + + + + + + + + + + + + + +Sets Git info,retrievedOn,last commit dataWorkflowRepositorysave(Workflow)PostgreSQL + + + + + + + + + + + + + +ROBundleFactorycreateWorkflowRO(Workflow)3Async3ROBundleFactorycreateWorkflowRO(Workflow)AsyncROBundleServicecreateBundleBundle + + + + + + + + + + + + + + + + + + +gitsemaphoreGitServicegetRepositorygitgit clone... Temporary files created(git semaphorecontrols it)Git (jgit)Git (jgit)ApacheTavernaCreates a Bundleusing ApacheTaverna's APIAdd files fromGit to Bundle + + + + + + + + + + + + + + + + + + +gitsemaphoreGraphVizServicegetGraphStream("png")getGraphStream("svg")CWLToolcwltoolcwltool --packgetPackedVersionRDFServicegraphExistsgetModelsaveToFile(Bundle)Temporaryfiles createdSet WorkflowROBundlePathWorkflowRepositorysave(Workflow)PostgreSQL + + + + + + + + + + + + + +Apache JenaTDB2/robundle/{domain}.com/{owner}...HTTP GetResearch Objects Bundlesare created asynchronouslyas QueueWorkflow are created.When a user browses thepage for a workflow, there isalso an Ajax call that happensevery 5 seconds to an HTTPGET endpoint.This endpoint will eventuallycall this same createWorkflowROas the Spring Async call above. diff --git a/docs/diagrams/cwlviewer_flowchat_01.png b/docs/diagrams/cwlviewer_flowchat_01.png new file mode 100755 index 00000000..81dfdfd2 Binary files /dev/null and b/docs/diagrams/cwlviewer_flowchat_01.png differ diff --git a/docs/diagrams/cwlviewer_flowchat_02.png b/docs/diagrams/cwlviewer_flowchat_02.png new file mode 100755 index 00000000..a5195bca Binary files /dev/null and b/docs/diagrams/cwlviewer_flowchat_02.png differ diff --git a/docs/diagrams/cwlviewer_flowchat_03.png b/docs/diagrams/cwlviewer_flowchat_03.png new file mode 100755 index 00000000..0d808645 Binary files /dev/null and b/docs/diagrams/cwlviewer_flowchat_03.png differ