Skip to content
This repository was archived by the owner on Aug 25, 2024. It is now read-only.

Commit 2e42032

Browse files
pdxjohnnypdxjohnny
committed
examples: swportal: Add example
Signed-off-by: John Andersen <[email protected]>
1 parent 8c3b1b2 commit 2e42032

21 files changed

+525
-0
lines changed

‎.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ docs/plugins/dffml_*.rst
2929
docs/api/**
3030
docs/changelog.md
3131
docs/shouldi.md
32+
docs/swportal.rst
3233
docs/contributing/consoletest.md
3334
/consoletest/
3435
tests/service/logs.txt

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
8181
- Gitter chatbot tutorial.
8282
- Option to run dataflow without sources from cli.
8383
- Sphinx extension for automated testing of tutorials (consoletest)
84+
- Example of software portal using DataFlows and HTTP service
8485
### Changed
8586
- Renamed `-seed` to `-inputs` in `dataflow create` command
8687
- Renamed configloader/png to configloader/image and added support for loading JPEG and TIFF file formats

‎docs/index.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ branch can be found `here <master/index.html>`_.
7070
:caption: Subprojects
7171

7272
shouldi
73+
swportal
7374

7475
.. toctree::
7576
:glob:

‎examples/swportal/README.rst

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
Software Portal
2+
===============
3+
4+
Example of using DFFML to create a web app to discover information about
5+
software.
6+
7+
All the code for this example project is located under the
8+
`examples/swportal <https://github.com/intel/dffml/blob/master/examples/swportal/>`_
9+
directory of the DFFML source code.
10+
11+
Setup
12+
-----
13+
14+
Install dependencies
15+
16+
.. code-block:: console
17+
:test:
18+
19+
$ python -m pip install -U pip setuptools wheel
20+
$ python -m pip install -U dffml dffml-service-http
21+
$ python -m pip install -r requirements.txt
22+
23+
Usage
24+
-----
25+
26+
Run the http service and navigate to http://localhost:8080/
27+
28+
.. warning::
29+
30+
The ``-insecure`` flag is only being used here to speed up this
31+
tutorial. See documentation on HTTP API
32+
:doc:`/plugins/service/http/security` for more information.
33+
34+
.. code-block:: console
35+
:test:
36+
:daemon: 8080
37+
38+
$ dffml service http server \
39+
-port 8080 \
40+
-mc-atomic \
41+
-mc-config projects \
42+
-static html-client \
43+
-insecure
44+
45+
Query all projects
46+
47+
.. code-block:: console
48+
:test:
49+
:replace: cmds[0][-1] = cmds[0][-1].replace("8080", str(ctx["HTTP_SERVER"]["8080"]))
50+
51+
$ curl -sf http://localhost:8080/projects
52+
53+
Get a specific project. This triggers a project's DataFlow to run.
54+
55+
.. code-block:: console
56+
:test:
57+
:replace: cmds[0][-1] = cmds[0][-1].replace("8080", str(ctx["HTTP_SERVER"]["8080"]))
58+
59+
$ curl -sf http://localhost:8080/projects/72b4720a-a547-4ef7-9729-dbbe3265ddaa
60+
61+
Structure
62+
---------
63+
64+
- The codebase for the client is in `html-client/`
65+
66+
- The DataFlows for each project are in `projects/`
67+
68+
- Custom operations reside in the `operations/` directory
69+
70+
- Depdendencies are listed in the top level `requirements.txt` file.
71+
72+
HTML Client
73+
+++++++++++
74+
75+
The website displayed to clients is all vanila HTML, CSS, and JavaScript.
76+
77+
Projects
78+
++++++++
79+
80+
Each project has a DataFlow which describes how data for the project should be
81+
collected. Static data can be added directly to the dataflow file. When
82+
gernating data dynamiclly is required, code can be added to `operations/`.
83+
84+
Notes
85+
-----
86+
87+
Run a single project's DataFlow from the command line
88+
89+
.. code-block:: console
90+
:test:
91+
92+
$ dffml dataflow run single \
93+
-dataflow projects/df/b7cf5596-d427-4ae3-9e95-44be879eae73.yaml \
94+
-log debug

‎examples/swportal/html-client/index.html

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
<!DOCTYPE html>
2+
<meta charset="UTF-8">
3+
<html>
4+
<head>
5+
<title>Software Portal</title>
6+
<script type="text/javascript" src="index.js"></script>
7+
<script
8+
src="https://unpkg.com/miragejs/dist/mirage-umd.js"
9+
crossorigin
10+
></script>
11+
<link rel="stylesheet" type="text/css" href="theme.css">
12+
</head>
13+
<body>
14+
<h1>Software Portal</h1>
15+
16+
<hr />
17+
18+
<div id="displayProjectContainer" style="display:none">
19+
<h3 id="displayProjectName"></h3>
20+
21+
<table id="displayProjectTable" style="width:400px">
22+
<tr>
23+
<th>Indicator</th>
24+
<th>Status</th>
25+
</tr>
26+
<tr>
27+
<td>Static Analysis</td>
28+
<td id="displayProjectStaticAnalysis"></td>
29+
</tr>
30+
<tr>
31+
<td>Legal Compliance</td>
32+
<td id="displayProjectLegal"></td>
33+
</tr>
34+
</table>
35+
36+
<br />
37+
<hr />
38+
<br />
39+
</div>
40+
41+
<div>
42+
<a href="#" onclick="app.refreshDisplayProject()">Unselect Project</a>
43+
<ul id="projectsList"></ul>
44+
</div>
45+
46+
</body>
47+
</html>

‎examples/swportal/html-client/index.js

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
class API {
2+
_getOnlyContext(results) {
3+
return results[Object.keys(results)[0]];
4+
}
5+
6+
async getProject (projectUUID) {
7+
return this._getOnlyContext(await (await fetch('/projects/' + projectUUID)).json());
8+
}
9+
10+
async getProjects () {
11+
return this._getOnlyContext(this._getOnlyContext(await (await fetch('/projects')).json()));
12+
}
13+
}
14+
15+
16+
class App {
17+
constructor(api) {
18+
this.api = api;
19+
this.elements = {};
20+
}
21+
22+
getElements(root) {
23+
var elements = {};
24+
25+
// Create an object to hold references to all elements, keys are element ids.
26+
for (var element of document.querySelectorAll("[id]")) {
27+
elements[element.id] = element;
28+
}
29+
30+
return elements;
31+
}
32+
33+
refreshElements(root) {
34+
this.elements = this.getElements(root);
35+
}
36+
37+
populateDisplayProject(project) {
38+
// Hide the displaed project if there is no data to display
39+
if (typeof project === "undefined" || project === null) {
40+
this.elements.displayProjectContainer.style.display = "none";
41+
return;
42+
}
43+
44+
this.elements.displayProjectContainer.style.display = "block";
45+
this.elements.displayProjectName.innerText = project.name;
46+
this.elements.displayProjectStaticAnalysis.innerText = project.staticAnalysis;
47+
this.elements.displayProjectLegal.innerText = project.legal;
48+
}
49+
50+
async refreshDisplayProject (projectUUID) {
51+
if (typeof projectUUID === "undefined" ||
52+
projectUUID === null ||
53+
projectUUID === "")
54+
this.populateDisplayProject();
55+
else
56+
this.populateDisplayProject(await this.api.getProject(projectUUID));
57+
}
58+
59+
populateProjectsList(projects) {
60+
// Clear the list
61+
this.elements.projectsList.innerHTML = "";
62+
// Create a list element for each project
63+
Object.entries(projects).forEach(([uuid, project]) => {
64+
var listItem = document.createElement("li");
65+
var listItemLink = document.createElement("a");
66+
67+
this.elements.projectsList.appendChild(listItem);
68+
listItem.appendChild(listItemLink);
69+
70+
listItemLink.innerText = project.name;
71+
listItemLink.href = "#" + uuid;
72+
listItemLink.onclick = (() => {
73+
this.refreshDisplayProject(uuid);
74+
});
75+
});
76+
}
77+
78+
async refreshProjectsList () {
79+
this.populateProjectsList(await this.api.getProjects());
80+
}
81+
}
82+
83+
var app = new App(new API());
84+
85+
window.addEventListener('DOMContentLoaded', async function(event) {
86+
// DOM loaded, grab all DOM elements we'll be working with
87+
app.refreshElements(document);
88+
89+
// Get the list of projects
90+
app.refreshProjectsList();
91+
92+
// If there is a project hash, display it
93+
app.refreshDisplayProject(window.location.hash.replace("#", ""));
94+
});

‎examples/swportal/html-client/theme.css

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
.good {
2+
background-color: lightgreen;
3+
}
4+
5+
.dangerous {
6+
background-color: red;
7+
}
8+
9+
table, th, td {
10+
border: 1px solid black;
11+
border-collapse: collapse;
12+
}

‎examples/swportal/operations/__init__.py

Whitespace-only changes.

‎examples/swportal/operations/definitions.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
from dffml import Definition
2+
3+
UUID = Definition(name="uuid", primitive="string")
4+
NAME = Definition(name="name", primitive="string")
5+
STATIC_ANALYSIS = Definition(name="staticAnalysis", primitive="string")
6+
LEGEL = Definition(name="legal", primitive="string")

‎examples/swportal/operations/get_from_http_server.py

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
"""
2+
This is an example operation of how one might query an API to get the status of
3+
a project's indicator. We use the httptest module to start an example API
4+
server.
5+
"""
6+
import random
7+
8+
import aiohttp
9+
import httptest
10+
from dffml import op, Definition
11+
12+
from .definitions import UUID
13+
14+
15+
class ExampleAPIServer(httptest.Handler):
16+
def do_GET(self):
17+
self.send_response(200)
18+
self.send_header("Content-type", "text/plain")
19+
self.end_headers()
20+
self.wfile.write(random.choice(["PASS", "FAIL"]).encode())
21+
22+
23+
@httptest.Server(ExampleAPIServer)
24+
async def make_request_to_example_server(session, ts=httptest.NoServer()):
25+
async with session.get(ts.url()) as resp:
26+
return (await resp.read()).decode()
27+
28+
29+
@op(
30+
inputs={"uuid": UUID},
31+
outputs={"result": Definition(name="api_result", primitive="string")},
32+
imp_enter={
33+
"session": (lambda self: aiohttp.ClientSession(trust_env=True))
34+
},
35+
)
36+
async def query_an_api(self, uuid: str, ts=httptest.NoServer()) -> str:
37+
return {
38+
"result": await make_request_to_example_server(self.parent.session)
39+
}

‎examples/swportal/operations/projects.py

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
import pathlib
2+
3+
import yaml
4+
from dffml import op, DataFlow
5+
6+
DATAFLOW_DIRECTORY = pathlib.Path(__file__).parent.parent / "projects" / "df"
7+
EXPORT_SEED_DEFINITIONS = ("uuid", "name")
8+
NON_PROJECT_DATAFLOWS = ("projects",)
9+
10+
11+
@op
12+
def projects() -> dict:
13+
return {
14+
path.stem: {
15+
i.definition.name: i.value
16+
for i in DataFlow._fromdict(
17+
**yaml.safe_load(path.read_text())
18+
).seed
19+
if i.definition.name in EXPORT_SEED_DEFINITIONS
20+
}
21+
for path in DATAFLOW_DIRECTORY.glob("*.yaml")
22+
if path.stem not in NON_PROJECT_DATAFLOWS
23+
}

0 commit comments

Comments
 (0)