Script management vs Thing management

[zolkis]

Moving the discussion from the WoT WG mailing list to this issue.

History:

• Issue Define UA/runtime for WoT #2 tries to clarify the definition and scope for WoT Runtime (WR) vs scripts vs Things
• earlier attempts to define a script management interface ran into the issue whether such interface is covered by the WoT WG charter.

Proposal: allow Thing creation and TD modifications not only from local, but also remote WoT Runtimes.

Advantages:

• no need for separate script management concepts and interface
• WoT application is defined as a set of Things running in one or more WoT Runtimes, and deployment reduces to using the Scripting API, in fact developing a deployment script run in a WR (e.g. in a browser) that would modify/add functionality of remote devices
• WoT Runtime would encapsulate security related checks and policies, and could validate deployed functionality based on the TD.

Disadvantages:

• there are strong opinions that a script should only affect local WR (i.e. create/modify only Thing local to the WR)
• assumption on a connection, proxy object and protocol/serialization of Thing
• conceptually speaking, managing remote Things is not different from managing scripts, so we'd rather manage scripts (proposal needed on how).

[zolkis]

Showcase scenario (from here):

Deploy WoT functionality to WoT Runtimes. This functionality is called an Application, and has an associated security identity and access rights.
An Application may be defined as a script, or a set of scripts, eventually with added configuration data and manifest data.
Alternatively, an Application may also be represented by a set of Things that are created and deployed in a WoT Runtime. The functionality is characterized by the Thing Descriptions of the participating Things.

The draft algorithm for creating a remote Thing is the following:

1. A script running in the initiating (controlling) WoT Runtime (WR) invokes the wot.create() API with the destination specified as the target (controlled) WoT Runtime. The method specifies the Thing Description and optionally also security related metadata (e.g. policy on further modification accesses). The implementation (bindings) of the participating WRs has been provisioned with identity, authentication mechanism, and access management functionality.
2. The target WR authenticates the initiating WR and checks for access rights, using either a local or a network based policy. If there are not sufficient rights for the request, reject the request with a security error.
3. If access rights are granted, the target WR creates the required Thing (involving full lifecycle integration, such as event loop, secure storage etc) and records the security metadata of the transaction (e.g. creator identity, group/security level, access policy etc). If creating fails, the transaction is rolled back and the request is rejected with an error.
4. If creating succeeds, the initiating WR generates an ExposedThing object for controlling the newly created Thing. Further modification requests are separately authenticated or a session may be maintained, depending on the capabilities of the underlying protocols.

The draft algorithm for script management:

• write script
• deploy script to target WoT Runtime (need to define how).
• run script on target WR (need to define how). It will create local Things that implement to solution.

Possible solutions:

1. Standardize a special Thing that will be able to manage scripts. Standardize its TD.
   If WoT requires special Thing to run it functionality, it may run into the hell of special Things, including the need to specify related protocol(s), provisioning etc.
2. Define a Script Management Interface in the Scripting API.
   Assumed to be out of scope for Scripting API and WoT WG charter. To be confirmed!
   It would not be much different from managing remote Things, but needs a new interface with new algorithms, concepts to operate with, and issues to solve.

[mkovatsc]

My proposal would look the following (might still have bugs, since I am currently very short on time):

• We identify which interactions are required to set up and manage Things in a remote runtime.
• We model these interactions in terms of Properties, Actions, and Events. These then can be offered together with additional management Interactions by a management Thing on the remote Servient; the latter might spawn management Things that are coupled with the instantiated Thing; or it might merge management Interactions and application Interactions into a single Thing -- it is the remote Servient that gets to decide. Interoperability stays at the Interaction level as for all over-the-network Interactions in the Web of Things.
• We (at the right point) standardize the TD vocabulary that semantically describes the modeled management interactions.
• We secure these management interactions using the same security mechanisms as for normal application-level interactions (whatever the implementing IoT platform can provide or what we recommend for new implementation, e.g., the JWT-based work used in previous PlugFests). This provides flexibility to be future-proof.
• We implement an orchestration application based on the show case. We only use the core/narrow-waist Scripting API (getProperty, setProperty, invokeAction, ...) without defining new convenience functions for the management use case on the WoT object.
• The common bits that ease operating on top of Properties, Actions, and Events can be put into a library (somewhat like jQuery), since we not only need this for management, but also for actual applications.
• Over time I imagine innovation for good WoT orchestration applications. There might be multiple, following different goals (easy-to-use, enterprise, domain-specifics, ...).
• Once we understood them, we can think about standardizing convenience functions in the runtime, just like it was in Web browsers, but in a different namespace than the core, long-lived WoT object.

[zolkis]

We identify which interactions are required to set up and manage Things in a remote runtime.

1. create Thing based on TD
2. add/remove properties, events, actions
3. set request handlers.

It can be done either by a script deployed and executed in the remote WR that uses the Server API, or by interacting in a special way with a special Thing (using the client API). If I understand you right, you propose the latter.

These then can be offered together with additional management Interactions by a management Thing on the remote Servient; the latter might spawn management Things that are coupled with the instantiated Thing

1. What about considering this special Thing part of the WoT Runtime, meaning this special Thing identifies the WoT Runtime? This is also advisable since the WR is the security enforcement point.
2. The details of this "additional management interactions" is the whole meat. We need to define them before continuing the discussion.

We implement an orchestration application based on the show case. We only use the core/narrow-waist Scripting API (getProperty, setProperty, invokeAction, ...) without defining new convenience functions for the management use case on the WoT object.

1. Please define "orchestration", and how does it relate to script deployment?
2. By "narrow-waist Scripting API" do you mean the Client API? Could we use unambiguous names?
3. Where and how the Server API would be used in your proposal? Is the Server API now replaced by one or more special Things with special interactions? (once we can set up Things using a client API interacting with a special Thing, we don't any more need the server API). It may be a good thing. But then why do we need the client API - we could just use an interaction model. No Scripting API needed! Good riddance :).

[mkovatsc]

Think of the runtime as a box. On one surface we cut a whole in this box; this is the interaction model (uniform interface with Properties, Actions, Events). If an interaction originates inside the box, goes through the hole, and targets something on the outside, the interaction makes use of the client role of the hole. If an interaction originates outside the box, passes through the hole, and targets something inside the box, the interaction makes use of the server role of the hole.

From the outside, we cannot look inside the box. We only see Thing Descriptions that tells us how to push stuff through the hole. Inside the box we cannot reach outside, unless we use the hole; the Scripting API tells us how to get stuff though it.

The hole is a single, nicely cut hole with clear rules: the narrow waist. We want to avoid randomly stabbing additional holes in the box; each would have its individual rules how to get stuff through (cf. tightly coupled, service-oriented approach).

[zolkis]

You have described the runtime model, but not the deployment model. This whole discussion is on how to relate the two. We know what a script is, what a Thing is. What we don't know (haven't agreed about) how a script gets to the target device.

Think of the runtime as a box. On one surface we cut a whole in this box; this is the interaction model (uniform interface with Properties, Actions, Events).

IOW this is the Client API. The interaction model is underneath and works with the same concepts, but they are different.

If we take the analogy of "holes", it would make it easier to picture if client and server holes were different, rather than having "roles".

From the outside, we cannot look inside the box. We only see the Thing Description that tells us how to push stuff through the hole.

IOW the TD provides structural and interaction definitions. But it's a static thing. You are not generating bindings when requests are served. You implement a functionality that takes into account the TD, but has to be deployed and run in the WR (box).

The hole is a single, extremely nicely cut hole with clear rules: the narrow waist

I cannot make any practical sense out of this sentence. Define "extremely nicely cut" (?), and the "clear rules" (defined by TD?). I understand that now the "narrow waist" is the other two :).

We want to avoid randomly stabbing additional holes in the box; each would have its individual rules how to get stuff through (cf. "service oriented").

That I understand, and it's a good goal. However, we are not talking about multiple random holes. We do need one extremely nicely cut hole also for getting functionality (script, things) into the box. Either through the same hole, or a different one. Your proposal still misses that.

The current assumption is that these are flashed into the box. This is not good enough in 2017. We can do much more flexible things already with OCF.

[draggett]

My use case about installing a service on a home hub from a public web page seems relevant. I am assuming that the hub has a thing that acts as a manager, and that with the right permissions, you can remotely invoke an action on this thing to request the installation of another thing. The manager thing then downloads and installs the new thing's implementation. The manager thing is also responsible for managing updates. The detailed requirements then follow from a consideration of security, privacy, and so forth.

[h0ru5]

I see a certain disconnect in the discussion and assume its mostly because we take too many steps at the same time and have two concepts that consist of more than one assumption/statement.

If we can collect the points/statements we see in common, maybe it helps to identify the ones we disagree on.

Do we agree:

• we have a WoT-Runtime that executes scripts, hosts Things and takes security decisions
• there are two application-facing interfaces that are used by the scripts - the Scripting APIs
• scripts can create and modify local Things by using the server-side API, including attaching handlers (behavior)
• scripts can access/interact with local Things via the client API
• there are local Things exposed via server-side network-facing interfaces
• there are client-side network-facing interfaces in the WR that can be used to access/interact with remote things using the client API

Point currently discussed, but approaching consent:

• there will be a management-purpose network-facing interface for the WoT Runtime
• we advise it should have a "Thing" as interface, described by a TD
• we need a way to deploy logic (resp. a Thing) via the management interface
• we will not standardize the interface nor the TD, but define a vocabulary of semantic "@types" for the offered functionality

If we can agree that this is the base, then we can proceed to the points that are unclear and in heavy discussion:

• The management interface allows to run scripts on the runtime
• The management interface allows to create Things on the runtime
• The management interface allows to modify Things on the runtime
• A Thing and the handlers on a remote runtime can be created modified using the server API
• The code to interact with the management interfaces' "Thing" interface is implicitly done in the runtime - the standard defines only passing the intent via server APIs
• All calls to the management interface's "Thing" interface are done explicitly through the client API, convinence layers are not part of the standard

Please comment and correct.
In my eyes this is indeed a topic that goes beyond scripting TF, but is mostly about scoping.

[mkovatsc]

You have described the runtime model, but not the deployment model.

Yes, I want to help clarifying the Servient architecture, as it is not clearly understood yet. You describe Client API, Server API, interaction model, and Things as if they were independent concepts. Please try to see why statements like "Server API now replaced by one or more special Things with special interactions" just do not compute.

There is a strict relationship between the Thing concepts:

• There cannot be any API for accessing the network without going through Properties, Actions, or Events.
• There cannot be a network interface (e.g., a server socket) without being connected to Properties, Actions, or Events
• There cannot be Properties, Actions, or Events at a runtime without a Thing

What we are currently defining is the generic, long-lived core API for Properties, Actions, and Events (this is the scope of the WoT Scripting API deliverable).

What you are describing is a higher-level API that eases using, but must rely on the core API to fulfill management tasks. While I fully understand that you need something to instantiate Things in a runtime remotely, I am against standardizing something in the wrong place that bypasses the overall WoT architecture.

You can achieve the same functionality by following the WoT architecture and provide Properties, Actions, and Events for your needs. We can ensure interoperability, meaning that a remote runtime will do what you expect, by defining the vocabulary of the Properties, Actions, and Events for management.

The important aspect of security is how to secure and control the access from outside. This is the same as for all interaction, management or launching a missile. There is nothing special about your use case.

While you might dislike the fact that your single function call for remote instantiation becomes a series of statements for the Client API, simply put it in a library. It will also be able to run in other runtimes because it is programmed against the core API and it will be able to manage any remote runtime because it is programmed against the management vocabulary. You can share this library, so not everyone has to rewrite it to get started.

[mkovatsc]

@h0ru5 I agree with the first part of your statements.

I am not sure, however, if everyone agrees on "Point currently discussed, but approaching consent": Zoltan's proposal (judged more by the given function declarations than the used terms, which are prone to misunderstanding) shows the characteristics of a scriptable shell that has access to all connected runtimes. Independent from where the script is executed, it can instantiate and modify Things everywhere in the network using API calls. @zolkis , can you recognize your proposal in this statement?

While this is a feasible solution that carries quite some merit, it is at a different level than the WoT architecture and building blocks that we are currently standardizing. It might well be implemented as an overlay, but it must not be rooted in the core of WoT.

Furthermore, I disagree with

A Thing and the handlers on a remote runtime can be created modified using the server API

If it is remote, you need to use a client API (applies in general, independent from this issue).

The code to interact with the management interfaces' "Thing" interface is implicitly done in the runtime - the standard defines only passing the intent via server APIs

We cannot expect this to be a fundamental part of the WoT runtime. It must be optional if standardized at all. On the Web, it is good practice to implement decorators and convenience libraries as third-party packages (e.g., jQuery). A user writing the script that deploys Things in different runtimes can easily choose the decorator; there is no need to make it standard. These scripts will still be portable. (This opens the questions how to import libraries/modules in WoT scripts; more important question than the management stuff. Security enforcement is not the argument here, since it cannot even be uniformly enforced, since different runtimes might have different purposes; if we standardize this, we need to provide for any possible use case; it's like standardizing how people have to implement their Web server -- we don't want that.)

All calls to the management interface's "Thing" interface are done explicitly through the client API, convinence layers are not part of the standard

The last one even appears to contradict the first statement I cited.

[mkovatsc]

Additional question: how would you deploy client functionality?

[h0ru5]

The last one even appears to contradict the first statement I cited.

Intended. I wanted to capture an overview of the points and arguments currently on the table and sort out what is consent-able and - (in last bunch of statements) where the divergence is.

[zolkis]

Zoltan's proposal shows the characteristics of a scriptable shell that has access to all connected runtimes. Independent from where the script is executed, it can instantiate and modify Things everywhere in the network using API calls. @zolkis , can you recognize your proposal in this statement?

As mechanism, yes. As policy, usually no: it's not independent from where the script is executed! That's a world of difference. The script would run in a WR that is identified, authenticated and has to have appropriate access rights in the WoT network (analogous to Provisioning Tool in OCF). Only if security checks are passed the calls would succeed.

Additional question: how would you deploy client functionality?

Client API is a set of mechanisms that are implemented in the WR. They get installed there together with the WR. Similarly, the server API is a set of mechanisms and gets deployed the same way.
But Things are data (properties, events) + scripts (actions, handlers).
We can already solve the static case, when Things are installed together with the WR.
However, various IoT standards/networks allow more flexible definitions of solutions functionality. In WoT case that would require to deploy Things + scripts dynamically to a running WR. Currently we are missing a mechanism for this.

My problem with the current architecture is that it's too rigid for theoretic reasons and misses the practical deployment scenarios that are absolutely critical in IoT.

I don't mind if you impose constraints on these mechanisms/APIs, as long as the use cases can be implemented securely without blowing up complexity or going out of charter scope.
Security is a key point as well, and analysis is needed for both types of design. I can track what needs to be done for a secure implementation of my proposal. However, if we define special Things with interactions with special meaning, how can we enforce security policies based on data content alone (when we don't have distinct API entry points)? That will be a problem.

I am waiting for a more elaborate proposal on special Thing(s) and their interactions. BTW where do we standardize these? They would become a dependency for Scripting. I guess Scripting work should be shelved until this is clarified.

[zolkis]

@draggett

My use case about installing a service on a home hub from a public web page seems relevant. I am assuming that the hub has a thing that acts as a manager, and that with the right permissions, you can remotely invoke an action on this thing to request the installation of another thing. The manager thing then downloads and installs the new thing's implementation. The manager thing is also responsible for managing updates. The detailed requirements then follow from a consideration of security, privacy, and so forth.

I think this is a good use case. I try to map it to my current terms.
Let's call the service to be installed the WoT application/solution.
The web page likely belongs to the portal associated with the WoT applications, and has the same trust origin.
The hub acting as a manager: the WoT Runtime in the hub that has a special management Thing running in it. One action on this Thing is installing another thing. Please define the signature of this action. I assume the function code (script) of the action will take a TD parameter (in general), and will create a Thing based on it locally on the hub WR.

So far the only difference to my proposal is that I consider this special Thing part of the WR, and I have sinfully mapped the invocation of the manager Thing install action as a separate API on the WoT object - think of it as a convenience function. I also stress that there are just 2 of these needed! While IMHO this is mostly a syntactic thing, it was deemed to want to open a new hole in the box. No, it doesn't - it all depends on implementation.

That being said, I am fine with this, as long as it's standardized, including security provisioning of these special Things. We need an owner for this.

[mkovatsc]

My problem with the current architecture is that it's too rigid for theoretic reasons and misses the practical deployment scenarios that are absolutely critical in IoT.

The architecture gives you everything you would need. In @draggett 's use case description I can see the same approach I am proposing.¹ Are there reasons for you not using it or are there aspects you are missing? I didn't see you trying to follow an approach that is in line with the WoT architecture. Maybe the concepts we have refined over the past two years are not clear yet?

Client API is a set of mechanisms that are implemented in the WR. They get installed there together with the WR. Similarly, the server API is a set of mechanisms and gets deployed the same way.

The question was about client logic (behavior and initiative of a Thing to interact with other Things), the complement to server logic (which is the sum of all Action and Event handlers). Not sure how you imagine "deploying an API", which again does not compute for me.

¹ one discussible detail is that Dave chose to trigger download of Thing logic (i.e., script / handler implementations / ...), while I was thinking of passing this logic as inputData of an Action directly to the manager Thing.

[mkovatsc]

Comment on security policy:

As policy, usually no: it's not independent from where the script is executed! That's a world of difference. The script would run in a WR that is identified, authenticated and has to have appropriate access rights in the WoT network

You want attach the security principal to the runtime.

If you assume a Servient on a hub or cloud service would have multiple runtimes to allow for multi-tenancy, then we are on the same page and only have to get terminology straight: I was expecting only one runtime instance that is able to sandbox each script. Should be equivalent.

Furthermore, I assume the runtime and its sandboxes as the fixed commodity that runs somewhere. It does not have any security principals yet, because those have to be attached to the application or business logic pieces (the WoT IG has been calling them App Scripts) that stakeholders deploy there. Stakeholder A deploys an app script alongside its security credentials etc.; so does stakeholder B with his app script and the credentials corresponding to it. The operator of the runtime would probably have provisioned security credentials to the management Thing that can be used by stakeholders to deploy their app scripts. How this management Thing is implemented in detail is up to the operator, his business model, payment methods, etc. It only needs to provide the management Interactions that are standardized through the managment vocabulary. Earlier I mentioned "additional management Interactions"; these could be the chosen payment Interactions or gold user management functions, which both are defined by one or two additional vocabularies.

The tool that stakeholder a uses must have a security principal as well. It must have the credentials, so that the management Thing authorizes the tool to deploy his app script. The tool acts as client for the Interactions of the management Thing. It must follow the auth* mechanisms that are defined in the TD of the management Thing. All of this is not management-specific. This is how WoT works for all Interactions.

Coming back to the app scripts. At first I simply thought, it would be a good convention to only put the implementation of one Thing into one script. Since this must be paired with a security principal, it seems more a MUST. However, I could imagine some kind of grouping, where one script creates two Things; yet they will then have the same principal and run in the same sandbox. Appears to be useful. Anyhow, I see the security principal (= can be identified, authenticated, ...) attached to the app script, not the runtime. The runtime enforces the policy (= makes sure credentials are safe, scripts are sandboxed, no spill); just like the browser with different Web pages in multiple tabs...

If you think deploying a complete script is bad to manage from a security perspective, we can add a number of management Interactions that allow to programmatically deploy a Thing similar to creating one in the local runtime/sandbox. The Action handlers will probably still have to be set up by deploying code snippets. So in the end you still deploy JavaScript code. Thus, I don't really see the problem with deploying a complete script that then creates the Thing locally in the remote runtime/sandbox. We will still need multiple steps to deploy logic (=script code) and the principal (=credentials). Not sure if it is worth designing a container for that; is then more a question of protecting the IP in commercial script code maybe...