Allow documenting HATEOAS APIs (pathless operation / interface / URI class / operation type)

[ePaul]

TLDR

(added after some discussion)

• "interfaces" (like path items, just without path parameters, and not linked to a path template)
• a path item can refer to an interface (which is implemented), possibly adding path parameters to it
• a string property (containing an URI) in a schema can refer to an interface (which means the consumer can know which operations can be used on this URI)

Background / Current Situation

I work at a company where our Restful API Guidelines prescribe both "document your API using OpenAPI" and "Use HATEOAS" ("Hypertext as the Engine of Application State", one of the core principles of REST).

Unfortunately both do not work together nicely:

• Swagger/OpenAPI 2.0's model is "we have paths which can be distinguished in some way, and describe the operations on those paths" (including their data format).
• The HATEOAS model is "The client gets a result which might contain links to other resources, and can decide to follow them, no matter which URI format they have".

Currently the intersection between both is "have links in your result, but just to URIs which are also described as paths in your Swagger definition, and describe in descriptions what kind of URIs you can expect".

Ideas

A core point of HATEOAS are media types and link relations. An API definition should mainly be a description of the media types and what operations they imply for links included in them. (Paraphrased from a blogpost of Roy Fielding.)

Media type definitions are analogous to what OpenAPI's definitions are currently, might even be an extension of that. For models with a property of type string, format uri we can then additionally refer to the URI class of that URI.

A URI class is defined analogously to the current path definitions, but with some name to be used in model definitions. The URI class defines what operations are possible on an URI, which parameters are allowed/needed (no path parameters, I guess), and what can be expected in return. (The URI class is not a property of the URI itself, but of its usage in a specific place. The same URI could appear with multiple URI classes.)

I'm not yet sure how link relations can come into this – I guess we might need a way to define the URI class depending on link relation or similar.

A client needs just some initial "bookmark URI" (so a single path definition might be fine), and then can take a link (with a given URI class) and use the operation behind the URI class to do what it wants to do. The format of the second URI doesn't matter anymore to the client – it can even have a different domain name than the bookmark one. But still everything can be strongly typed if wanted.

[ePaul]

I guess this is a bit related to #576 (that discussion caused me to write this now down, though I had the ideas in me for a while), but this is much wider in scope.

[mpnally]

See my comments on #576 for one possible approach to this.

[ePaul]

@mpnally's comment on #576 gives some ideas on how this could be done, thanks.
I would prefer not to use YAML-specific features and stay within what can be expressed in JSON, too.

I can imagine something like this (modified from the linked comment):

uri-classes:
   Widget:
      get:
        ...
      put:
        ...
paths:
   /widget/{widget_id}:
      parameters:
         - name: widget_id
           in: path
           ...
      uri-class:
        $ref: '#/uri-classes/Widget'

definitions:
  Doohickey:
    properties:
      widget:
        description: hyperlink to a Widget
        format: uri
        type: string
        uri-class:
          $ref: '#/uri-classes/Widget'

(The path definition here isn't even needed if clients don't have to know the widget_id, and can reach that kind of uri from somewhere else. It might be needed for implementation help on the server side, but is not needed in the interface to be used by the client.)

I'm not sure if interface (or x-interface) is a good name for this "kind of resource to be find behind a URI", which I called "URI class", missing a better term.

Of course, using $ref here seems to imply an URI class can also be defined inline instead of referring to one defined in a special section.

[mpnally]

Terminology aside, your use of $ref is better than using the YAML merge operator (<<). I used YAML merge because I needed a solution that was legal today in swagger 2.0 and so would work with existing tools.

[DavidBiesack]

The use of x-interfaces / x-interface in #576 seems to solve a different problem (that of several APIs or API subresources implementing an API contract/interface/pattern.)

I'm more interested in the OP question of defining and documenting an API via HATEOAS. We'd like to hide the paths in the API (i.e. in the UI) and encourage clients to consume links rather that hard-code the API to fixed paths. (The paths become more of an implementation detail. They still exist and could be revealed in the UI for reference / lookup, or presented in an alternate view.)

Thus, an API starts at the root and the client can discover the set of links to nested resources via compound keys that include a link relation (name), a verb (PUT, POST, etc.), a type (request and/or response media type), and a description. The definition of a link may also include a path but that is more of an implementation attribute. Thus, each path in an OAS can be associated with one or more contained resources

For example if the API has two primary resources, models and activities, the root
would contain

paths:
  /:
    resources:
      root:
        links:
          models: 
               rel: models
               method: GET
               uri: /models
               type: application/vnd.collection+json
               description: Retrieve a paginated list of models
          createModel:
               method: POST
               uri: /models
               type: application/vnd.example.model+json
               description: Create a new model resource in this collection
          activities: 
               method: GET
               uri: /activities
               type: application/vnd.collection+json
               description: Retrieve a paginated list of activities
          createActivity:
               method: POST
               uri: /models
               type: application/vnd.example.activity+json
               description: Create a new activity resource in this collection


  /models:
    resources:
      models:
        collectionItems:
          model:
             uri: /models/{modelId}
        links:      # these are links associated with the collection
          self:
            ...
          next:
            ...
          prev:
            ...
          first:
            ...
          last:
            ...
          create:
            ...
  /models/{modelId}:
    resources:
      model:
        links:
          self:
            method: GET
            type: application/vnd.example.model+json
            description: Fetch a representation of this model
          image:
            method: GET
            type: image/png
            description: Fetch a graphical representation of this model
          pdf:
            method: GET
            type: application/pdf
            description: Fetch a PDF representation of this model
          update:
            method: PUT
            type: application/vnd.example.model+json
            description: Replace the representation of this mdoel
          patch:
            method: PATCH
            type: application/vnd.example.model+json
            description: Update the representation of this mdoel with partial representation
          delete:
            method: DELETE
            description: Delete this model.
          analyze:
            method: POST
            type: application/vnd.example.activities+json
            uri: /activities?model={modelId}
            description: Create an analysis activity

So from the root, one can find models and activities (as resources), and also discover how to create activities, and from the collection, one can access an individual model. From a model, there are links for operations on that model (the uri is inherited from the current path context).

The UI could then navigate the API by exploring links (hiding the URI by default).

[ePaul]

@DavidBiesack I'm not sure if I understand you right ... is the example code there a proposed syntax for a API specification, example output, or something else?

I guess resource (or rather resource-type) could be a better name than interface for what I want to describe in my API, though your description still seems a lot too path-centric for me.

[DavidBiesack]

It is a sketch for additions to OAS -- i.e. addition of a resources element within a path element to allow specifying what resource(s) are found at that path, and what links the API will return on GET operations to the collection or item resources.

Yes, It is path-centric in that I'm trying to integrate with the existing Swagger 2.0 schema of paths rather than completely undo it all. I'm trying to express resource-oriented architecture, not RPC style. again, the UI or other tools can hide the paths (optionally) and present a link/resource/discovery focused view of the API.

[ePaul]

@DavidBiesack I would define the resource types separately, and then in the paths (and in content schema definitions for URIs) just refer to them.

The resource types in your case would be (if I understood right):

• entry point (= bookmark URI)
  • allows retrieval (GET), gives links to list of models, model creation, list of activities, and activity creation.
• list of models
  • allows retrieval (GET), gives list links to single models (maybe with those models already inlined)
• model creation
  • (by POST) → results in a new single model resource being created.
• single model
  • allows retrieval (GET) → gives JSON representation, includes links to PDF and PNG versions
  • allows update (PUT/PATCH)
  • allows removal (DELETE)
• list of activities
  • allows retrieval (GET)
• activity creation
  • (by POST) → results in a new single activity resource being created
• single activity
  • allows retrieval (GET)
  • allows update (PUT/PATCH)
  • allows removal (DELETE)

I guess it would be possible to merge the "list of" with the "creation" resources, if that is supposed to always be the same URI.

Then we can map the paths to those resource types:

• / → entry point
• /models → list of models, model creation
• /activities → list of activities, activity creation
• /models/{modelId} → single model
• /activities → activity creation, list of activities
• /activities/{activityId} → single activity
• /activities?model={modelId} → activity creation, list of activities

Except for the entry point this mapping doesn't need to be public in the API description (but it can, for legacy users/tools who don't really know how HATEOAS works). The actual links are then provided by actually invoking the resources containing the links, they are not hardcoded in the OpenAPI definition.

(My point of view is that the API definition is something stable, from which e.g. a client library can be generated and used in some program using it. The actual URIs in use can then change on the fly without any changes necessary to the client. This is something separate from the use of tools like Swagger UI, who are able to construct the UI and do the interactions on the fly from the API definition, and thus can work easily with changing definitions).

[DavidBiesack]

@ePaul yes, defining resources separately and using a $ref mechanism is fine. I just started with a sketch, it certainly needs refinement. For example, some APIs expose resources via multiple paths, so sharing the resource definitions (which includes links but may have more...) is important. For example, the sets of links would need to be mergable/inherited -- that is, it is important to be able to use composition to define a specific resource. (The creation operations/links are an example of that: we wish to expose the createModel link both at the API root and also in the "list of models" result.) We've been calling these aspects of the API 'traits' (a word we borrowed from RAML).

Regarding stability -- our view is that we also want clients to be coded to consume links at runtime, so that the client has as few hard-coded paths as possible (for example, the root). Start at the root, get the "models" or "createModel" link (where the API provides the uri) and invoke that operation from that information.

[mpnally]

@DavidBiesack, the x-interfaces / x-interface design in #576 is trying to address the problem you want to solve. When you write:

definitions:
  Doohickey:
    properties:
      widget:
        description: hyperlink to a Widget
        format: uri
        type: string
        x-interface:
          $ref: '#/x-interfaces/Widget'

the x-interface declaration is saying that the 'widget' property of a Doohickey will hold an opaque URI—there is no path declaration for this URI and its format is not modeled is any other way. Further, this opaque URI supports the methods defined at #/x-interfaces/Widget. Of course, just because this design addresses your problem, doesn't mean you like it :) An advantage of this design is that it is quite incremental over the current Swagger 2.0—in fact it is legal Swagger 2.0 today. Note that the definition at /x-interfaces/Widget is also legal Swagger 2.0—it is the same Swagger that would normally be nested under a path. It can be included in the definition of a regular Swagger path, or, as above, used to define the interface that is supported by an opaque URL.
In practice, in our usage, we define one of these interfaces for each resource type, so the x-interface declaration is really saying "the opaque URL in the widget property of a Doohickey resource supports the method interface defined by Widget resources". We discovered these patterns while generating Swagger from another format that is custom-designed for hypermedia APIs.