Tutorial

A step by step tutorial for example at examples/jplaceholder.

Please go through this explanation of various steps involved before reading this tutorial.

Bootstrapping

Navigate to your project folder inside GOPATH and run:

codon init

Setting up Upstream Services

In this example we will be hitting fake API called JSON Placeholder. You can find the swagger specification of this API here. Place this file at spec/clients/jplaceholder.yml.

You will notice that all response definitions are set to #/definitions/ObjectResponse. This is done because complex response structures are not supported yet but generic objects are. Once the support is added, this will no longer be needed.

Next we would need to define an instance of this service in config.yml file. The result would look like this:

endpoints:
  jplaceholder:
    scheme: https
    host: jsonplaceholder.typicode.com
    client: jplaceholder

This will define a service with name as Jplaceholder and using the client library jplaceholder generated from jplaceholder.yml. scheme and host define the upstream url where this service resides. You can define multiple services using the same client library.

Designing your APIs

Swagger specification of the REST API being served is to be defined at spec/server/main.yml.

As with upstream APIs, you will notice that all response definitions are set to #/definitions/ObjectResponse. Complex response definitions are not supported yet. Apart from this, there are x-workflow swagger extensions used for every method object. For every method of a path you can specify the workflow name to trigger.

Health API

This implements a simple API for ELB health check. In paths add:

/health:
  get:
    x-workflow: get_health
    responses:
      200:
        description: Feed from API Response
        schema:
          $ref: '#/definitions/ObjectResponse'

Hitting /health with GET request will return a response based on get_health workflow. We will define a get_health workflow in the file spec/server/workflows/health.yml. The workflow is pretty simple:

name: get_health

output:
  body: <%jmes `"OK"` %>
  status_code: <%jmes `200` %>

This workflow simply responds with status code 200 and body as "OK" (json string) using the jmespath representation of these constants.

Latest Comment API

This API fetches the latest post for a user and then fetches the latest comment on that post. This is just an example to show how tasks can be chained. The API is defined as in main.yml as:

/latest_comment:
  get:
    parameters:
      - name: userId
        in: query
        type: integer
        required: false
    x-workflow: get_latest_comment
    responses:
      200:
        description: Latest Comment
        schema:
          $ref: '#/definitions/ObjectResponse'

The workflow get_latest_comment can be found at spec/server/workflows/latest_comment.yml.

We specify the name of the workflow as:

name: get_latest_comment

We define a task get_posts which first fetches the posts for a user:

get_posts:
  action: clients.jplaceholder.get_posts
  input:
    userId: <%jmes main.userId %>
  publish:
    latest_post: <%jmes action.max_by([], &id) %>
  on-success:
    - get_comments: true

The action name clients.jplaceholder.get_posts is based on the upstream service defined in the previous section. That action takes in a parameter userId (It would help to look at the upstream swagger api), so we would need to provide that parameter in input.

The main bucket contains all the parameters of the serving API as we would defined in the swagger spec for the service in main.yml. main.userId refers to input parameter to this service. userId: <%jmes main.userId %> signifies that the parameter is passed on from client to the upstream API as it is.

Once this action is run, we want to publish the response back to the main bucket so that it is available to next tasks. Here we publish the variable latest_post by performing a max_by. The post with maximum id is selected and put in main.latest_post.

On success of this task we will call get_comments unconditionally. So we provide an on-success parameter:

get_comments: true

Since the value always evaluates to true, the task is always run.

get_comments tasks is defined as:

get_comments:
  action: clients.jplaceholder.get_comments
  input:
    postId: <%jmes main.latest_post.id %>
  publish:
    latest_comment: <%jmes action.max_by([], &id) %>

It is defined in the sames lines as get_posts. The latest comment is published to main.latest_comment. Publishing of new variables is always done to the main variable bucket.

The output section can be finally defined as:

output:
  body: <%jmes main.latest_comment %>
  status_code: 200

The latest_comment object is sent as body of the response with status code 200.

Get Posts API

This example helps explain how loops work. Posts API is defined in main.yml as:

/posts:
  get:
    parameters:
      - name: userId
        in: query
        type: integer
        required: false
    x-workflow: get_posts_comments
    responses:
      200:
        description: Posts with comments
        schema:
          $ref: '#/definitions/ObjectResponse'

The workflow get_posts_comments can be found at spec/server/workflows/posts_comments.yml. This workflow must get the posts for a user and the comments for each post in the response.

The first task get_posts is similar to previous API:

get_posts:
  action: clients.jplaceholder.get_posts
  input:
    userId: <%jmes main.userId %>
  publish:
    posts: <%jmes action %>
  on-success:
    - get_all_comments: true

All the posts for the user is published to main.posts variable. On success of this action get_all_comments task is called:

get_all_comments:
  with-items: <%jmes main.posts %>
  loop:
    task: get_comments
    input:
      postId: <%jmes item.id %>
    publish:
      combined: <%jmes {"post_details":item,"comments":task.comments} %>

Loop is run over the list main.posts we published in the previous task. On each iteration of the loop a task get_comments is called with input as postId. Here the main variable bucket is reconstructed for the task. The task get_comments only has access to postId by referring to main.postId but no other variables. postId passed to each task called in iteration is different as each task gets its own main bucket. We define postId as <%jmes item.id %> where item refers to the item currently being iterated over. Since we are iterating over main.posts, item refers to individual post object in the loop.

Once this task finishes, the result is posted to the variable main.combined in the original main bucket. combined is a list with each element being defined as <%jmes {"post_details":item,"comments":task.comments} %>. This is the difference between loop task and simple task. A loop task publishes an array and the expression defined for that variable refers to individual element of that array.

task in task.comments which is part of the expression being published to combined, refers to the main variable bucket which was created for the task get_comments on every iteration. task would also contain all the variables published as by the child task in its main bucket. This would become clear after looking at get_comments task:

get_comments:
  action: clients.jplaceholder.get_comments
  input:
      postId: <%jmes main.postId %>
  publish:
    comments: <%jmes action %>

In this task main is not the same variable bucket which was provided to this workflow at the start. Since this task is being called in a loop, its main context is set by the caller get_all_comments. The result of this task is published to the variable comments at the end of this task and it would be available as main.comments inside this bucket but once control is given back to the calling task (which has its own main bucket) comments can be referred by using task.comments.

The output is specified as:

output:
  body: <%jmes main.combined %>
  status_code: 200

In summary, this workflow will get_posts and then for every post it will get_comments and then return a list of combined objects with each object having post_details and comments.

Generating and Building code

Once all the spec is written, Go code can be generated by using the command:

make generate

The code can be built using:

make build

Running the service

Run the service using:

./Jplaceholder --port=40000

The service will be available at http://localhost:40000/health