Roald Nefs, SRE/DevOps engineer and open-source contributor

How to Automatically Generate Clients for your REST API

While helping a colleague with adding some code to the bunq Python SDK to allow him to retrieve some additional information from the API (bunq/sdk_python#148), we noticed that the SDK was automatically generated. We’ve eventually ended up monkey patching the SDK, as we couldn’t make a pull request to the SDK and the API specification or SDK generator wasn’t publicly available. However, this aroused some interest about the automatic generation of API clients.

In this post we will automatically generate a REST API client based upon an OpenAPI specification. The OpenAPI specification allows REST APIs to be described in a standard programming language-agnostic way, making it possible for both humans and computers to interpret the specification.

Write the OpenAPI specification

Even though quite a lot of REST APIs nowadays come included with an OpenAPI specification, this is not the case for all of them. In this post we will be using the JSON interface on xkcd as an example. We will start by describing the OpenAPI specification for the two existing endpoints:

GET: http://xkcd.com/info.0.json (current comic)
GET: http://xkcd.com/614/info.0.json (comic #614)

Let’s start by creating a file called openapi.yaml. After filling in some basic information such as the API information and available server(s), we can add the individual endpoints. The specification below only contains the endpoint for retrieving a xkcd comic by its identifier.

openapi: 3.0.0

info:
  version: 1.0.0
  title: xkcd
  description: 'A webcomic of romance, sarcasm, math, and language.'

servers:
  - url: https://xkcd.com/
    description: Official xkcd JSON interface

paths:
  # Retrieve a comic by its identifier
  /{id}/info.0.json:
    get:
      tags:
        - comic
      description: Returns comic based on ID
      summary: Find comic by ID
      operationId: getComicById
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: integer
      responses:
        '200':
          description: Successfully returned a commmic
          content:
            application/json:
              schema:
                type: object
                properties:
                  link:
                    type: string
                  img:
                    type: string
                  title:
                    type: string

Automatically generate a Python SDK

The OpenAPI Generator allows the generation of API client libraries and documentation for multiple languages and frameworks given an OpenAPI specification. This includes generators for Python, Go, Java and many more.

Even though the OpenAPI Generator is written in Java, you can use the pre-built Docker image to act as a standalone executable. The command below will generate a Python SDK based upon the specification (openapi.yaml):

$ docker run --rm \
  -v ${PWD}:/local openapitools/openapi-generator-cli generate \
  -i /local/openapi.yaml \
  -g python \
  -o /local/out/python \
  --additional-properties=packageName=xkcd_client,projectName=xkcd_client

After execution the newly generated Python SDK is available in ./out/python and comes included with the necessary documentation to start using it. It even comes with a .openapi-generator-ignore file which let you specify files to prevent them from being overwritten by the generator.

Improve the OpenAPI specification

Even though we can now use the newly generated SDK, it comes with some very generic class names and ill-favoured function names, e.g.:

import xkcd_client


# Initialize a xkcd API client
configuration = xkcd_client.Configuration(host="https://xkcd.com")
client = xkcd_client.ApiClient(configuration)
api_instance = xkcd_client.api.default_api.DefaultApi(api_client)

# Retrieve a comic by identifier
api_response = api_instance.id_info0_json_get(614)

Let’s improve the OpenAPI specification to allow the generation of some more human friendly code. To rename the DefaultApi class we will need to logical group endpoints by adding the tags option:

...
paths:
  # Retrieve the current comic
  /{id}/info.0.json:
    get:
      # A list of tags to logical group operations by resources and any other
      # qualifier. 
      tags:
        - comic
      description: Returns comic based on ID
...

In order to rename the function DefaultApi.id_info0_json_get we can specify a unique operationId to allow tools and libraries to uniquely identify an operation:

...
paths:
  # Retrieve the current comic
  /{id}/info.0.json:
    get:
      # A list of tags to logical group operations by resources and any other
      # qualifier. 
      tags:
        - comic
      description: Returns comic based on ID
      # Unique identifier for the operation, tools and libraries may use the
      # operationId to uniquely identify an operation.
      operationId: getComic
...

The full OpenAPI specification of the xkcd JSON interface is available on GitHub.

Using the generated Python SDK

After generation of the Python SDK we can create a Python virtual environment and install the generated xkcd Python API client using the following commands:

$ python3 -m venv env
$ source env/bin/activate
$ pip install -e ./out/python

After installing the generated API client, we could for example retrieve the xkcd comic about an API using the following Python snippet:

from pprint import pprint

from xkcd_client import Configuration, ApiClient, ApiException
from xkcd_client.api.comic_api import ComicApi


configuration = Configuration(host="https://xkcd.com")

# Enter a context with an instance of the API client
with ApiClient(configuration) as client:
    # Create an instance of the API class
    instance = ComicApi(client)

    try:
        # Retrieve the API comic
        api_response = instance.get_comic_by_id(1481)
        pprint(api_response)
    except ApiException as exc:
        print("Exception when calling ComicApi->get_comic_by_id: {0}\n".format(exc))

Conclusion

In general, it’s quite possible to automatically generate a REST API based upon OpenAPI specification. Even for REST APIs that do not include a OpenAPI specification by default you can easily describe the API in a specification file to allow the API client generation. Although this post didn’t go in depth about API authentication, cookie usage or other HTTP methods, they are all possible features of the OpenAPI specification and OpenAPI Generator. However, results and options may differ based upon the language or framework you would like to generate the SDK in.

Even though the generated code may seem a bit rough, it’s well documented and could be made more human friendly by improving the API specification.

comments powered by Disqus