Behavior-Driven RESTful APIs

In the RESTBucks example, the authors present a useful state diagram that describes the actions a client can perform against the service.

Where does such an application state diagram come from? Well, it’s derived from the requirements, of course.

Since I like to specify requirements using examples, let’s see how we can derive an application state diagram from BDD-style requirements.

Example: RESTBucks state diagram

Here are the three scenarios for the Order a Drink story:

Scenario: Order a drink

Given the RESTBucks service
When I create an order for a large, semi milk latte for takeaway
Then the order is created
When I pay the order using credit card xxx1234
Then I receive a receipt
And the order is paid
When I wait until the order is ready
And I take the order
Then the order is completed

Scenario: Change an order

Given the RESTBucks service
When I create an order for a large, semi milk latte for takeaway
Then the order is created
And the size is large
When I change the order to a small size
Then the order is created
And the size is small

Scenario: Cancel an order

Given the RESTBucks service
When I create an order for a large, semi milk latte for takeaway
Then the order is created
When I cancel the order
Then the order is canceled

Let’s look at this in more detail, starting with the happy path scenario.

Given the RESTBucks service
When I create an order for a large, semi milk latte for takeaway

The first line tells me there is a REST service, at some given billboard URL. The second line tells me I can use the POST method on that URI to create an Order resource with the given properties.
bdd-rest-1

Then the order is created

This tells me the POST returns 201 with the location of the created Order resource.

When I pay the order using credit card xxx1234

This tells me there is a pay action (link relation).
bdd-rest-2

Then I receive a receipt

This tells me the response of the pay action contains the representation of a Receipt resource.
bdd-rest-3

And the order is paid

This tells me there is a link from the Receipt resource back to the Order resource. It also tells me the Order is now in paid status.
bdd-rest-4

When I wait until the order is ready

This tells me that I can refresh the Order using GET until some other process changes its state to ready.
bdd-rest-5

And I take the order

This tells me there is a take action (link relation).
bdd-rest-6

Then the order is completed

This tells me that the Order is now in completed state.
bdd-rest-7

Analyzing the other two scenarios in similar fashion gives us a state diagram that is very similar to the original in the RESTBucks example.
bdd-rest-8
The only difference is that this diagram here contains an additional action to navigate from the Receipt to the Order. This navigation is also described in the book, but not shown in the diagram in the book.

Using BDD techniques for developing RESTful APIs

Using BDD scenarios it’s quite easy to discover the application state diagram. This shouldn’t come as a surprise, since the Given/When/Then syntax of BDD scenarios is just another way of describing states and state transitions.

From the application state diagram it’s only a small step to the complete resource model. When the resource model is implemented, you can re-use the BDD scenarios to automatically verify that the implementation matches the requirements.

So all in all, BDD techniques can help us a lot when developing RESTful APIs.

HyperRosetta

rosetta-stoneThe Rosetta stone is a rock with the same text inscribed in three different languages. This allowed us to decipher Egyptian hieroglyphs.

In this post I’ll introduce a similar “stone” for hypermedia formats, using the RESTBucks example.

I think that seeing concrete hypermedia messages in different formats will make the similarities and differences clearly visible, which will hopefully make it easier to choose the right format for your API.

The text that I’ll use for HyperRosetta, the hypermedia Rosetta stone, is Example 5.6 of the REST in Practice book. For those hypermedia formats that can include templates, I’ll use the payment link with the representation in Example 5.7 of the book.

These are the media types available on HyperRosetta:

If you’d like to see another media type added to this list, please add a comment to this post.

application/vnd.restbucks+xml

This is the representation used in the book:

<order xmlns="http://schemas.restbucks.com/" 
    xmlns:dap="http://schemas.restbucks.com/dap">
  <dap:link mediaType="application/vnd.restbucks.com"
      uri="http://restbucks.com/order/1234"
      rel"http://relations.restbucks.com/cancel"/>
  <dap:link mediaType="application/vnd.restbucks.com"
      uri="http://restbucks.com/payment/1234"
      rel"http://relations.restbucks.com/payment"/>
  <dap:link mediaType="application/vnd.restbucks.com"
      uri="http://restbucks.com/order/1234"
      rel"http://relations.restbucks.com/update"/>
  <dap:link mediaType="application/vnd.restbucks.com"
      uri="http://restbucks.com/order/1234"
      rel"self"/>
  <item>
    <milk>semi</milk>
    <size>large</size>
    <drink>cappuccino</drink>
  </item>
  <location>takeAway</location>
  <cost>2.0</cost>
  <status>unpaid</status
</order>

This is an example of a domain-specific media type, so I will not be able to re-use existing client libraries to parse it. Since this format is based on XML, I can use an existing XML parser, but it won’t be able to recognize links in this representation.

This media type doesn’t tell me in the message what HTTP methods to use to dereference the link URIs. So the client has to be programmed with the knowledge that it has to use PUT on the update link, for instance. If the service should ever change that to PATCH, the client will break.

The message does tell me the media type to expect in the response, making it easier for the client to decide whether it should follow a link.

The RESTBucks media type is a fiat standard that isn’t registered with IANA.

application/atom+xml

<entry xmlns="http://www.w3.org/2005/Atom">
  <author>
    <name>RESTBucks</name>
  </author>
  <content type="application/vnd.restbucks+xml">
    <order xmlns="http://schemas.restbucks.com/">
      <item>
        <milk>semi</milk>
        <size>large</size>
        <drink>cappuccino</drink>
      </item>
      <location>takeAway</location>
      <cost>2.0</cost>
      <status>unpaid</status
    </order>
  </content>
  <link type="application/vnd.restbucks.com"
      href="http://restbucks.com/payment/1234"
      rel"http://relations.restbucks.com/payment"/>
  <link type="application/vnd.restbucks.com"
      href="http://restbucks.com/order/1234"
      rel"edit"/>
  <link type="application/vnd.restbucks.com"
      href="http://restbucks.com/order/1234"
      rel"self"/>
</entry>

Atom is a domain-general media type that implements the collection pattern. Since it is not specific to a single service, I can use an Atom library to parse the message and it will be able to find links. It will also know that the edit link will allow me to update and delete the order (using PUT and DELETE respectively).

The Atom library won’t help me with parsing the content, but I can still use an XML library for that. As with the domain-specific media type, I will need to embed knowledge of the HTTP method used for paying into my client.

A disadvantage of this particular domain-general media type is the overhead of things like author that don’t particularly make sense for RESTBucks.

application/xml

<order xmlns="http://schemas.restbucks.com/">
  <link href="http://restbucks.com/order/1234"
      rel"http://relations.restbucks.com/cancel"/>
  <link href="http://restbucks.com/payment/1234"
      rel"http://relations.restbucks.com/payment"/>
  <link href="http://restbucks.com/order/1234"
      rel"http://relations.restbucks.com/update"/>
  <link href="http://restbucks.com/order/1234"
      rel"self"/>
  <item>
    <milk>semi</milk>
    <size>large</size>
    <drink>cappuccino</drink>
  </item>
  <location>takeAway</location>
  <cost>2.0</cost>
  <status>unpaid</status
</order>

XML is a domain-agnostic format, which means it can be used for anything. The downside is that you won’t know anything about the content until you look at it.

Formally, this means I can’t dispatch on the media type alone anymore, but need to look at the namespace inside the message. In practice people dispatch on the media type anyway. This can lead to parsing problems when the expectation isn’t met, e.g. when an error document is sent instead of an order.

Worse, XML isn’t even a hypermedia type, since it doesn’t describe links. There is the XLInk standard for that, but I haven’t seen that used in APIs.

application/json

{ "order": {
  "_links": {
    "http://relations.restbucks.com/cancel": { 
      "href": "http://restbucks.com/order/1234"
    },
    "http://relations.restbucks.com/payment": {
      "href": "http://restbucks.com/payment/1234"
    }
    "http://relations.restbucks.com/update": { 
      "href": "http://restbucks.com/order/1234"
    },
    "self": { 
      "href": "http://restbucks.com/order/1234"
    },
    "item": {
      "milk": "semi",
      "size": "large",
      "drink": "cappuccino"
    }
    "location": "takeAway",
    "cost": 2.0,
    "status": "unpaid"
  }
}

JSON is a another domain-agnostic format without linking capabilities, so the same caveats apply as for XML. Despite these significant drawbacks, this is what high-profile projects like Spring use.

application/hal+json

{ "_links": {
    "self": { 
      "href": "http://restbucks.com/order/1234"
    },
    "curies": [{
      "name": "relations",
      "href": "http://relations.restbucks.com/"
    }],
    "relations:cancel": { 
      "href": "http://restbucks.com/order/1234"
    },
    "relations:payment": {
      "href": "http://restbucks.com/payment/1234"
    }
    "relations:update": { 
      "href": "http://restbucks.com/order/1234"
    }
  },
  "_embedded": {
    "item": [{
      "milk": "semi",
      "size": "large",
      "drink": "cappuccino"
    }]
  },
  "location": "takeAway",
  "cost": 2.0,
  "status": "unpaid"
}

HAL is a another domain-agnostic format, so the same caveats apply as for JSON.

However, HAL is a real hypermedia type, since it standardizes how to find links (using the _links property). In addition, it also defines how to find embedded objects (using the _embedded property).

HAL uses curies to make the representation a bit more compact. This is nice for humans, but another thing to program into your clients.

HAL is currently an Internet-Draft. There is also an XML version of HAL registered.

application/collection+json

{  "collection": {
    "version" : "1.0",
    "href" : "http://restbucks.com/orders",
    "items" : [{
      "href": "http://restbucks.com/order/1234",
      "data": [
        { "name": "item1.milk", "value": "semi" }, 
        { "name": "item1.size", "value": "large" }, 
        { "name": "item1.drink", "value": "cappuccino" }, 
        { "name": "location", "value": "takeAway" }, 
        { "name": "cost", "value": 2.0 }, 
        { "name": "status", "value": "unpaid" }
      ],
      "links" : [{ 
        "rel" : "http://relations.restbucks.com/cancel", 
        "href" : "http://restbucks.com/order/1234"
      }, { 
        "rel" : "http://relations.restbucks.com/payment", 
        "href" : "http://restbucks.com/payment/1234"
      }, { 
        "rel" : "http://relations.restbucks.com/update", 
        "href" : "http://restbucks.com/order/1234"
      }, { 
        "rel" : "self", 
        "href" : "http://restbucks.com/order/1234"
      }]
    }]
  }
}

Collection+JSON (Cj) is the translation of Atom into JSON. Like Atom, it’s a domain-general format for collections.

In Cj, properties are single values only, so I had to cheat and use the item1. prefix to keep the item data in the message. A better solution would probably be to extract the order items into its own collection, but I didn’t want to change the granularity of the messages.

Cj provides a template property that specifies how to add an item to the collection or update an existing one. Unfortunately, that mechanism is specific to collection actions, so we can’t use it to specify how to add a payment, for example. I don’t show the template in this example because it suffers from the same problem as the data property in that it can’t embed objects.

application/vnd.mason+json

{ "@namespaces": {
    "relations": {
      "name": "http://relations.restbucks.com/"
    }
  },
  "item": [{
    "milk": "semi",
    "size": "large",
    "drink": "cappuccino"
  }],
  "location": "takeAway",
  "cost": 2.0,
  "status": "unpaid",
  "@links": {
    "self": { 
      "href": "http://restbucks.com/order/1234"
    }
  },
  "@actions": {
    "relations:cancel": { 
      "href": "http://restbucks.com/order/1234",
      "type": "void",
      "method": "DELETE"
    },
    "relations:payment": {
      "href": "http://restbucks.com/payment/1234",
      "title": "Pay the order",
      "type": "any",
      "method": "PUT",
      "template": {
        "payment": {
          "amount": 2.0,
          "cardholderName": "",
          "cardNumber": "",
          "expiryMonth": "",
          "expiryYear": ""
        }
      }
    },
    "relations:update": { 
      "href": "http://restbucks.com/order/1234",
      "type": "any",
      "method": "PUT",
      "template": {
        "item": [{
          "milk": "semi",
          "size": "large",
          "drink": "cappuccino"
        }],
        "location": "takeAway",
        "cost": 2.0,
        "status": "unpaid"
      }
    }
  }
}

Mason is a superset of HAL. It adds actions and errors (not shown in this example), which turns it into a full hypermedia type (level 3b). Mason also supports curies.

A peculiarity of the actions is the type property, which can be void, json, json-files, or any. I’d expected a media type there.

application/vnd.siren+json

{ "class": [ "order" ],
  "properties": [{
    "location": "takeAway",
    "cost": 2.0,
    "status": "unpaid"
  },
  "entities": [{
    "class": [ "item" ],
    "rel": [ "http://relations.restbucks.com/item" ],
    "properties": {
      "milk": "semi",
      "size": "large",
      "drink": "cappuccino"
    }
  }],
  "actions": [{
    "name": "http://relations.restbucks.com/cancel",
    "title": "Cancel the order",
    "method": "DELETE",
    "href": "http://restbucks.com/order/1234"
  }, {
    "name": "http://relations.restbucks.com/payment",
    "title": "Pay the order",
    "href": "http://restbucks.com/payment/1234",
    "type": "application/vnd.siren+json",
    "method": "PUT",
    "fields": [{
      "name": "amount",
      "type": "number",
      "value": 2.0
    }, {
      "name": "cardholderName",
      "type": "text"
    }, {
      "name": "cardNumber",
      "type": "text"
    }, {
      "name": "expiryMonth",
      "type": "number"
    }, {
      "name": "expiryYear",
      "type": "number"
    }],
  }, {
    "name": "http://relations.restbucks.com/update",
    "href": "http://restbucks.com/order/1234",
    "method": "PUT",
    "type": "application/vnd.siren.json"
  }],
  "links": [{ 
    "rel": "self",
    "href": "http://restbucks.com/order/1234"
  }]
}

Siren is a full hypermedia type that includes the HTTP methods and media types to use. It also allows specifying the application semantics using the class, rel, and name properties. This makes it suitable for level 4 APIs.

Siren specifies the type for fields, so that clients can render appropriate UIs.

application/vnd.uber+json

{ "uber": {
  "version": "1.0",
  "data": [{
    "rel": [ "self" ],
    "url": "http://restbucks.com/order/1234"
  }, {
    "id": "order",
    "data": [{
      "name": "item",
      "data": [{
        "name": "milk",
        "value": "semi"
      }, {
        "name": "size", 
        "value": "large",
      }, {
        "name": "drink",
        "value": "cappuccino"
      }]
    }, {
      "name": "location",
      "value": "takeAway"
    }, {
      "name": "cost",
      "value": 2.0
    }, {
      "name": "status",
      "value": "unpaid"
    }]
  }, {
    "rel": "http://relations.restbucks.com/cancel",
    "url": "http://restbucks.com/payment/1234",
    "action": "remove"
  }, {
    "rel": "http://relations.restbucks.com/payment",
    "url": "http://restbucks.com/payment/1234",
    "sending": "application/vnd.uber+json",
    "action": "replace",
    "data": [{
      "name": "amount",
      "value": 2.0
    }, {
      "name": "cardholderName",
    }, {
      "name": "cardNumber",
    }, {
      "name": "expiryMonth",
    }, {
      "name": "expiryYear",
    }],
  }, {
    "rel": "http://relations.restbucks.com/update",
    "url": "http://restbucks.com/order/1234",
    "action": "replace",
    "sending": "application/vnd.uber.json"
  }]
}

UBER is a full hypermedia format with a lean message structure. It combines in the data property what other media types separate out in e.g. links, actions, and properties, which makes it a bit hard to read for humans.

UBER allows specifying application semantics using the name and rel properties, making it a level 4 capable media type.

There is also an XML variant of UBER.

RESTBucks Evolved

restbucksThe book REST in Practice: Hypermedia and Systems Architecture uses an imaginary StarBucks-like company as its running example.

I think this is a great example, since most people are familiar with the domain.

The design is also simple enough to follow, yet complex enough to be interesting.

Problem Domain

RESTbucks is about ordering and paying for coffee (or tea) and food. Here is the state diagram for the client:
restbucks-states

  1. Create the order
  2. Update the order
  3. Cancel the order
  4. Pay for the order
  5. Wait for the order to be prepared
  6. Take the order

Book Design

The hypermedia design in the book for the service is as follows:

  1. The client POSTs an order to the well-known RESTBucks URI. This returns the order URI in the Location header. The client then GETs the order URI
  2. The client POSTs an updated order to the order URI
  3. The client DELETEs the order URI
  4. The client PUTs a payment to the URI found by looking up a link with relation http://relations.restbucks.com/payment
  5. The client GETs the order URI until the state changes
  6. The client DELETEs the URI found by looking up a link with relation http://relations.restbucks.com/receipt

The book uses the specialized media type application/vnd.restbucks.order+xml for all messages exchanged.

Design Problems

Here are some of the problems that I have with the above approach:

  1. I think the well-known URI for the service (what Mike Amundsen calls the billboard URI) should respond to a GET, so that clients can safely explore it.
    This adds an extra message, but it also makes it possible to expand the service with additional functionality. For instance, when menus are added in a later chapter of the book, a second well-known URI is introduced. With a proper home document-like resource in front of the order service, this could have been limited to a new link relation.
  2. I’d rather use PUT for updating an order, since that is an idempotent method. The book states that the representation returned by GET contains links and argues that this implies that (1) PUT messages should also contain those links and (2) that that would be strange since those links are under control of the server.
    I disagree with both statements. A server doesn’t necessarily have to make the formats for GET and PUT exactly the same. Even if it did, some parts, like the links, could be optional. Furthermore, there is no reason the server couldn’t accept and ignore the links.
  3. The DELETE is fine.
    An alternative is to use PUT with a status of canceled, since we already have a status property anyway. That opens up some more possibilities, like re-instating a canceled order, but also introduces issues like garbage collection.
  4. I don’t think PUT is the correct method. Can the service really guarantee under all circumstances that my credit card won’t get charged twice if I repeat the payment?
    More importantly, this design assumes that payments are always for the whole order. That may seem logical at first, but once the book introduces vouchers that logic evaporates. If I have a voucher for a free coffee, then I still have to pay for anything to eat or for a second coffee.
    I’d create a collection of payments that the client should POST to. I’d also use the standard payment link relation defined in RFC 5988.
  5. This is fine.
  6. This makes no sense to me: taking the order is not the same as deleting the receipt. I need the receipt when I’m on a business trip, so I can get reimbursed!
    I’d rather PUT a new order with status taken.

Service Evolution

Suppose you’ve implemented your own service based on the design in the book.

evolutionFurther suppose that after reading the above, you want to change your service.

How can you do that without breaking any clients that may be out there?

After all, isn’t that what proponents tout as one of the advantages of a RESTful approach?

Well, yes and no. The media type defined in the book is at level 3a, and so will allow you to change URIs. However, the use of HTTP methods is defined out-of-band and you can’t easily change that.

Now imagine that the book would have used a full hypermedia type (level 3b) instead. In that case, the HTTP method used would be part of the message. The client would have to discover it, and the server could easily change it without fear of breaking clients.

Of course, this comes at the cost of having to build more logic into the client. That’s why I think it’s a good idea to use an existing full hypermedia type like Mason, Siren, or UBER. Such generic media types are much more likely to come with libraries that will handle this sort of work for the client.