How To Design a REST API

rest-easyThere is a lot of interest in REST APIs these days. Unfortunately, most APIs I see are not very mature.

In this post I’d like to share my approach to designing REST APIs:

  1. Understand the problem domain and application requirements and document them as a state diagram
  2. Discover the resources from the transitions
  3. Name the resources with URIs
  4. Select one or more media types to serialize the various representations identified in the resource model
  5. Assign link relations to each of the transitions
  6. Add documentation as required

Note that this is a variation of the design process described in RESTful Web Services. Now, let’s look at each of the steps in detail.

Step 1: Understand the problem domain and application requirements and document them as a state diagram

Without understanding the domain, it’s impossible to come up with a good design for anything.

We want to capture the domain in a way that makes sense for REST APIs. Since the purpose in life for a REST API is to be consumed by REST clients, it makes sense to document the application domain from the point of view of a REST client.

A REST client starts at some well-known URI (the billboard URI, or the URI of the home resource) and then follows links until its goal is met:
restClientFlow
In other words, a REST client starts at some initial state, and then transitions to other states by following links (i.e. executing HTTP methods against URIs) that are discovered from the previously returned representation.

A natural way to capture this information is in the form of a state diagram. For bonus points, we can turn the requirements into executable specifications using BDD techniques and derive the state diagram from the BDD scenarios.

For each scenario, we should specify the happy path, any applicable alternative paths and also the sad paths (edge, error, and abuse cases). We can do this iteratively, starting with only the happy path and then adding progressively more detail based on alternative and sad paths.

We can first collect all scenarios and build the entire state diagram from there. Alternatively, we can start with a select few scenarios, work through the design process, and then repeat everything with new scenarios.

In other words, we can do waterfall-like Big Analysis/Design Up Front, or work through feature by feature in a more Agile manner.

Either way, we document the requirements using a state diagram and work from there.

Step 2: Discover the resources from the transitions

You can build up the resource model piece-by-piece:
transitions

  1. Start with the initial state
  2. Create (or re-use) a resource with a representation that corresponds to this state
  3. For each transition starting from the current state, make sure there is a corresponding method in some resource that implements the transition
  4. Repeat for all transitions in each of the remaining states

Step 3: Name the resources with URIs

Every resource should be identified by a URI. From the client’s perspective, this is an implementation detail, but we still need to do this before we can implement the server.

We should follow best practices for URIs, like keeping them cool.

Step 4: Select one or more media types to serialize the various representations identified in the resource model

mediaWhen extending an existing design, you should stick with the already selected media types.

For new APIs, we should choose a mature format, like Siren or Mason.

There could be specific circumstances where these are not good choices. In that case, carefully select an alternative.

Step 5: Assign link relations to each of the transitions

A REST client follows transitions in the state diagram by discovering links in representations. This discovery process is made possible by link relations.

Link relations decouple the client from the URIs that the server uses, giving the server the freedom to change its URI structure at will without breaking any clients. Link relations are therefore an important part of any REST API.

We should try to use existing link relations as much as possible. They don’t cover every case, however, so sometimes you need to invent your own.

Step 6: Add documentation as required

learn moreIn order to help developers build clients that work against your API, you will most likely want to add some documentation that explains certain more subtle points.

Examples are very helpful to illustrate those points.

You may also add instructions for server developers that will implement the API, like what caching to use.

Conclusion

I’ve successfully used this approach on a number of APIs. Next time, I’ll show you with an example how the above process is actually very easy with the right support.

In the meantime, I’d love to hear how you approach REST API design. Please leave a comment below.

REST Maturity

rest-maturity2In 2008, Leonard Richardson published his Maturity Heuristic that classified web services into three levels based on their use of URI, HTTP, and hypermedia.

Back then, most web services were stuck at either level 1 or 2. Unfortunately, not a whole lot has improved since then in that respect: so-called pragmatic REST is still the norm.

BTW, I really dislike the term “pragmatic REST”. It’s a cheap rhetoric trick to put opponents (“dogmatists”) on the defensive.

More importantly, it creates semantic diffusion: pragmatic REST is not actually REST according to the definition, so please don’t call it that way or else we’re going to have a hard time understanding each other. The term REST hardly means anything anymore these days.

Anyway, there is some light at the end of the tunnel: more services are now at level 3, where they serve hypermedia. A good example by a big name is Amazon’s AppStream API.

The difference between plain media types, like image/jpeg, and hypermedia types, like text/html, is of course the “hyper” part. Links allow a client to discover functionality without being coupled to the server’s URI structure.

JSONBTW, application/json is not a hypermedia type, since JSON doesn’t define links.

We can, of course, use a convention on top of JSON, for instance that there should be a links property with a certain structure to describes the links, like Spring HATEOAS does.

The problem with conventions is that they are out-of-band communication, and a client has no way of knowing for sure whether that convention is followed when it sees a Content-Type of application/json. It’s therefore much better to use a media type that turns the convention into a rule, like HAL does.

Speaking of out-of-band communication, the amount of it steadily decreases as we move up the levels. This is a very good thing, as it reduces the amount of coupling between clients and servers.

Level 3 isn’t really the end station, however. Even with a hypermedia format like HAL there is still a lot of out-of-band communication.

HALHAL doesn’t tell you which HTTP method to use on a particular link, for instance.

The client can only know because a human has programmed it with that knowledge, based on some human-readable description that was published somewhere.

Imagine that the human Web would work this way. We wouldn’t be able to use the same browser to shop at Amazon and read up at Wikipedia and do all those other things we take for granted. Instead, we would need an Amazon Browser, a Wikipedia Browser, etc. This is what we do with APIs today!

Moving further into the direction of less out-of-band communication requires more than just links. Links only specify the URI part and we also need the HTTP and media type parts inside our representations. We might call this level 3b, Full Hypermedia.

Siren gives you this. Uber even goes a step further and also abstracts the protocol, so that you can use it with, say, CoAP rather than HTTP.

These newer hypermedia types allow for the use of a generic client that can handle any REST API that serves that hypermedia type, just like a web browser can be used against anything that serves HTML. An example of such an effort is the HAL browser (even though HAL is stuck at level 3a).

However, even with the inclusion of protocol, media type, and method in the representation, we still need some out-of-band communication.

The HAL browser can navigate any API that serves HAL, but it doesn’t understand the responses it gets. Therefore it can’t navigate links on its own to reach a certain goal. For true machine-to-machine (M2M) communication, we still need more.

ALPSIf we ever get the whole semantic web sorted out, this might one day be the final answer, but I’m not holding my breath.

In the meantime we’ll have to settle for partial answers.

One piece of the puzzle could be to define application semantics using profiles, for instance in the ALPS format. We might call this level 4, Semantic Profile.

We’d still need a human to read out-of-band communication and build a special-purpose client for M2M scenarios. But this client could handle all services in the application domain it is programmed to understand, not just one.

Also, the human could be helped a lot by a generic API browser that fetches ALPS profiles to explain the API.

All this is currently far from a reality. But we can all work towards this vision by choosing generic, full-featured hypermedia types like Siren or Uber for our APIs and by documenting our application semantics using profiles in ALPS.

If you need more convincing then please read RESTful Web APIs, which Leonard Richardson co-wrote with Uber and ALPS creator Mike Amundsen. This is easily the best book on REST on the market today.

REST 101 For Developers

rest-easy

Local Code Execution

Functions in high-level languages like C are compiled into procedures in assembly. They add a level of indirection that frees us from having to think about memory addresses.

Methods and polymorphism in object-oriented languages like Java add another level of indirection that frees us from having to think about the specific variant of a set of similar functions.

Despite these indirections, methods are basically still procedure calls, telling the computer to switch execution flow from one memory location to another. All of this happens in the same process running on the same computer.

Remote Code Execution

This is fundamentally different from switching execution to another process or another computer. Especially the latter is very different, as the other computer may not even have the same operating system through which programs access memory.

It is therefore no surprise that mechanisms of remote code execution that try to hide this difference as much as possible, like RMI or SOAP, have largely failed. Such technologies employ what is known as Remote Procedure Calls (RPCs).

rpcOne reason we must distinguish between local and remote procedure calls is that RPCs are a lot slower.

For most practical applications, this changes the nature of the calls you make: you’ll want to make less remote calls that are more coarsely grained.

Another reason is more organizational than technical in nature.

When the code you’re calling lives in another process on another computer, chances are that the other process is written and deployed by someone else. For the two pieces of code to cooperate well, some form of coordination is required. That’s the price we pay for coupling.

Coordinating Change With Interfaces

We can also see this problem in a single process, for instance when code is deployed in different jar files. If you upgrade a third party jar file that your code depends on, you may need to change your code to keep everything working.

Such coordination is annoying. It would be much nicer if we could simply deploy the latest security patch of that jar without having to worry about breaking our code. Fortunately, we can if we’re careful.

interfaceInterfaces in languages like Java separate the public and private parts of code.

The public part is what clients depend on, so you must evolve interfaces in careful ways to avoid breaking clients.

The private part, in contrast, can be changed at will.

From Interfaces to Services

In OSGi, interfaces are the basis for what are called micro-services. By publishing services in a registry, we can remove the need for clients to know what object implements a given interface. In other words, clients can discover the identity of the object that provides the service. The service registry becomes our entry point for accessing functionality.

There is a reason these interfaces are referred to as micro-services: they are miniature versions of the services that make up a Service Oriented Architecture (SOA).

A straightforward extrapolation of micro-services to “SOA services” leads to RPC-style implementations, for instance with SOAP. However, we’ve established earlier that RPCs are not the best way to invoke remote code.

Enter REST.

RESTful Services

rest-easyRepresentational State Transfer (REST) is an architectural style that brings the advantages of the Web to the world of programs.

There is no denying the scalability of the Web, so this is an interesting angle.

Instead of explaining REST as it’s usually done by exploring its architectural constraints, let’s compare it to micro-services.

A well-designed RESTful service has a single entry point, like the micro-services registry. This entry point may take the form of a home resource.

We access the home resource like any other resource: through a representation. A representation is a series of bytes that we need to interpret. The rules for this interpretation are given by the media type.

Most RESTful services these days serve representations based on JSON or XML. The media type of a resource compares to the interface of an object.

Some interfaces contain methods that give us access to other interfaces. Similarly, a representation of a resource may contain hyperlinks to other resources.

Code-Based vs Data-Based Services

soapThe difference between REST and SOAP is now becoming apparent.

In SOAP, like in micro-services, the interface is made up of methods. In other words, it’s code based.

In REST, on the other hand, the interface is made up of code and data. We’ve already seen the data: the representation described by the media type. The code is the uniform interface, which means that it’s the same (uniform) for all resources.

In practice, the uniform interface consists of the HTTP methods GET, POST, PUT, and DELETE.

Since the uniform interface is fixed for all resources, the real juice in any RESTful service is not in the code, but in the data: the media type.

Just as there are rules for evolving a Java interface, there are rules for evolving a media type, for example for XML-based media types. (From this it follows that you can’t use XML Schema validation for XML-based media types.)

Uniform Resource Identifiers

So far I haven’t mentioned Uniform Resource Identifiers (URIs). The documentation of many so-called RESTful services may give you the impression that they are important.

identityHowever, since URIs identify resources, their equivalent in micro-services are the identities of the objects implementing the interfaces.

Hopefully this shows that clients shouldn’t care about URIs. Only the URI of the home resource is important.

The representation of the home resource contains links to other resources. The meaning of those links is indicated by link relations.

Through its understanding of link relations, a client can decide which links it wants to follow and discover their URIs from the representation.

Versions of Services

evolutionAs much as possible, we should follow the rules for evolving media types and not introduce any breaking changes.

However, sometimes that might be unavoidable. We should then create a new version of the service.

Since URIs are not part of the public interface of a RESTful API, they are not the right vehicle for relaying version information. The correct way to indicate major (i.e. non-compatible) versions of an API can be derived by comparison with micro-services.

Whenever a service introduces a breaking change, it should change its interface. In a RESTful API, this means changing the media type. The client can then use content negotiation to request a media type it understands.

What Do You Think?

what-do-you-thinkLiterature explaining how to design and document code-based interfaces is readily available.

This is not the case for data-based interfaces like media types.

With RESTful services becoming ever more popular, that is a gap that needs filling. I’ll get back to this topic in the future.

How do you design your services? How do you document them? Please share your ideas in the comments.