Chapter 8. Building RESTful APIs

What is an API? If you're reading this book, you know that it's an application programming interface, but in the context of a RESTful API, we're specifically talking about an interface that allows users and applications to interact with your application over HTTP using REST architecture (REpresentational State Transfer).

For a very basic overview of REST architecture, see “RESTful JSON Web Services”. You'll have a chance to explore it in-depth later in this chapter.

Why should you build an API? First of all, you're going to need a way to connect the user interface to your backend resources, like a database, logging services, and maybe some gateways to third-party APIs. But there are good reasons to expose parts of that API to the public as well.

Think carefully about the some of the most successful apps out there: Facebook, Twitter, Google Maps, Amazon (yeah, it's an app—lots of them, actually). I can keep going, but I'd fill the rest of the book with examples. What do all of these have in common?

They all offer public APIs that developers can use to enhance the value of the platform for its users. With a public API, your application is more than an app: it's a platform that other developers will flock to and integrate their apps with. They'll expose your app to new users and make your app more attractive to your existing users, because now it works with other tools that they already use.

But you don't want just any API. You want a beautifully designed API. As Brandon Satrom said at Fluent 2013, "API design is developer UX". It's important to be friendly to developers—yourself included.

Your API must be easy to learn, easy to use, easy to explore, and fast.

All of that starts with design. Design should be planned. I don't mean that you should be able to predict the future—only that you should build your API such that it can change over time in ways that are beautiful, not ugly.

So what makes an API beautiful?

Beautiful APIs are:

Usable

Deliver useful services without confusing users.

Self-describing

Don't make your API users read the manual; embed the manual in the API.

Efficient

Bandwidth is expensive. Don't send more data than the user needs.

Responsive

Make your API respond as quickly as it can.

As you can see, a beautiful API satisfies all of the goals you saw a few paragraphs back:

Usable

There are two foundational principles of API design that can help improve the usability of your API and reduce the time it takes for new developers to learn it:

Focus

The purpose of your API and each API endpoint should be clear, just by interacting with the API.

Consistency

Each API resource should share the same conventions as the other resources for the same API.

Focus

To focus your API, present only the information your users need and eliminate clutter. If your application is for music lovers, don't provide links to cooking recipes or abstract art.

It sounds easy, but it goes much deeper than that, and a lack of focus plagues many APIs. For example, many endpoints will list nested objects—resources with related data embedded directly in the response. That practice can be convenient, but it can also be confusing. Don't overwhelm API users with gigantic records when they may be looking for a small, specific subset.

For example, consider an API that delivers information about music albums:

  {
    "id": "chmzq50np0002gfixtr1qp64o",
    "name": "Settle",
    "artist": {
      "id": "chmzq4l480001gfixe8a3nzhm",
      "name": "Disclosure",
      "tourDates": [
        {
          "id": "chmzq45yn0000gfixa0wj6z9j",
          "date": "Sat Oct 19 2013",
          "name": "Treasure Island Music Festival",
          "location": "San Francisco, CA, US",
          "attending": [
            {
              "id": "chmzq7tyj0003gfix0rcylkls",
              "name": "Guy Lawrence"
            },
            {
              "id": "chmzqougy0004gfixuk66lhv4",
              "name": "Howard Lawrence"
            }
          ]
        }
      ]
    },
    "...": "..."
  }

If all the user is looking for is a list of album titles and artist names, this is going to be overload. Instead of this, you can focus on the particular information that your users are most likely to need. Of course you should make it easy for them to get more if they want it, too. Just make the default sensible.

Take a look at this version and see if it makes more sense:

  {
    "id": "chmzq50np0002gfixtr1qp64o",
    "name": "Settle",
    "artist": "Disclosure",
    "artistId": "chmzq4l480001gfixe8a3nzhm",
    "coverImage": "/covers/medium/zrms5gxr.jpg",
    "year": "2013",
    "genres": [
      "electronic", "house", "garage", "UK garage",
      "future garage"
    ]
  }

Instead of inlining a bunch of completely unrelated data about the artist, you get the artist name (which you'll likely need to display in any album listing) and an ID you can use to query more information about the artist if it's needed.

The listing now is focused on the particular resource that's being queried right now: the album. Focus may be the most important principle of good UX, whether you're designing the UI for a shopping cart or the shopping cart's API.

Consistency

Consistency is all about letting users build on prior knowledge while they learn how to use your service. Once they learn how to complete one task, they should be able to apply that learning to the next task. You can make your learning curve a lot easier to climb by making your API as consistent as possible.

There are a couple of great ways to do that with a RESTful API:

  • Use standard REST conventions.

  • Adopt a style for your API endpoints, and stick to that same style for every endpoint (including errors).

Use common conventions

A lot of consistency is built into the REST architecture. Sticking to the conventions of REST can go a long way toward making your API more consistent, both internally and with other APIs that the users might be familiar with.

A defining characteristic is that REST does not deal with remote procedure calls (RPC). Instead, it is only concerned with the transfer of state. The difference is verbs versus nouns. When you call a procedure, you're asking the API to do something. When you transfer state, you're sharing data about something. RPC is control oriented, while REST is data oriented.

The advantage of a data-oriented architecture style is that it can map to any domain without embedding knowledge of the particular domain in the client. In other words, everything you need to know to interact successfully with a RESTful API should be contained in:

  1. The protocol(s) (HTTP URIs, GET, POST, etc.)

  2. The media type (rules for interpreting messages)

  3. The messages from the server (hyperlinks, templates, etc.)

The goal of REST is to avoid coupling the client application and the server application. In fact, successful REST clients should be able to browse completely unrelated RESTful APIs. The most obvious example is the web browser. It doesn't know anything about a website until you browse to it and start to explore the hyperlinks contained in the requested document. An example of a generic JSON hypermedia browsing client is Jsonary.

That consistency is useful for more than usability. It's also great for application maintainability, because it decouples your client application from your server application such that changes to your API can be published without breaking existing clients.

One requirement of REST is that you respect the methods defined by the protocol. With few exceptions (such as the /log.gif tracking pixel hack covered in “Logging Requests”), the HTTP methods should be respected by your API. Don't use GET when you really mean POST or PUT. Exceptions should be practical workarounds for problems such as holes in specifications or implementations, and those workarounds should be made with the expectation that they will eventually be obsolete.

For example, HTML forms only support GET and POST methods, so if you want to allow users to access other features of your API without using JavaScript, you'll need to make some practical concessions.

Here’s another example: many users are still using IE 8, and most of us still need to support it. IE 8 supports PATCH, but only via the proprietary ActiveXObject for XMLHTTP.

Method override works by passing an X-HTTP-Method-Override header (a common convention), or an optional _method key set to the method you wish to emulate in your request. So, if you want to PUT a record using an HTML form, simply POST the form with _methdod parameter set to PUT. If you want to use PATCH, you can set the X-HTTP-Method-Override and POST the contents.

Express can automatically check for a method override setting and route the request to the appropriate method handler. To make that happen, use the connect methodOverride middleware:

  // You need to parse the body to get the method param:
  app.use( express.json() );
  app.use( express.urlencoded() );

  // Now methodOverride() will work:
  app.use( express.methodOverride() );

The really important thing to get right is that you honor the correct methods for clients that support them. That way, modern clients can consistently do things the right way, while methodOverride() provides a fallback for legacy clients.

Another comment about methods: If your server supports any methods for a given resource, it should deliver a 405 Method Not Allowed error as opposed to a 404 Not Found error for all the other methods. If your API and your documentation says that an endpoint is there, but the user gets a 404 Not Found error, that's confusing. Luckily, Express makes this easy:

  app.get( '/albums', albumHandler() );
  app.all('/albums', function (req, res, next) {
    var err = new Error();
    err.route = '/albums';
    err.status = 405;
    next(err);
  });

If you're using an error-handler middleware that will catch all your routing errors and deliver appropriate error messages (see “Logging Errors”), you're in business. Otherwise, you could do this instead (not recommended):

  app.get( '/albums', albumHandler() );
  app.all('/albums', function (req, res, next) {
    res.send(405);
  });

The disadvantage of the latter approach is that if you later want to add support for custom JSON responses for errors to add helpful links to working methods, you'll have to find every route endpoint that did its own error handling and update them one at a time.

Another serious problem with both of the previous examples is that they return a Content-Type: text/plain header. Since you're building an API that works on the application/json media type, that inconsistency will be confusing to your users.

If you use the express-error-handler module in discussed the section on error handling, there's a lot less to remember:

  app.get( '/albums', albumHandler() );
  app.all( '/albums', errorHandler.httpError(405) );

This will deliver the 405 message in application/json.

In the context of a simple albums service, complete with proper error handling and logging, it might look something like this:

  'use strict';

  var express = require('express'),
    http = require('http'),
    logger = require('bunyan-request-logger'),
    errorHandler = require('express-error-handler'),
    app = express(),
    log = logger(),
    server,
    port = 3000;

  app.use( express.json() );
  app.use( express.urlencoded() );
  app.use( express.methodOverride() );
  app.use( log.requestLogger() );

  // Respond to get requests on /albums
  app.get('/albums', function (req, res) {
    res.send({
      chmzq50np0002gfixtr1qp64o: {
        "id": "chmzq50np0002gfixtr1qp64o",
        "name": "Settle",
        "artist": "Disclosure",
        "artistId": "chmzq4l480001gfixe8a3nzhm",
        "coverImage": "/covers/medium/zrms5gxr.jpg",
        "year": "2013",
        "genres": [
          "electronic", "house", "garage", "UK garage",
          "future garage"
        ]
      }
    });
  });

  // Deliver 405 errors if the request method isn't
  // defined.
  app.all( '/albums', errorHandler.httpError(405) );


  // Deliver 404 errors for any unhandled routes.
  // Express has a 404 handler built-in, but it
  // won't deliver errors consistent with your API.
  app.all( '*', errorHandler.httpError(404) );

  // Log errors.
  app.use( log.errorLogger() );

  // Create the server
  server = http.createServer(app);

  // Handle errors. Remember to pass the server
  // object into the error handler, so it can be
  // shut down gracefully in the event of an
  // unhandled error.
  app.use( errorHandler({
    server: server
  }) );

  server.listen(port, function () {
    console.log('Listening on port ' + port);
  });

Resourceful routing

Imagine we're talking about the process of adding new album cover art to an albums collection, and the UI exposes a way for you to select an album cover via HTTP. The server goes and fetches the image from the URL, encodes it, and adds the resized/encoded image to the album record. Instead of exposing an /encode-image endpoint, you could PUT to the /albums endpoint:

PUT /albums/123:

{
  "coverSourceURL":
    "http://cdn2.pitchfork.com/news/50535/f40d167d.jpg",
  "id": "chmzq50np0002gfixtr1qp64o",
  "name": "Settle",
  "artist": "Disclosure",
  "artistId": "chmzq4l480001gfixe8a3nzhm",
  "year": "2013",
  "genres": [
    "electronic", "house", "garage", "UK garage",
    "future garage"
  ]
}

The route can check for the existence of the coverSourceURL, fetch it, process the image, and replace the key with:

"coverImage": "/covers/medium/zrms5gxr.jpg"

Resources describe your data, represented by nouns, and HTTP provides the verbs to manipulate the nouns. You might remember the basic set of manipulations from Chapter 1:

  • Create a new entry in the resource collection: HTTP POST.

  • Retrieve a resource representation: HTTP GET verb.

  • Update (replace) the resource: HTTP PUT.

  • Delete a resource: HTTP DELETE.

That leaves out PATCH, which has been supported in most major browsers since IE 7, though you'll need the methodOverride() hack mentioned before.

CRUD has long been a convenient way to deal with records, but maybe it has outlived its usefulness as the default way to manipulate state.

It's a really simplistic way of viewing the REST architecture, and perhaps not the best way. For instance, PUT is often used for resource creation.

The major advantage of PUT is that it's idempotent. If you PUT the same resource twice, it doesn't change anything. The classic example is a shopping cart order. If you POST a shopping cart order, and POST again, it will trigger two different checkouts. That's probably not what the user wants. That's why you'll see warning messages if you POST something and then hit the back button. Browsers will warn that the data will be posted again. If you're building the server implementation, it's your job to enforce idempotency for PUT operations.

PUT can also help you build applications that are capable of functioning offline. To do that, you need to be able to create a complete record on the client side, without the need to connect to the server first to retrieve the ID for a newly created record. Using PUT for resource creation can also impact the perceived performance of the app. If users don't have to wait for a spinner every time they create something, they're going to have a much smoother experience.

Of course, you'll still want to validate all the data that the client eventually sends. Just do it as soon as you can, and deal with the errors as the app becomes aware of them. Being able to gracefully resolve data conflicts is becoming a necessary part of application design as realtime collaboration features find their way into a growing number of apps.

A resource is just a collection of related data, such as the /albums endpoint mentioned before. Routing for the resource could be mapped as follows (opinions differ on the correct mapping):

GET     /albums           ->  index
POST    /albums           ->  create, return URI
GET     /albums/:id       ->  show
PUT     /albums/:id       ->  create or update
DELETE  /albums/:id       ->  destroy

Start by making the index route behave more like an index:

// GET     /albums    ->  index
app.get('/albums', function (req, res) {
  var index = map(albums, function (album) {
    return {
      href: '/albums/' + album.id,
      properties: {
        name: album.name,
        artist: album.artist
      }
    };
  });
  res.send(index);
});

That slims it down a little:

{
  "chmzq50np0002gfixtr1qp64o": {
    "href": "/albums/chmzq50np0002gfixtr1qp64o",
    "properties": {
      "name": "Settle",
      "artist": "Disclosure"
    }
  }
}

Support POST:

// POST    /albums    ->  create, return URI
app.post('/albums', function (req, res) {
  var id = cuid(),
    album = mixIn({}, req.body, {
      id: id
    });

  albums[id] = album;
  res.send(201, {
    href: '/albums/' + id
  });
});

Deliver helpful messages for the /albums index:

// Send available options on OPTIONS requests
app.options( '/albums', function (req, res) {
  res.send(['GET', 'POST', 'OPTIONS']);
});

// Deliver 405 errors if the request method isn't
// defined.
app.all( '/albums', errorHandler.httpError(405) );

Allow users to get more detail for a particular album:

// GET     /albums/:id    ->  show
app.get('/albums/:id', function (req, res, next) {
  var id = req.params.id,
    body = albums[id],
    err;

  if (!body) {
    err = new Error('Album not found');
    err.status = 404;
    return next(err);
  }

  res.send(200, body);
});

Allow users to PUT complete albums with a client-generated ID:

// PUT     /albums/:id    ->  create or update
app.put('/albums/:id', function (req, res) {
  var album = mixIn({}, req.body),
    id = req.params.id,
    exists = albums[id];

  album.id = id;
  albums[id] = album;

  if (exists) {
    return res.send(204);
  }

  res.send(201, {
    href: '/albums/' + album.id
  });
});

Your users need a way to delete albums:

// DELETE  /albums/:id    ->  destroy
app.delete('/albums/:id',
    function (req, res, next) {
  var id = req.params.id,
    body = albums[id],
    err;

  if (!body) {
    err = new Error('Album not found');
    err.status = 404;
    return next(err);
  }

  delete albums[id];

  res.send(204);
});

The /albums/:id endpoint needs its own helpful errors:

  // Send available options on OPTIONS requests
  app.options( '/albums', function (req, res) {
    res.send(['GET', 'PUT', 'DELETE', 'OPTIONS']);
  });

  // 405 Method Not Allowed
  app.all( '/albums/:id', errorHandler.httpError(405) );

That should get you started. A lot of people think that if they've got this far, they've created a RESTful API. It's RESTish, but remember that list of goals from before:

  • Easy to learn: usable, self-describing

  • Easy to use: usable, self-describing, efficient

  • Easy to explore: self-describing, usable, responsive

  • Fast: efficient, responsive

It's RESTish and resourceful, but there's one more important step you need to take to satisfy those goals: it needs to be self-describing.

Self-Describing: Hypermedia

Hypertext is a text-based resource with embedded references to other text resources. Hypermedia applies the concept to rich media resources, such as images, video, and audio. The Web started out as a system built on hypertext resources, and has since evolved into a system built on hypermedia resources.

Affordances

 

When I say Hypertext, I mean the simultaneous presentation of information and controls such that the information becomes the affordance through which the user obtains choices and selects actions.

 
  --Roy T. Fielding

Affordances are all of the possible actions you can perform on a resource. In the context of hypermedia, an affordance might be a link for clicking or a form to manipulate the resource. The buttons on your keyboard afford pressing. The mouse affords cursor movement, and so on.

Roy Fielding envisioned an architectural style whereby all affordances for an API are delivered by means of hypermedia.

Mike Amundsen took that a step further and specified nine different affordances that document aspects of state transitions over the network (described in much more detail in his book, Building Hypermedia APIs with HTML5 and Node [O’Reilly, 2011]). He calls them H-Factors and breaks them down into two categories:

  1. Control data support:

    [CR] Modify control data for read requests

    For example, HTTP Accept-* headers

    [CU] Modify control data for update requests

    For example, HTTP Content-* headers

    [CM] Modify control data for interface methods

    For example, select between POST and PUT

    [CL] Modify control data for links

    Add semantic meaning to links (relations); for example, rel attribute

HTML actually has a couple other important affordances worth mentioning that are not members of the aforementioned transition-focused H-Factors:

Specify presentation rules

For example, CSS

Specify behavior

Code on demand (see Roy Fielding's famous REST Dissertation, "Architectural Styles and the Design of Network-based Software Architectures"); for example, JavaScript

The more affordances a media type provides, the closer you can get to HATEOAS.

HATEOAS

HATEOAS (Hypermedia As The Engine Of Application State) is an important but often overlooked method of improving an API. Essentially, it means that your server gives the client the information it needs to interact with your API so that the client doesn't have to remember out-of-band information, like endpoint URLs, data formats, available features, etc.:

 

All application state transitions must be driven by client selection of server-provided choices that are present in the received representations or implied by the user’s manipulation of those representations [...] Failure here implies that out-of-band information is driving interaction instead of hypertext.

 
  --From the blog post "REST APIs must be Hypertext Driven” by Roy T. Fielding

The idea is to decouple the client from the server so that they are literally two completely independent applications. There are several advantages:

  • Your API is more browsable and self-documenting.

  • You can upgrade or make changes to your API at any time without breaking clients, and all clients will receive the changes automatically.

  • Reduce long-term maintenance costs. The more clients assume about your API, the more logic you'll need to support changes you make to the API.

  • Clients are more adaptable to temporary changes; for example, if a host becomes unreachable, you can update the URI in your hyperlinks.

  • You can use the presence or lack of links in place of a traditional feature toggle system. In other words, all client features exist because the RESTful service informs the client that the features exist.

A good rule of thumb to achieve this level of affordance in your API is to write a client test implementation in parallel with your API design, and see if your client can do everything it needs to do following one simple rule:

Code to the media type, not to the message.

What does that mean? It means that your client should be aware of the rules for interpreting the messages, but not aware of the contents of the messages themselves. If you can successfully build your app with that restriction, you can achieve a high degree of loose coupling, which will make your client very responsive to API changes.

The primary disadvantages are:

  • Developer education and compliance. Once you deliver a link to clients, it's hard to prevent them from embedding that link in their applications.

  • Efficiency. If you have to make a call to get a link to make another call, you pay for two server round trips instead of one.

The first issue can be managed by writing good SDKs for resource consumers. If the SDKs wrap your API and comply with the best practices of the API, and consumers have incentive to use the SDK (because presumably, it makes working with your API easier), that can go a long way.

The second can be managed by making affordances in your API that allow clients to discover resources efficiently and cache URLs on the client side so that they don't have to look them up every time they need access to a particular resource. That process can also be aided with a good SDK.

One of the great advantages of a truly HATEOAS API is that a single SDK could be used to power any number of different API services across an organization. It should even be possible to create a client SDK that works for any system that happens to rely on the same hypermedia type, provided that the target APIs make enough affordances.

HTML as an API Media Type

Recently, more adventurous API designers have been experimenting with using HTML instead of JSON-based media types to communicate with their APIs. The thinking is that HTML powers the biggest success case of the REST architectural style and already supports many features that are missing from many other hypermedia types, including the ability to include JavaScript and style the output so that it's easier for humans to view.

An interesting perspective on this topic is that the website and the API could literally be the same thing, as long as you're careful to embed enough affordances in the HTML to serve the needs of your developers.

This isn't a terrible idea, but I think that HTML is a little too verbose for this purpose. HTML also lacks a couple of key affordances, as you'll see in a minute. There is a possible alternative.

Jade

Jade was designed as a minimal template markup that you can use to generate HTML. It offers a lighter weight, more convenient syntax, while retaining the expressive power of HTML.

Here's how you might represent the /albums collection (minus some navigation links, for brevity):

  head
    title Order
  body.order
    h1 Order
    ul.properties
      li.property
        label Order Number
          span.orderNumber 42
      li.property
        label Item Count
          span.itemCount 3
      li.property
        label Status
          span.status pending

    ul.entities
      li.entity.items.collection
        a(rel='http://x.io/rels/order-items',
          href='http://api.x.io/orders/42/items')
          | Items

      li.entity.info.customer
        a(rel='http://x.io/rels/customer'
          href='http://api.x.io/customers/pj123'),
          ul.properties
            li.property
              label Customer ID
                span.customerId pj123
            li.property
              label Name
                span.name Peter Joseph

    ul.actions
      li.action
        // Action is one of:
        // index, create, show, put, delete, patch
        // The method in html is determined by the
        // mapping between actions and HTML methods.
        form(action='create',
          href='http://api.x.io/orders/42/items',
          type='application/x-www-form-urlencoded')
          fieldset
            legend Add Item
            label Order Number
              input(name='orderNumber', hidden='hidden', value='42')
            label Product Code
              input(name='productCode', type='text')
            label Quantity
              input(name='quantity', type='number')

    ul.links
      a(rel='self', href='http://api.x.io/orders/42',
        style='display: none;')
      a(rel='previous', href='http://api.x.io/orders/41') Previous
      a(rel='next', href='http://api.x.io/orders/43') Next

Here's the equivalent HTML:

  <head profile="http://ericelliott.me/profiles/resource">
      <title>Albums</title>
  </head>

  <body class="albums">
      <h1 class="title">Albums</h1>

      <ul class="properties">
          <li class="property">
              <p class="description">A list of albums you should listen to.</p>
          </li>

          <li class="property"><!-- A count of the total number of entities-->
          <!-- available. Useful for paging info.-->
          <label for="entityCount">Total count:</label> <span class="entityCount"
          id="entityCount">3</span></li>
      </ul>

      <ul class="entities">
          <li class="entity album">
              <a href="/albums/a65x0qxr" rel="item">
              <ul class="properties">
                  <li class="property name">Dark Side of the Moon</li>

                  <li class="property artist">Pink Floyd</li>
              </ul></a>
          </li>

          <li class="entity album">
              <a href="/albums/a7ff1qxw" rel="item">
              <ul class="properties">
                  <li class="property name">Random Access Memories</li>

                  <li class="property artist">Daft Punk</li>
              </ul></a>
          </li>
      </ul>

      <ul class="links">
          <li class="link">
              <a href="/albums?offset=2&amp;limit=1" rel="next">Next</a>
          </li>

          <li class="link">
              <link href="http://albums.com/albums" rel="self, canonical">
          </li>
      </ul>
  </body>

Jiron

Jiron is a hypermedia type inspired by Siren that extends it with the semantics of HTML. The Jade-only version is missing a couple of important affordances:

  • Idempotent updates [LI], and

  • Control data for reads [CR]

However, Jiron can fix both issues:

Starting with the idempotent updates, imagine the following syntax:

  form(method='put', href="/names/123")
    label(for='name')
    input#name

which maps to the broken HTML:

  <form method="PUT" href="/names/123">
    <label for="name"></label>
    <input id="name"/>
  </form>

You'll need a media type that includes information that allows your client to handle such requests. Imagine that your client is built to handle all requests via Ajax. It intercepts them all, grabs the method parameter, and passes it through to the server without examining it (trusting the server to validate it).

Now, any method supported by the server will be supported by both the media type and the client, automatically.

Great! Now you need a way to ask the server for this new media type in your links. No problem:

  a(headers='Accept:application/vnd.jiron+jade') Some Jiron resource

And the HTML (again, this won't work with vanilla HTML):

  <a headers="Accept:application/vnd.jiron+jade">Some Jiron resource</a>

This allows you to pass arbitrary headers when you click on links. The client will intercept all link activations and serve the links via Ajax.

Take another look at that earlier /albums example. That isn't any ordinary HTML. It contains class and rel attributes that obviously have some semantic meaning. That's because it's actually based on Siren+JSON. It's a mapping of Siren entity semantics on to HTML documents and ul elements.

Here are some things you can do with Jiron that you can't do with JSON:

  • Deliver code on demand with JavaScript links (including any client SDK you need to customize any Jiron browser for your particular API)

  • Deliver default templates that make browsing your API directly in a browser pleasant

  • Deliver clickable links and usable forms while your users are browsing the API directly

  • Use CSS for styling default views

  • Intersperse human-readable text and media with hypertext controls and affordances—like HTML, only structured specifically for APIs

And stuff you can't do with HTML:

  • Support any method type. PUT, PATCH, DELETE? No problem.

  • Support header changes in links.

Since Jiron is based on the existing Jade HTML template syntax, the documents are easy to interpret with existing Jade tools and browser tools. Browserify users can use https://github.com/substack/node-jadeify. You can also use Browserify to export an AMD module or standalone module for the browser if you don't want to use Browserify for module management in the client.

Using it in the browser is simple:

  jade.render('a.album(href="/albums/123") Pretty Hate Machine');

which creates the string:

  <a href="/albums/123" class="album">Pretty Hate Machine</a>

Now you can add that to a documentFragment and use CSS selectors to query it. Better yet, just slap some CSS on it and render it as-is. Try that with JSON.

Even if you use Jiron, you'll still be sending data to the server using URL parameters and JSON bodies. Those formats are universally understood by web servers, and that will save you the trouble of parsing Jiron documents on the server side.

If you want to support the application/vnd.jiron+html type, or Jiron over text/html, just process the Jade template on the server before you send it to the client.

Responsive APIs

Different API consumers have different needs. One way to make your API more valuable is to make it responsive to those different, often conflicting needs. You can enable a more responsive API by looking at request parameters and changing the output based on the user's requests. Here is a list of commonly supported request options:

Accept

Allow users to request alternative content types. At a minimum, make it possible to add them later.

Embed

Retrieve and embed related entities in place:

  # Get all albums, embed artist representations
  # in the response.
  GET /albums?embed=artist
Sideline

Another way to include related entities. Like embedding, only the resource link is left in the original resource. For sidelining, the linked resource is included in the returned collection as a top-level element that can be retrieved by keyed lookup:

  # Fetch the artist represented by the 
  # `artist` rel in the album `links` 
  # collection.
  GET /albums?sideline=artist
Sort

Different applications will have different requirements for sorting complexity, but most should support basic sorting:

  # Sort albums by year, oldest to newest
  GET /albums?sort=year+asc

  # Sort albums by artist, alphabetical
  GET /albums?sort=artist+asc

  # Sort albums by year, newest to oldest, then album name, alphabetical
  GET /albums?sort=year+desc,name+asc
Paging

Remember to return relevant links, such as previous/next/first/last:

  GET /albums?offset=10&limit=10
  # Or...
  GET /albums?page=2&limit=10
Fields

Send only the fields specified. The default response should just set defaults for the fields param:

  GET /albums?fields=name,year,artist
Filter

Show only the results matching the given filter criteria:

  GET /albums?artist="Disclosure"

  # Get albums from all these years:
  GET /albums?year=1969,1977,1983,1994,1998,2013

  # Get albums from any of these years with the keyword "EP":
  GET /albums?year=1969,1977,1983,1994,1998,2013;keyword=EP

  # Get all albums except those with genre=country:
  GET /albums?genre=!country

This is an area where the tooling for Node could use a lot of improvement. Some early-stage projects worth following include:

Most of the examples in this chapter were generated using siren-resource. It was written for that purpose, but it supports some interesting features that the others lack, and I literally took the bullet points right off the siren-resource roadmap.

Optimizing for Speed

You can have the best API design in the world, but if it's too slow, nobody will want to use it. Here are some quick tips to keep things moving quickly:

  • Store all commonly viewed resources and queries in an in-memory cache. Consider Redis or Memcached

  • Make sure your cache headers and ETags are set properly for all cacheable resources.

  • Make sure you're compressing your resource endpoints. See the express.compress() or equivalent middleware.

  • Use paging. Make it automatic on every resource, and make sure there is a maximum page size enforced.

  • Stream large responses so that user agents can start processing it as soon as possible.

  • Index any keys that are commonly used for filtering and searching.

  • Limit access to queries that require processing time.

  • Avoid making unnecessary network calls. If you can colocate backend services and side-band resources on the same server, you'll trim a lot of latency.

  • Avoid making unnecessary IPC calls. If you can accomplish in a library what you have been doing in a separate process, do it, unless doing so could slow down the event loop.

  • Avoid blocking. Always use the asynchronous versions of Node function calls during request and response cycles. The only times you should consider synchronous versions are for application startup and command-line utilities.

  • Always use nonblocking algorithms for query processing. Don't process large collections with synchronous iterator functions. Consider time slicing or workers instead.

  • Take advantage of load balancing to spread the load across multiple CPU cores. The cluster module or HAProxy should do the trick. As of this writing, HAProxy handles it better, but the cluster module is undergoing a rewrite to improve its performance.

  • Offload all static assets to CDNs.

  • Profile your app performance. Use an app monitoring tool like New Relic to detect potential issues.

  • Load test your application. Carefully monitor latency and memory use characteristics under strain.

Conclusion

This chapter could have been a whole book on its own. I could not possibly tell you everything you need to know about building an API to prepare you for large-scale production within the scope of this book, but hopefully you have been introduced to most of the key concepts, and you at least know which keywords to Google for if you want to learn more.

Here is a list of best practices you may want to refer back to:

  • Use consistent routing.

  • Use hypermedia as the engine of application state.

  • Program to the media type, not to the API data.

  • Use a media type that supports the right number and right type of affordances (H-Factors) for your application.

  • Consider the benefits of using an HTML-based media type.

The UX is the soul of an application; the API is the heart. Take the time to make sure your application has a good one. Your users will love you for it.