How RESTful is Your API?

Posted byCory House 58 Comments on How RESTful is Your API?

It’s been over a decade since Roy Fielding wrote his seminal dissertation on Representation State Transfer (REST). Over this period we’ve seen SOAP/WSDL fall out of favor as the cool kids transition their services over to the REST paradigm. Or so it seems on the surface.

In reality, we’ve spent the last 10 years building various ad-hoc services over HTTP that borrow bits and pieces from the grand vision that Roy outlined. We’ve settled into a rhythm that, depending on your outlook, is either a naive implementation or an enlightened massaging of the initial approach. Is the glass half empty, or half full?

A Religious Debate

Your answer depends on your faith. I say faith because a group of RESTafarians defend Roy’s initial vision with near spiritual fervor. Yet, those with such devotion must accept that they’re on the fringe. Why? Virtually no popular APIs that claim to be RESTful today actually honor the complete tenets of the faith. The market has spoken and chosen to be pragmatic. We, the wild unruly developers, have chosen to do our own thing. Check out Facebook, Twitter, and Twilio. All have APIs that claim to be RESTful but have various unique properties that go against REST tenets.

Pragmatism has a variety of benefits which I’ll get to in a moment, but many APIs that claim to be RESTful vary greatly in their implementation. Thus, REST has lost any concrete meaning since the canonical reference is rarely honored in the real world. Fact is, there’s no jargon to describe the architecture of the vast majority of “REST” APIs in use today1.

This inconsistency complicates the entire ecosystem around HTTP based APIs. Consider the following aspects:

  • Learning – Learning REST is difficult with such an amorphous definition. There’s no canonical reference and those who subscribe with devotion to Fielding’s work must accept a striking disconnect between his vision and a large body of popular “frauds” in the wild.
  • Development – Developers often rely upon benchmark implementations, but today’s “RESTful” APIs provide wildly inconsistent conventions. Even after ten years, the tooling is weak and much of the work remains ad-hoc.
  • Consumption – Since every “REST” API is a unique flower, consumers must carefully read documentation to assure they understand the unique approaches of each provider. Lack of convention slows both understanding and consumption.

So Is Anyone Doing it “Right”?

Well, this isn’t about right and wrong, but here are a select few APIs that fully honor REST:

John Moore presented an excellent example of a truly RESTful API at Oradev. Wow, that’s a short list. There’s a sea of “REST” APIs by major players that just don’t fit this bill.

REST Cheat Sheet

So let’s get down to brass tacks. Outside of the rare exceptions above, today if you see an API that claims to be RESTful, it’s basically saying

“To work with our data, send the proper HTTP verb to the URIs in our online docs and specify whether ya want JSON or XML”.

Yep, it’s that simple, and about that casual too. We API developers are doing the simplest thing that could possibly work. No shame in that.

Here’s a number of common ways today’s RESTful APIs deviate from Fielding’s REST, along with an explanation of why the deviation may be useful.

REST Principle   Why Deviate?
Honor HTTP Verb Semantics Although REST prescribes using HTTP GET, PUT, POST, and DELETE verbs for CRUD operations, some clients can’t generate the less common PUT and DELETE requests. In addition, some overzealous firewalls block PUT and DELETE. Thus, some RESTful APIs accept all requests via HTTP POST or GET and place the HTTP verb in the querystring. For example, to delete user 124, the POST request would be for the following URI: /users/124?method=delete This technique is especially common in Ruby circles.
Utilize HTTP Status Codes HTTP offers a rich set of 71 separate status codes which should be used to describe response status. Some “RESTful” APIs always return an HTTP 200, even when errors occur. This can be necessary when the client won’t allow the developer to elegantly handle any HTTP responses other than HTTP 200, such as when working with Flash. Twitter provides the best of both worlds by allowing the caller to request supression of HTTP status codes and always return a 200. Most RESTful APIs utilize a small subset of the potential status codes.
Self Descriptive Messages REST specifies that the HTTP accept header should declare a custom and specific format. However, very few real world APIs do so. Instead, two popular compromises exist:

  1. HTTP Accept Header – Set the HTTP Accept Header to Accept: application/json or Accept: application/xml to request JSON or XML respectively.
  2. Pseudo URI file extension – Manipulating HTTP headers is slightly more work than simply changing the URL, so for convenience many APIs place the format in the URI by appending the format to the end of the URL. For example, to get a list of users in JSON: /users.json. This approach makes for longer URIs and is less RESTful since it ignores HTTP semantics, but it makes testing in the browser easier since no HTTP headers have to be manipulated.
Discovery via Linking and Hypermedia (HATEOAS) Here’s the unicorn. Virtually none of today’s APIs honor this. To prevent tight coupling between the client and the service, truly RESTful APIs provide a discovery based API. Each call provides a reference to related calls. This allows the API to be highly evolvable because it avoids creating a coupling between the client and the server. This aspect is nearly universally ignored by today’s popular APIs, as made evident by the common pattern of publishing a list of URIs.
Hypermedia Aware Media Type Examples of hypermedia aware media types include HTML, XHTML, Atom, SVG. Notice what this doesn’t include? XML and JSON. Neither offers a native way to convey a hyperlink. This constraint is necessary to support discovery. So, if you’re not interested in the benefits of discovery (and based on real-world uptake, few are), feel free to return XML or JSON.
No Version Number Since fully RESTful APIs are discovery based, the API is expected to change at any moment. Thus, no version number is required. However, as we’ve discussed, the majority of today’s popular “REST” APIs document a static list of URIs that they support. Once you’ve accepted discovery is off the table, version numbers are beneficial to assure all clients aren’t required to simultaneously upgrade whenever the API changes. Version should either be the first segment /v1/users/ or in the querystring /users?v=1. If you plan to support concurrent versions of your API, specifying version should be required on day one.
No static URIs A discoverable API can change regularly and thus should be interacted with solely through the links provided through API calls. However, the vast majority of today’s “REST” APIs document precise URIs and their corresponding return types. The benefit is efficiency since you can directly make the desired call, but at the expense of flexibility due to the tight coupling it creates between the consumer and producer. With “hard coded” documented URIs, if you change the API you need to support concurrent versions as described above, otherwise both consumer and producer have to change simultaneously.

For more insight on Pragmatic REST, check out this excellent video by Brian Malloy at Apigee.

So Is Microsoft Pragmatic?

In the .Net space, we have two common options for creating RESTful APIs: WCF’s REST Template and the new kid on the block, Web API. The former was recently deprecated in favor of Web API. Although both technologies are completely capable of supporting a fully RESTful API, out of the box neither does so. Both direct developers down the path of creating a pragmatic RESTful API in today’s most popular style. In short, this means neither directs the developer to utilize either discovery or leverage the power of native HTTP response codes.

Hammock

So what’s the big deal?

Okay, full disclosure, I believe in pragmatic REST. Hey, I strive to be a pragmatic programmer. Although I have no qualms with the deviation, the fragmentation and inconsistency need to be addressed with a canonical reference or convention that shines light on the deviations. In the meantime, developers must strive to make conscious and rational decisions when designing RESTful APIs. REST is an overloaded term, so it’s important that we’re clear and deliberate about the tradeoffs we accept when deviating from the initial spec.

Edit: This post hit the front page of Hacker News! There is plenty of great conversation over there in addition to the comments below.

1 The same issue has occurred today with AJAX. We’ve long since moved on to JSON from XML but AJAX remains in the vernacular. It’s understood that returning JSON over XmlHttpRequest can still be referred to as an AJAX call. (Yes, some pushed for using the term AJAJ but it didn’t catch on).

More Resources:

58 replies on “How RESTful is Your API?”

    1. Good point Tomasz. I’ve done some reading on ServiceStack and seen nothing but positives. The problem is in many Microsoft shops it’s a hard sell to get open source software approved. I’m currently working in the banking industry where open source is viewed (arguably justifiably) as a security risk. Licensing is also more difficult in larger organizations where a lot of red tape restricts people from trying anything with an unfamiliar license.

      Finally, I’ve seen many .Net shops have an illogical mindset that anything Microsoft has created must be of class leading quality. This clearly isn’t always the case, but many developers and managers prefer the safe and familiar. That’s a hard mindset to break.

        1. Echoing this.

          BTW – I work in banking too and the developers and teams I’ve come across tend to vary all the way from those following almost all the corporate rules regarding libs, tooling, etc. to those following almost none of these rules. To date I’ve not heard of anyone in the latter end of the spectrum encountering problems due to their lack of box ticking.

  5. “XML and JSON. Neither offers a native way to convey a hyperlink.”

    Could you elaborate a bit on this? What do you mean “native way to convey a hyperlink”?

    1. Great question Geoffrey. In HTML the standard way to convey a hyperlink is via an anchor tag. e.g. bitnative.com . Your browser knows how to parse this and what to do when you click on it. There is no such convention in JSON or XML. If you check out the video from Ordev I linked to in the article the importance of hyperlinking in Hypermedia REST APIs.

      1. Um…this is because HTML is for formatting, JSON and XML are self-descriptive data formats unconcerned with formatting. In that sense, it’s infinitely more generic than HTML.

      2. XML has no native way to convey anything. That’s the point: that it provides standard syntax while leaving the definition of semantics entirely up to the developer of individual tagging languages. Any tagging language you can design using XML syntax can have its own way of creating a link. Why shouldn’t a RESTful API expect the application that requests it to understand how links are created in that tagging language, just as much as it would need to understand the meaning of any other tags in that language?

        Isn’t that the implication of “REST specifies that the HTTP accept header should declare a custom and specific format”? In other words, that the application should specify not just that it accepts anything in XML syntax, but that it accepts a specific set of XML semantics, which it is designed to understand and interpret?

        That aside, there is the XLink standard which, though little used, does specify a standard way for any XML document to express a link.

        1. Great point, Mark! XML isn’t a protocol any more than REST is, I don’t see why this is a ‘deviation’.

        2. XLink solves about half of the problem, or maybe only about one third.

          One third of the problem is “how are links represented.” Xlink addresses this directly. Without Xlink (and speaking my native language JSON), I might represent links as
          { “rel”: “self”, “url”: “http://blah/blah”}
          or
          { “links” : { “rel”: “self”, “url”: “http://blah/blah”},
          { “rel”: “create”, “url”: “http://blah/blah”}}
          or even, as is sometimes proposed
          { “id”: “37”,
          “links”: { “rel”: “self”, “url”: “http://blah/{id}” }
          }
          Xlink helps reduce this ambiguity.

          But another part of the problem is, “which links shall I actually represent?” There are some obvious candidates … when showing a resource, include the DELETE and UPDATE links, for example. But what about relationships? What about search terms? What about properties not visible in the resource paths or descriptors (for example, in our API we allow the caller to create resources, but these must have unique names; as a helper, we provide an “isNameUnique?” URL — but how is that represented, and where?)

          Finallly, there’s the problem of “what links do I want?” The whole HATEOAS notion seems to be “what works for web pages will work for APIs.” But this is not necessarily so. Web pages have a human in the loop of every transaction. If a web page suddenly sprouts a “Delete” button, this human can be trusted to understand what it means, and to use it or not, as desired. But if an API originally lacks a DELETE function, and then at some point sprouts one, how does the pre-programmed client application even know to go looking for it? And DELETE is easy, a standard function; what about “isNameUnique?” What about “search for a resource containing the string ‘bazooka'”?

  7. Great post and presentation!!

    Its also worth to mention that returning valid error codes prevents proxies from caching. So if you return 201 like facebook does with a specific error msg in the body, it could happen that this request is considered as valid and is beeing cached in the proxy.

  8. I started writing a reply about the discoverability feature in REST and how it’s not implemented, and I ended up turning it into a post on my blog 🙂

    Quick summary: There is no standard for discoverability, making it hard to implement on the server. Furthermore, I don’t think there’s any interest in a “rest browser” client app – client apps are where developers can control the UX and design specific to the API(s) they’re using, end users don’t want a generic api browser app, which I think the discoverability feature is really geared towards.

    http://rtigger.com/blog/2012/08/27/discovering-the-value-in-discoverable-apis/

    1. Great stuff Chad. I found the point that the client needs to be custom to the api most compelling. It boils down that the primary benefit of the loose coupling is eliminating the need to version the api. Though without versioning even a discoverable api can’t remove functionality without breaking clients. So hey, we’re nearly back where we started! These points help to clarify why pragmatic REST has become the mainstream choice.

    2. Agree fully with your low opinion of an API Browser. But there’s another user story where discovery is more meaningful: at the point of development. If a developer, creating his app, hits an error, or gets unexpected returns, or just loses track of “what comes next,” then some well-identified URLs in the response can be enough to get them back on track. At the very least, I like to return a help-docs link with any error; returning “likely next steps” with any success is also helpful, though a bit more speculative (you don’t know what the developer is trying to do).
      This story doesn’t require a custom browser/client. In fact, this story works just fine with something as primitive as a log file!

  9. My favorite hilariously-non-REST API is TeamCowboy’s “REST API”: http://api.teamcowboy.com/v1/docs/

    Somebody should make a big scorecard for web APIs, so we can see at a glance just how much in violation of REST principles they are. Of course, it might well turn out to be a big table of pure red X’s.

    1. Ouch, MessageComment_Add, MessageComment_Delete, etc? Yeah, that’s rough. Thanks for sharing!

      Regarding the scorecard, I had roughly the same idea and was actually working on it for this blog post. I ended up tabling the idea for the moment as it became too much of a time sink. I hope to draw up a proposal in a future post.

  10. I don’t understand. The bullet point is no version number. Yet you advocate using a version number? I understand for a brief moment you say not to have one, but then you go on for the rest of the paragraph explaining how you should do it. If you are recommended no version number, why not give solutions to problems when you don’t have version numbers, instead of saying “have versions”?

    1. Hi Brad – The left column is the REST principle. The right column explains why you might decide to deviate. Since virtually no one is following the REST principle of discovery and instead publishing a static list of resources, we have no choice but to accept the necessary “evil” of version numbers. Version numbers are the only way we can avoid breaking consumers when the API changes. I’m not preaching that pure REST or pragmatic REST is right or wrong, but if you choose to go pragmatic then specifying and requiring version in all calls is necessary to avoid breaking customers still expecting the “old” version of your API. I understand your confusion and will edit the point for clarity.

  11. What are your thoughts on the linked data space? They seem to have it right in a lot of ways that are hypermedia friendly, and prefer sparql endpoints over custom Apis for the GET side of things…

  12. A couple of comments…

    A. “With “hard coded” documented URIs, if you change the API, both consumer and producer have to change.” – not if you version your API. With versioning, consumers don’t have to change unless they choose to do so.

    B. I see a lot of arguments in favor of specifying the API version through the Accept header, rather than as part of the URI path or query string. (Granted, it’s harder to test the API via the browser in that case but then again why test through the browser?)

    C. I see a surprisingly small amount of discussions about how to formalize a given RESTful service with something like WADL or perhaps WSDL 2.0 (http://www.ibm.com/developerworks/webservices/library/ws-restwsdl/). I’m very curious about folks’ experiences with grammar-oriented tools like those.

    1. Hi Dmitry,

      A: Agreed! I edited that point to clarify.
      B: Do you have a link to a specific API which is doing so?
      C: First I’ve seen of this, but reading up now.

      Thanks for the feedback!

  14. On the point about status codes, if the API supports JSONP then it needs to always respond with a 200 status code. This is because most clients will not execute the JavaScript/JSON file that is returned with a status code other than 200.

  15. can anyone explain what is “Self Descriptive Messages REST specifies that the HTTP accept header should declare a custom and specific format”?

  16. Hi Cory! Thanks for a good read.

    I agree with your assertion that that majority of today’s “REST” APIs does not actually meet the criteria to be called such. And that’s *perfectly fine*. As I recently wrote, your API *doesn’t have to* be RESTful.

    However there’s a perfectly clear definition of what the REST architectural style is. Your implementation either fulfills the criteria or it doesn’t. It’s either RESTful or (most likely) it isn’t.

    I am 100% convinced that all these faulty labels comes from ignorance. Most developers doesn’t know better and they honestly confuse REST with HTTP. Maybe you’ll agree with this?

    But where we seem to disagree is how to meet this ignorance. I believe there’s value in calling cows cows and horses horses. Rather than giving in I would love to see more effort in educating the fledgling developers.

    Perhaps all we’re missing is some abbreviation for Nice HTTP Based APIs?

    1. I totally agree. Pragmatic REST is the closest I’ve seen to something that has hit the vernacular. But that’s so amorphous, and exactly why I wrote this post.

      I don’t believe the labels are from ignorance. Big players with bright minds consistently choose pragmatic REST. The market has chosen simplicity and efficiency over flexibility and discovery.

      1. “I don’t believe the labels are from ignorance. Big players with bright minds consistently choose pragmatic REST.”

        The two are not mutually exclusive.

        What I’m suggesting is that the majority of API developers today seem to never even heard of HATEOAS and yet they actually believe their APIs are RESTful. Understandably, as they are most likely spurred on by all the half-witted “Create a REST API in 5 minutes” tutorials.

        Big players and enlightened developers might still (and in many cases probably *should*) opt for a non-RESTful architecture. But making an informed decision and then calling it “just” a HTTP based API is a whole other matter.

        I would say “the market” hasn’t chosen anything, but rather been led to believe that REST = HTTP.

  17. Fielding defines REST as client/server, stateless, cache-able, uniform interface, layered system, and code-on-demand (optional).

    He says it doesn’t even have to be HTTP and he certainly doesn’t discuss HTTP verbs or status codes.

    This article should titled “Does your API conform to my personal, subjective definition of what comprises a RESTful API”.

    1. John,

      Yes, Fielding doesn’t specify such detail. That said, HTTP is nearly universally the protocol being used today by those creating “RESTful” APIs. And once you accept the HTTP protocol, the conversation turns to best practices for using it in a RESTful way. HTTP verbs are therefore an obvious point of conversation.

      I hear your point that best practices for REST over HTTP are subjective, but that’s no news. There is no detailed spec, just some high level principles. And that’s precisely why I wrote this. To help bring some clarity to the trade-offs we are all considering using HTTP in an ad-hoc way. If there are specific recommendations that you disagree with, I welcome your feedback.

  18. Pingback: How RESTful is Your API? « The Blog of Anthony Diaz

    1. Good find! However, the JSON-Schema you linked to was merely a draft and as displayed in the header it expired in 2011. Of course, when/if this is approved, to be useful for a HATEOAS (aka a fully RESTful API) browsers would need to support parsing this schema as hypermedia so that the links in each JSON response could be followed.

  21. Pingback: Distributed Weekly 170 — Scott Banwart's Blog

  22. Overall I like your post, however…..I have a few bones to pick on your representation of Web API.

    I think your “Does Microsoft Get it” section does not accurately represent Web API at all and lumping it in with the WCF REST template is a big mistake. I agree with your statement that there are some things that are not included in the box, like support for ETags for example.

    However, I’d argue (and we worked with a many folks from the REST community to ensure) that it has all the foundational pieces to enable building a RESTful system and does not stand in your way. I’d also argue that it naturally leads you toward a RESTful system however it is not opinionated about REST and does not FORCE you down a RESTful path. So yes, it allows a more “pragmatic” path.

    Based on your comments, I’d would ask you though if you actually did look at it?

    1. Web API supports custom media types including those that use hypermedia. It includes an extensibility point for custom formatters. A formatter can be any representation including one with hypermedia. Web API ships out of the box with support for one one media type that can use linking, OData. However it can support a much broader set. For example if you head on over to Web API contrib you will see support for HAL via a HAL formatter.

    2. Web API supports conneg out of the box. Media types are selected by default based on the accept header. This includes custom media types such as the HAL formatter I described above.

    2. Web API maps to the uniform interface conventionally. An API controller is design to be a handler for a resource with each method (action) mapping against an HTTP method.

    3. Web API supports a new easily accesible HTTP programming model. This allows you to directly manipulate HTTP requests and responses in order to adhere to the spec. For example it is very easy to return a status 201 and set your location header on a POST, something that the HTTP spec describes.

    4. Web API supports a pipeline for addressing other standard HTTP concerns like ETAGs.

    1. Hi Glen,

      Thanks for the detailed feedback! It’s great to have a Microsoft REST expert chime in. I indeed use (and enjoy) Web API. I lumped it together with WCF because neither technology leads the developer to utilize HATEOS out of the box. As a fan of pragmatic REST I like that decision, but this post seeks to shed light on the logical reasoning for the tradeoffs.

      I totally agree that Web API offers a great deal of configurabilty and is a clear improvement over the WCF REST template.

      I will edit my post to clarify the variety of RESTful capabilities Web API offers. Thanks again for the feedback!

      1. Cool 🙂 Glad you like it. I guess you struck a bit of a nerve :p

        Yes we definitely didn’t want to force people down that path. Web API by design is open to different styles / not opinionated. However the difference is that we thought very intentionally about REST / things like hypermedia formats so you CAN do it. I don’t think you can say the same about our previous efforts 😉

        Lack of in-the-box support for HATEOAS (hypermedia) is a point in time thing. The fundamentals are there which as I mentioned using something like the HAL formatter demonstrates today. I expect we will have something in the box though in the near future.

        Still it will always be a choice, not something you are forced to do.

  23. Aside from that last rant, I do agree with the crux of the post. I am pretty hardcore about REST when the business constraints warrant it. However, it’s not the only way to build a system exposed over HTTP. It is however excellent for building systems where you need client and server evolvability over a long period of time, or where you have many different types of clients and a rapidly changing system. Many systems however do not fit that bill. Beyond that whether they do or do not, it’s up to the business to decide whether or not the investment is warranted. I agree with you there are a lot of misconceptions around REST vs NOT. For many folks in the REST community / the restifarians the concern is not so much whether or not you build a RESTful system or not, it is whether or not the term is used appropriately.

  24. In terms of the WCF REST Template, I don’t think it actually is easily capable of a truly RESTful api. For example, see how easy it is to plug in a custom hypermedia format or to support ETAgs. Yes it’s technically possible but ALOT of work. The template makes it easiest to simply expose your existing web services via URLs, there’s a big difference between that and a RESTful system. Also you find yourself constantly knocking your head against the wall based on early design decisions which were heavily impacted by SOAP. That’s one of the reasons we built Web API and ultimately pulled it out of WC altogether.

    OK, I am done….

    1. Good stuff Glenn. This background certainly helps me understand why me placing WCF and Web API in the same paragraph felt so off to you. I completely agree that Web API has a more thoughtful design. I’m currently debating building all my team’s future web services in Web API rather than WCF, but our in house performance testing using Web API (admittedly beta at the time) showed WCF had the performance edge. You seen/done any benchmarks?

      1. Did you profile on the RTM bits, they have been heavily optimized. I believe self-host gives you the best performance if that is what you are looking for as it’s pretty raw over HTTP listener. Also you can use the OWIN support to use other OSS servers like Kayak.

  25. Pingback: How RESTful is Your API?
  26. Pingback: How RESTful is Your API?

  27. Trying to find the origin of the demarcation between REST and non-REST-but-claiming-to-be-REST APIs is interesting. But I believe you are wrong in identifying it as “pragmatism”. And the reason is that it can often be very pragmatic to build what you call “discoverable” APIs. I do that all the time when we need loosely-coupled, evolvable systems: that’s a very pragmatic choice. Thus, this ideal-REST vs pragmatic-REST demarcation isn’t the right one, I beleive.

    So, why are there non-RESTful API that call themselves RESTful ? In my 10+ years of REST aware consulting, in about 99% of cases I came across it was simply due to ignorance (I did it at some point too, of course). Usualy, people knowing what REST is and producing a non-RESTful API don’t call it RESTful (unless they are insane).
    BTW, I also observed this in big shops (MS, Amazon, etc.) If you think people there magicaly aren’t as clueless as everyone else, I think it’s probably bc you haven’t meet them! Hell, even someone of the caliber of Don Box said that they did SOAP the way they did bc at that time they didn’t understand the Web and were beginers Web wise.

  28. Great post. Two other reasons to deviate: JSONP forces you to return 200s, and trying to constrain response size to work well with mobile devices forces you to avoid discoverable APIs. Also, it seems like even a discoverable API requires some shared understanding of what should be available, and versioning when the server wishes to break that contract?

  29. Pingback: How One Blog Post Landed 30,000 Readers in Two Days | BitNative
  30. Pingback: Do you know what a REST API is? | jQuery4U | No.1 Resource for jQuery Developers
  31. Pingback: REST Maturity | Secure Software Development

Comments are closed.