This question was recently put to me by an architect at a Sydney based cloud SAAS business I have done Architectural consultancy with, and I thought it would be good to publish my answer, as this impacts quite a lot of businesses and systems architecture decision making all over the place.

The question is an interesting one and essentially cover two ‘camps’ of thinking:

• The API should be ‘enough’, roll your own client code;
• The API is just conduit for data, the client library is the real endpoint…

Myself I tend to fall into neither camp, as both are valid viewpoints, it just depends on the environment in which the API is going to be used and the intended audience.

Not all API’s are equal

There are API’s and there are API’s – some big, some small; some fast, some slow, etc. Basically how much effort you put into defining and standardizing your API rigorously comes down to how and who is going to use it. The factors I consider are:

• Does the API need to be quick like lightening or just sufficient?
• Is the API expected to grow and change over time?
• Does it have to be secure? Does it need to follow certain security standards?
• Internal/External. Is the API for internal use only (just your business) or is it for public consumption?
• I18N support. Does it need to deal with multiple time zones, languages and locales?

These factors should give you a pretty good handle on how much actual definition needs to be done with respect to the API; basically how much effort is spent documenting and testing your API in different circumstances.

Now back to the original question. Given the above what benefits does providing a client library deliver?

• You have the ability to enact client side caching and other strategies (routing, rate limiting, profiling, etc).
• You have moved your interface description to a rigid library interface rather than just the API itself.
• The library can enact the security on behalf of its users.
• Internal/External. Both benefit in improved speed of development.
• I18N support. Again this can be codified in the client.

In essence the real win here for API clients is twofold:

• The ‘assumption surface’ (for want of a better defining term) is moved away from just resting on the API to been codified ‘within’ the API client, so rather than running the risk with a naked API of its users coding with a different set of assumptions to the API developers, instead you have ‘fixed’ a large percentage of your assumptions into the client. As we all know assumptions exist everywhere – programmers have a duty almost to automate out assumptions where they can. So moving an assumption out of people’s heads and into something ideally a compiler can check is a Good Thing.
• Change’s to the API become something that can be managed at both ends of the API ‘pipe’ – in effect users of the client library are not ‘bound’ to the API, they are instead bound to the client library. This gives you more (and even dynamic) control in how exactly you implement your API. You might decide one day to refactor or split up your endpoints across different server pools to get a speed up (you know this is a good move because of the profiling you put in the client earlier) – all you do is create a new version of your client library that supports this and put it through your release process. No ‘you broke it!’ calls in the middle of night. No need for X many developers to write and test their code to adapt, etc. It’s smiles all-round. You might even give certain clients their own API endpoints, they need not know!

Noisy Neighbor Problems…

The other significant pain point with naked API’s is what I term the noisy neighbor problem, i.e. you have a 3^rd party program that is calling your API and suddenly the darn thing goes rogue and starts spamming your endpoint to such an extent that it actually impacts on your SLA (or your elastic cluster grows 10 fold, with an associated large bill at the end of the month). Not a nice problem to have. Yet if you enforce everyone to be using your client library, you can embed into it caching techniques and rate limiting; so they cannot become a noisy neighbor – instead at the worst case they get an error message (yes I know as a principal you should implement server side rate limiting as well, but I’d rather consider that a train line end buffer solution technique, better the client stops themselves before it gets to that point).

Owning your pipe

As I mentioned above if you own both ends of the API pipe, you can actually do a lot of smart things to make everybody’s lives more productive. Some additional points to consider:

• Back-grounding and data flow scheduling – you decide when data is transferred
• Standardized Debugging – have verbose switches!
• Logging and replay support – log in enough detail to allow a meaningful replay if required
• Flexible caching techniques – you know what the client calls and the frequency, use their resources to make your system scale more cheaply..
• Unit testing
• Community development
• Code generation – you could generate the client API off the description of the API itself, but usually only worth doing for the big end of town.

Now I know what you are thinking – bugger that, or words to that effect. Well trouble is, if you don’t do some of the above, you will actually end up more than paying for it in lost time and efficiency if things take off – don’t just look at your immediate cost, consider the costs to the end users of your client library and then the ‘return’ costs to you later on. Sometimes its better to nip things in the bud before they become a headache, especially when you know what is coming round the corner… Now you might get some push back off management, etc – first, please refer them to this article; in essence the client library assists in achieving take up and scaling with the least ‘speed bumps’; yes, there is a small upfront cost but the payback if things take off is massive. This is why the likes of Stripe have gone out of their way to make it easy to use their API’s by covering off a whole set of programming languages with client API libraries – reduce the friction and the developers will come!

Owning your data types

Another advantage, often overlooked, is if you provide an API client library you can be very strict upfront on the types of data accepted by your client and often nip in the bud ‘lost in translation’ problems that can occur. For instance, if you accept a monetary value, which is it:

• 50
• 1000,50
• 1 000,50
• 100050
• 100050 USD
• 50 USD

Depending on where you are (and if you operate in cents/pennies) it could be any one of the above. You might just do on the API endpoint a simple ‘read decimal’ type string to decimal conversion – depending on how that works, the 3^rd format might just return 1 and you would never know. The 2^nd format could return 1000 and again you would never know, that 50 cents just vanished into the ether to (maybe) show up on some accountant’s spreadsheet when it comes time to do an audit.

You can get a similar problem with date, time, duration and distance types, although these are usually better behaved you can still get issues like ‘5/6/10’ – there are least 3 valid ways of parsing that. Better to avoid all doubt and force the user of the API to be specific with their data up front via your client.

For an example of how API specifications and assumptions can cause problems see this

http://www.nature.com/news/software-error-doomed-japanese-hitomi-spacecraft-1.19835

In essence because of an error in reading gyroscopes and firing the wrong thrusters due to the wrong command being uploaded – the spacecraft is spinning out of control and is now a complete $286 million write off. Now if they had proper client API libraries in place that reduced the ‘assumption space’ – they might not have had such issues. For another example (this time English to metric conversion failure) see http://articles.latimes.com/1999/oct/01/news/mn-17288 (not that I’m picking on space programs…).

Are you a programmer or a hacker?

To me, if you are providing an API for general consumption and you have known language user base; why are you forcing every separate user of your API to write their own glue code? Isn’t that a bit ‘evil’? or are you expecting the community to write a client for you? To me a programmer does their best to support their community and encourage best practices – to do otherwise is to tempt fate, which doesn’t often go well for you…

When you don’t need a client library

There are a few special cases where you don’t need a client library as such:

• Very simple REST api – no complex types, low number of arguments, few operations, etc
• Purely ‘internal’ to a system or group – nobody else is going to use
• You are pulling instead of pushing, i.e. you are calling out with well defined calls (which need defining) rather than receiving calls.

You might also have the case where there is a large variety of programming environments being used on the client side, although in my experience at most 3 languages will cover the majority. You could provide one reference implementation and see if anybody takes that and rewrites in other languages. Don’t let the variety prevent you doing the right thing. Also its highly likely that if you have complex API’s that you are also allowing ‘extensions’ into your systems anyways, so your language of implementation will ‘leak’ and influence the language used by the clients…

Conclusion

I would recommend that if you have an API that displays any 2 of the following qualities:

• Critical to your ongoing business operations
• Needs to scale & change over time
• Will be used by other parties outside of your control

Write a proper API client immediately and consider adding in (or at least leaving space for) the features I mentioned above. It could literally save your business and help create a community around your API.

I’ll leave you with this thought: would you rather have it in your house that the mains was terminated by just 3 wires at each outlet, or would you prefer a proper socket? The first option carries the risk that everything could break and that it might be the last thing you do…