Don’t have a meltdown: API Traffic Management »

(This is part 2 in our series on "Is your API naked?  10 API Roadmap Considerations")

Customers come up with far better sound-bites than we do. One asked us if we could protect them from “back-end melt down.”   They had opened an API for customers, but it was the same one their Web apps was using, so bursts of traffic from customer apps had their Web product running hot – and not in a good way.

A back-end melt down will lead to a front-end melt down in a hurry. That's a major risk to the business.

It's also one reason why traffic management is usually one of the first management features added to an open API. But usually there are both technical and business reasons for this.

Technical folks want throttling, or rate-limiting, to regulate traffic flow. Your customer's developers might inadvertently write some inefficient code. Someone might try to download all your data through your API. Rarely you might get some malicious use. Rate limiting or transaction throttling (X transactions per second usually) can be like a circuit breaker that protects your own apps and keeps your Web and API customers happy.

Business managers may need to measure consumption against a daily quota. This might be to find growing customers to upsell, or keep track of customer data usage (especially if you are paying for that data). Or they may want to create different or custom 'service tiers. '

Pitfalls to avoid with Rate limits and quotas

We see a few API providers substituting ‘quotas’ for rate limits – telling developers they can have X transactions per day. But just measuring against daily quotas can leave your ‘back end exposed’ if you get a short burst of traffic. And 'per second ' rate limits aren't a great way to create tiers of business-level consumption. You might need both.

Or, we see customers that define an SLA only on paper and say “call us if you need more. “ We had a SaaS customer that started this way but then found it was tough to tell important customers they needed to manage their API usage. They eventually found a good way to ease into the traffic management discussion - by first providing the customer with usage reports. ( “look how all your requests come in Monday at 9 am, we love you but can you help us?” )

And you may not consider all requests equal – you may find that some requests contain many transactions or big messages, so a simple “X requests per second” rate limit might not be enough.

So consider these questions for your open API roadmap:

What kinds of rate limiting do you need?

• Do you need to rate limit by developer, app, or customer organization?
• Can you rate limit a particular client (key and IP address)?
• Are all messages the same or do you need to adjust based on message size, or records per message, or bandwidth ?
• How do you throttle differently for your own web or internal apps vs. API traffic?

Does your business need quotas on API usage?

• Do you need to keep track of daily or monthly usage to measure business consumption?
• Do you need to define different consumption levels for different service tiers?
• If someone goes over the quota, what is the best business recourse? Call them up and upsell them? Or shut them off?
• If you are paying for the data are you measuring this for billing and pricing?

How do you monitor and respond to traffic management?

• How will you know when someone goes over a rate limit or quota?
• Are quota or rate limit overages a trend or a 'one time thing'?
• Can you define flexible actions? (i.e. drop requests, do nothing, send an alert?)
• Can you provide customers with data so they can help by metering themselves?

What about bad stuff like unauthorized access, Denial-of-Service, or XML specific attacks?

More on that in the next couple sections – Security and Data protection

(And here's one solution for help with API Traffic Management)