Video demo: New updates to the API Console »

This week we released some updates to our Apigee API Console. Here is a short demo of:

• a new API provider page that lists all the API consoles
• a new API Resources page that lists all of the API operations 
• UI changes to the Console itself (split screen, awesome bar, and more)

For now these updates are limited to just the Twitter console, but we'll soon roll them out more broadly.

We'd love your feedback on these changes - you can drop us a line at .(JavaScript must be enabled to view this email address).  Thanks!

'

Conquer OAuth once with the Apigee OAuth API »

We're delighted to make the new Apigee OAuth API available for public beta. 

Get started here!

The OAuth API makes it easier to use OAuth by giving you a single keychain for authenticating to multiple APIs.

You avoid the OAuth Headache because you delegate the OAuth dance to Apigee and use a keychain to invoke multiple APIs in the same way.

Here is what the code looks like before and after:

For today's public beta, we're starting with support for the Twitter and the Salesforce and Chatter APIs, and adding more soon.

You can get started at developer.apigee.com.  We also have Javascript and Java client libraries available as examples.  We'd love your feedback!

Apigee API Console on Dev.Twitter.com »

If you've been over at the Twitter Developers site recently, you might have seen their new Apigee API Console, which makes it easy to explore the API, send and view requests and responses, share what you are seeing and solve problems faster.

Twitter launched the API Console along with some great updates to the Twitter libraries page and a refresh of the resources documentation. 

We hope you like the new Twitter console and if you want one for your own API, check out Apigee To-Go - it's free! 

Do you need an SDK for your API? »

In our last couple API strategy workshops, we've had good discussions on a common question for API providers - do you need to offer an SDK (software dev kit) for your API?

In this context we defined an SDK as going beyond offering an API to include platform-specific code that developers use in their apps to invoke API operations, also including the source code, and documentation that developers might need. Here's what we came up with:

Reasons to consider an SDK:

Speed adoption on a specific platform - for example Objective C SDK for iPhone.  Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.

Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing. 

An SDK can help reduce bad or inefficent code that might slow down service for everyone.

As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation.  (See some good examples here and here. )

To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's exisitng developer community.

Reasons you might not need an SDK:

If your API is designed for adoption -  standards-based, well documented, etc.. developers may be able to get rolling without a client SDK.

Resources - it's tough to do a SDK for each platform you target, you'll have to pick carefully. Also, it can be time consuming and expensive to create SDKs (tip: maybe packaging up internal samples or test cycles can kickstart things)

Maintenance and versioning - convincing clients to upgrade to the newest version can be rough. Also, capturing the "local flavor" of each language you support can be a challenge.

Complexity -On each platform there might be use cases you don't expect, like keeping application-level secrets off clients, debugging, etc.

More is not necessarily better - fewer, more well-documented, code-level samples can often be the best resource. Facebook provides Android, C#, iPhone, JavaScript, PHP, and Python libraries (all documented differently), but Twitter supplies none. Facebook has more resources but still struggles a bit at times.

Our view: whenever possible, forget the SDK, and spend the time making your API better and more useable, and give good API resources such as example code.  

Agree or disagree? 

Developing on the Facebook and Twitter APIs with Robert Scoble »

Recently Apigee's Sam Ramji sat down with tech guru Robert Scoble to discuss the API-driven web and the evolution of development in a world of app stores and millions of connected devices. Take a deep dive into the Facebook and Twitter APIs with Apigee's "X-ray for APIs" over at building43 or embedded below:

Hope you are enjoying the new Apigee site! Make sure to sign up for a preview of Apigee Premium and if you love APIs, send us your address and we'll mail you an "I <3 APIs" sticker - anywhere in the world. 

OAuthpocalypse? Have No Fear, ApiGeeks! »

The Twitter API is one of the most popular APIs on our platform and with Twitter shutting off basic auth and moving to OAuth tomorrow, we wanted to reassure you that Apigee is ready for the OAuthpocalypse with full support for APIs which use OAuth.

Here's how to do it, using the Twitter API as an example:

1. Do the OAuth dance with the canonical API endpoint (e.g. api.twitter.com).

2. Once you have a user's token, use it to sign requests' base strings, which you would also build using the canonical endpoint from step 1.

3. Then send those signed requests to your Apigee URL, such as twitter.myusername.apigee.com.

In this way traffic can flow to and from Twitter via your Apigee API using OAuth.

Let us know if you run into any questions or issues- visit our support site or ping us on Twitter at www.twitter.com/apigee. Good luck with the big switch!

Hooks, Sockets and Firehoses: Streaming API Technologies Getting Us to REALLY Real-Time »

If you rely on Twitter to see what your friends are having for lunch, waiting 24 seconds to get an update is fine. If you're a stock trader who could have lost a few thousand dollars or more while reading this post, 24 seconds is a very long time to learn that your favorite company just got acquired.

Real-time runs on APIs as much as it runs inside a web browser. This means that the "real-time web" isn't always "real-time"- a web service API is typically a request-response technology, requiring that the client makes a request of the server, waits for a response, and then processes it. This can add critical seconds or milliseconds.

Earlier this month I attended the ReadWriteWeb Real-Time Summit in New York City. In an "unconference" session on streaming API technologies, we discussed 3 technologies bringing us closer to REALLY real time:

HTTP streaming- This technique uses the HTTP protocol, but instead of responding to a request with a single piece of data, the server responds with many small pieces of data that are published as soon as they are available. It's as if you're using HTTP to retrieve a very, very, large file -- the server just keeps pushing out data, one piece at a time. Twitter uses this technique in its Streaming API.

Since HTTP Streaming doesn't work quite the same way that most HTTP-based services work, the program that reads the stream has to be written to process the incoming data, handle reconnects, and so on. It's also difficult to use in a web browser without doing some JavaScript trickery; HTTP Streaming is mainly suitable for server-to-server communication.

WebHooks is a real-time tech with a simple concept: whenever something happens on server A, it makes an HTTP POST to server B and sends it the update directly. Like HTTP Streaming, it only works for server-to-server communications, but unlike HTTP Streaming the barrier to entry is very low for the programmer - anyone who can write a program or script that can process an HTTP POST can consume a WebHook. Unlike HTTP Streaming, however, a WebHook uses one HTTP request for each event, which means that with today's servers a WebHook is going to top out at several thousand events per second, whereas HTTP Streaming can theoretically process many more events.

WebSockets, unlike HTTP streaming and WebHooks, is designed for real-time streaming to the client- useful for creating real-time apps that run in the browser. WebSockets allows a client and server to use the HTTP protocol for some initial interchange, and then agree to communicate bidirectionally. The result is just like a TCP socket -- the client and server can send whatever data they'd like, in any order, in either direction, for as long as they want. WebSockets also supports JavaScript, ensuring you can easily build a high-fidelity, near-real-time application in a web browser using Internet-standard protocols. WebSockets is part of the HTML 5 family of standards, already implemented in Google Chrome and Apple Safari, and will eventually come to all the browsers.<;It was good to see Kaazing, who is leading the development of WebSockets, hanging out at the RRW Real Time Summit. With some existing technology out there and some new stuff on the way, the stage is set to build a real-time web that's a bit closer to real-time.

With some existing technology out there and some new stuff on the way, the stage is set to build a real time web that's a bit closer to real time.

This Week in APIs- June 7-11, 2010 »

Check out the top news in APIs this week!

S3 Import/Export API- Amazon Web Services added an Import/Export API for its Simple Storage Service (S3), an infrastructure service for storing data. The new API makes it easier to move large amounts of data in and out of Amazon- a great move for data portability.

Twitter API Meetup- Twitter held a meetup in its San Francisco offices this week to talk about the Twitter API, showcase some cool apps and celebrate Apple's WWDC. Twitter's Taylor Singletary presented on XAuth, a new authentication method for APIs. Twitter will no longer allow support for basic authentication at the end of the month so Twitter devs, it's time to brush up on the new method! The meetup made it very clear that the XAuth standard was developed to be used across the API community and promises to make authentication safer and easier, so keep an eye out for adoption.

APIs in the world of PaaS- Over at CloudAve, Krishnan Subramanian has a post and some love for Apigee on why APIs are the oxygen of Platform as a Service offerings- check out the article for some stark examples of how APIs can mean the difference between success and failure.

New APIs! A number of companies debuted new APIs- here are a few we thought were especially cool:

• HubSpot, which launched a Lead API to connect HubSpot's lead capture tools to other CRM software. Excellent example of how APIs can be used to connect tools and unleash new functionality. 
• The New York Times introduced a Most Popular API to track the most viewed and shared content- fodder for mashups here! 
• Microsoft introduced its Trial API that allows developers to include free trial versions of their apps in the Phone 7 App Market- more signs that the mobile API world is maturing. 
• As always, ProgrammableWeb's Newest APIs list is a great way to keep on the top of the all the new delicious APIs out there. 

Anything important we missed this week in APIs? Shoot me an email at .(JavaScript must be enabled to view this email address).

Scaling Twitter Applications Without Blinking »

Twitter has attracted not just a large quantity of users, but a huge number of developers.  The core utility of having a short messaging service that is both open and connected to an expanding social graph continues to find new uses every day as developers invent new applications that use the Twitter API.

We recognized this early on as we built Twitter-specific capabilities into Apigee (the console and Twitter analytics) and launched it at Chirp on April 14th. We are truly grateful to all our new developers for their adoption and the insights they've offered us about growing the service.

The next logical step is to bring our service to where the developers are, rather than making them come to us.  Today we’re announcing our partnership with Heroku, with Apigee available as a Heroku Add-on.

We want to help Ruby developers scale up their Twitter apps to meet demand.  Heroku makes it easy to scale up a Ruby application and Apigee makes it easy to scale up access to the Twitter API.  Without these kinds of services, developers usually go through a painful rearchitecture process as their application becomes more popular and demand for their computing resources increases beyond their planned limits.

From what we’ve heard we think that this will be a very helpful combination, letting successful Twitter applications rise to meet demand gracefully.

You can try the add-on yourself at Heroku and check out this short video by Marsh on how to use it:

Sam Ramji discusses Apigee’s new Twitter API Tools with Robert Scoble. »

Robert Scoble of Rackspace caught up with Sam Ramji to discuss the latest release of Apigee and all the new Twitter centric features. Check out the full interview below!

@sramji demos @apigee @twitterapi features at #chirp »

Sam demoed Apigee's new Twitter API test and debug console at the Twitter Chirp conference yesterday.

There's both a video and the screenr screencast for a clearer view of the screen below. Enjoy!

Introducing new Apigee features for API testing and debugging »

Our Apigee team is delighted to announce powerful new features for API developers for the Twitter's Chirp conference.

Today, Twitter API developers have a new API Testing console that provides a way to explore and share Twitter API testing results via URL with other developers (see and example here).

And the new API Debugger gives developers API message capture, record, and playback for any API that is proxied with Apigee.  You can share your debug snapshots with other developers via URL.

A couple other changes:  we've removed our 'private preview' invitation requirement - anybody can sign up and get started now.   And we've increased our free traffic limit from 10K to 50K per hour.

Finally, if you've used Apigee before, check out our new proxy creation flow.

For a detailed perspective, check out the Apigee blog for more on the evolution of Apigee.

Apigee’s Evolution »

Launch

Apigee was launched in late August 2009 based on a vision of "APIs Everywhere." We think that this decade will be dominated by APIs in the way that the late 90s were dominated by web pages. We saw how powerful APIs were for building some of the world's most useful services and how they were letting developers and providers innovate faster. We believed that the burning need for the growing API Economy was to enable powerful analytics on APIs - in the same way that Google Analytics powers the web economy. We focused on late-stage API providers who already had APIs in production and needed to measure them ASAP.

We attracted hundreds of users, learned a great deal, and made changes in our interface and architecture. We are indebted to the people who gave our service a try, sent us their thoughts on improving the service, and came back for more. We listened very hard and found we'd attracted not just providers but developers as well, and that it was now our responsibility to serve them well.

Learn

The most striking thing we realized was that as great as APIs are, they're still harder for developers to use than they should be.

Many people write about APIs as if the providers are the supply and developers are the demand, competing for the providers' bandwidth and attention. We think that it's exactly the opposite: developers are the supply, and providers are demanding developers' attention, competing for developers who they are relying on to make their service successful.

We believe that developers power the innovation that everyone is asking for. Making developers' lives easier becomes the shortest path to enabling the API Economy.

Simultaneously we saw that getting the attention of API developers was not easy. You can't go into a tech conference and say "raise your hand if you're an API developer" and expect anything but blank looks. However, specific APIs have very loyal communities. Going into that same conference and asking for "Twitter API developers" will get a very different response.

This brings me to the Twitter API. In a relatively short time this API has grown enormously and attracted tens of thousands of developers who are recognized as collaboratively growing Twitter's presence. It's a rich and well-designed API, and a great one to apply tools to support in testing, debugging, and analysis.

Launch @Chirp

So we've dramatically expanded the Apigee platform. Today we're launching two new tools specifically for developers: the API Console and the API Debugger.

The API Console enables developers to learn the structure of an API, send and received test messages, and share snapshots of those messages with other developers on forums, in email, or anywhere a URL is welcome. For our Chirp release, the API Console supports the Twitter API only, including all of the methods listed at http://api.twitter.com including basic authentication and OAuth (the technology under the "sign in with Twitter" feature on so many sites). We'll continue to expand our support for Twitter over the coming weeks and months.

The API Debugger lets developers drop into calls to any API - including their own - and record all of the requests and responses. These can be filtered out to isolate messages from a specific IP address, those that include a particular header, or all messages that generated errors (400+ return codes). This means no mysteries about whether messages were sent or not, no questions about the parameters or the formatting of the response - it's all displayed on a web page in a readable way. This should greatly reduce the effort required in building and launching API-driven applications.

We've also evolved our previously release tools for API Analysis and API Protection. API Analysis has gained a geolocation report to show where in the world API calls are coming from and a "most popular methods" list for a given API. Appropriate to the Chirp launch, we've also added reports for the number of Tweets and Retweets sent by a given application if it's using the Twitter API. For API Protection we've quintupled the rate limit for our free platform - now supporting up to 50,000 calls per hour per API used by a given application.

Finally, we've learned enough from our users that we are ready to drop the "private preview" from Apigee's service and make it a public beta. If you're interested, sign up and get rolling for free.

Thank you to all our users for their support of Apigee so far, and a special thanks to the Twitter team for their help and inspiration in bringing in the new phase of our service. We think developers deserve "a better way to API" and we are working hard to make our API platform as useful and as fun as possible.

Apigee team members will be at Chirp - both at the conference sessions and Hack Day. You can send a message to @sramji or @earth2marsh if you have feedback or want a hands-on demo. We look forward to seeing you there!

MotorMouths: An API Provider that Reviews Cars. »

Buying a car is stressful. Getting access to useful, trustworthy information helps. MotorMouths.com pulls together the best reviews for all new cars in the market today. Mashable described them as, "The RottenTomatoes of car reviews." To do this, MotorMouths gathers thousands of reviews from leading critics and then calculates an overall score for every car, based on the average of individual critics' scores. So, for example notice the critic's versus the average score for the Hyundai Genesis:

MotorMouths uses Apigee to gather stats for their own API, so we asked them to share their experience and insight:

What insights into your app or other benefits have you gotten from working with Apigee?

Jeff Smick: We get a much better sense of when and how our affiliates are consuming our API. We can gather a lot of data with Google Analytics on the frontend. Without Apigee gathering data about API usage would have required us to implement our own stats system. As a very small, fast moving, bootstrapped company we wouldn't have had the time to do so.

What new Apigee feature would you most like to see from Apigee?

Jeff Smick: I'd like to see the error code in the error reports. Apigee provides a good overview of which urls are producing errors, but no breakdown as to what those errors are. We use error codes to let API consumers know that they're unauthorized or doing something bad. So it'd be nice to separate those from the true errors.

What mashup or app do you admire that uses the same API(s)?

Jeff Smick: I've always been a big fan of Twitter's API. They've been doing a great job from day one. We also consume the grader.com APIs (twitter grader and blog grader) which are easy to use.

The next time your shopping for a car and want to do some consumer research, do it the fast and simple way by going to MotorMouths.

Free vs. Open API: Is there a difference? »

"Open and Free" are often used interchangeably when talking about APIs.  But if you look at in terms of the "API Economy" -  open and free API are orthogonal.   

'Freedom' in the cloud is more typically discussed as your rights to move data from one service to another.    Application portability has also been raised as important but current application models are both early in their maturity curve and vary greatly.
 
Open and free are both crucial attributes in order for a market economy to grow.  There are many aspects of cloud computing but for the developers and users of cloud services, the atomic unit of the cloud is the API.    

The link between openness and economic growth is a deep subject that may provide clues for how to build a better cloud –  this paper makes me think a company’s APIs might represent the “export goods in which it has a comparative advantage”.

In the software industry open platforms have typically outperformed closed platforms in the long run due to the economies that develop on top of them, cementing those platforms’ place in a range of markets.

Open APIs are:

1)      Openly documented
2)      Available via self-service (i.e. developers can sign up and get a key on a website)
3)      Using open technologies (SOAP, REST, RSS)

A good example is bit.ly - a simple URL shortening service that also lets you see how many people have clicked on your shortened version of that URL.  It’s really useful if you want to both project important articles on the web and understand the reach of your projection.   Is it an Open API?

Test 1: Bit.ly's API is openly documented (here)
Test 2: It's available via self-service.  You can get an account and a key right away.
Test 3: It uses open technologies: It's a REST API (granted, it is a mix of verbs and nouns)

Open APIs lead to 3rd party innovation

Tweetdeck users put their bit.ly API key into Tweetdeck, which automatically uses their bit.ly account (indicated by the API key) to shorten URLs that are typed into tweets.  It made Tweetdeck better and probably increases the traffic to bit.ly.
 
This could also work on the iPhone application of Tweetdeck but it’s not yet implemented in the version I have.  Many iPhone applications work use one or more cloud APIs that provide access to services in a clean, machine-friendly way.

From the efficiency of innovation perspective, keep in mind that Twitter most likely never contacted Tweetdeck to use their API, nor did bit.ly (as far as I know).  The Tweetdeck guys simply built a killer application that uses those services via APIs rather than scraping their web sites.  In the last month, Tweetdeck also added Facebook and MySpace support via their APIs.

Open APIs lead to innovation, efficiency and reach through designing your core business service to be “remixed” is found through APIs – Tweetdeck users got new value through the app they like and Facebook and MySpace got a new stream of user-driven content, all without sales or business development teams engaging at the outset.

Next time: Free vs. Paid APIs...

(Sam Ramji is the VP of Strategy for Apigee.  Previously, Sam drove many of Microsoft's Open Source Initiatives)

Turbocharged Twitter Followers : A killer tool using Yahoo Pipes and YQL »

Who needs Twitter follower notifications clogging up their inbox? Why doesn't Twitter provide an RSS feed? And why isn't it easier to identify spam-followers?

Turn off those pesky emails and get your updates in RSS with Twitter Follower Notifications.

This tool uses Yahoo! Pipes to create an RSS feed that gives a ton more info about your new followers, like:  

-Their bio and location
-Their last five tweets
-Whether or not they follow you, too
-Whether their updates are private or public

Bonus: you could use this tool for any Twitter user, which could be useful for seeing whos following friends or other 'persons of interest'.

Walk through the code and see how amazing Yahoo Pipes is - instant mashups without the need for tools or a server, all in a beautiful UI that makes it easy to share, clone and build on other ideas.   And YQL makes the web a huge database for your app. 

Twitter Follower Notifications was built by Marsh Gardiner (@earth2marsh), who uses Apigee to give him traffic stats and alerts to help him better understand how people use this tool. He's also been very active in the Apigee feedback forum, pushing us to make Apigee even better. Thanks Marsh, we're listening!

How Apigee calculates API error rates (featuring Twitter Growl and the Detroit Tigers) »

Last time we showed how Apigee calculates API response time. This time we want to discuss how Apigee calculates API error rates.

Apigee provides charts and tables with API error rates by day, hour or minute with drill downs by API URL. 

The 'error rate' calculation itself is simply # of errors divided by # of requests. Errors include both any Apigee error (including those that you might configure) or any target error (such as a rate limit error from the Twitter API).

Let's show this with Brian's Twitter Growl mashup.  Twitter Growl provides alerts on topics using Mac's Growl alerts.   For example, Brian gets alerts for Tweets on Detroit and Michigan news.

Brian monitors Twitter Growl API activity using Apigee.  You can see he had some stretches in mid August and early September with high error rates.  

You can also drill into the days or hours where these spikes in errors occurred, such as this 3 day period in September - high error rates on a single IP that was being rate limited by the Twitter API.

What was happening over this time?   The Detroit Tigers - in a pennant race for the AL Central -  were swept in a 3 game series with the Royals, resulting in rate-limit busting volumes of angst-ridden tweets from the Detroit Twitterati.

So there you have it.  The Tigers are behind all the errors.   (not sure we needed Apigee to tell us that.) And they still have a 4 and 1/2 game lead, FWIW.

If you'd like Twitter Growl to keep you up to date about the Tigers or any other topics on Twitter, thanks to Brian - his version is a fork of a project by visnup on github

Protect your API: Twitter’s Denial of Service Attack and API Security »

Lots of news on the Twitter attack last week. There’s general consensus that it was a distributed denial-of-service attack (DDOS) that it targeted a particular account.

DDOS attacks are tricky things. There’s no one technique, product, or protocol that will stop them. That’s what makes them so nasty. To defend against one, a company needs to be able to quickly take countermeasures at all the different levels of the protocol stack, including firewalls, routers, load balancers, and even in the application itself. And even more importantly, you need to have experienced, well-trained operations people who know how to quickly identify an attack and come up with a way to minimize its effects.

When a DDOS attack affects a web service – and there’s an good chance that the Twitter attack made use of the Twitter API – then proper traffic management at the API level is an important part of this stack of protection.

For instance, if a DDOS attack targets a particular user account, or API key, it may be possible to block those particular requests before they get to the back-end application servers, so that the effect of the attack is limited. Or, if the attack always sends some of the same parameters in an HTTP query string, or in the body of an HTTP POST, it may be possible to detect those patterns and reject the requests, again before they reach the back-end application servers. In the worst case, it may even be necessary to shut off access to a particular web service or web service operation, allowing other services to function as best they can while the attack is neutralized by other means.

In other words, just as it may be possible to stop a DDOS attack by blocking a range of IP addresses, or redirecting certain URLs, or configuring traffic shaping in a router or an ADC, it may sometimes be necessary to stop a DDOS attack closer to the application level. The ability to inspect requests at the web services level and take action based on the request content gives an operations team one more weapon to use against an attack.

The Twitter API already has a usage quota on each user account, a rate limit on each IP address, and extensive caching. Imagine how much easier a DDOS attack might have been if they didn’t even have those things.

-Greg

(Greg Brail is the CTO of Sonoa and writes on API Security topics such as API Keys, OAuth, and API Security Recommendations.) 

Tech Talk: API Security and Threat Protection »

Greg recorded a few whiteboard talks last month - this one is a good summary of recent posts on API Keys, API security recommendations, and OAuth best practices. 

(And here are some of our high-level API security features if you are looking into this.)

Don’t roll your own: API Security Recommendations »

Let's boil up the examples and common pitfalls from our last two entries on API Identity and Authorization and more API security choices.   

Use API Keys for non-sensitive data (only): 

If you have an “open” API - one that exposes data you’d make public on the Internet anyway -  consider issuing non-sensitive API keys. These are easy to manipulate and still give you a way to identify users. Armed with an API key, you have the option of establishing a quota for your API, or at least monitoring usage by user. Without one, all you have to go on is an IP address, and those are a lot less reliable than you might think. (Why don’t web sites issue “web site keys?” Because they have cookies.)

For example, the Yahoo Maps Geocoding API issues API keys so it can track its users and establish a quota, but the data that it returns is not sensitive so it’s not critical to keep the key secret.

Use username / password authentication for APIs based on web sites: 

If your API is based on a web site, support username/password authentication. That way, users can access your site using the API the same way that they access it using their web browser. For example, the Twitter API supports username/password authentication, so when you access it using a Twitter API client like Spaz or TweetDeck you simply enter the same password you use when you use the twitter.com web site.

However, if you do this, you may want to avoid session-based authentication, especially if you want people to be able to write API clients in lots of environments. It is very simple to use “curl,” for instance, to grab some data from the Twitter API because it uses HTTP Basic authentication. It is a lot more complex if I instead have to call “login,” and extract a session ID from some cookie or header, and then pass that to the real API call I want to make...

OAuth for server-to-server APIs that are based on web sites:

If your API is based on a web site (so you already have a username / password for each account) and the API will also be used by other sites, then support OAuth in addition to username / password authentication. This gives users a lot of peace of mind, because they can authorize another site to access their sensitive data without giving that site their password permanently.

Use SSL for everything sensitive:

Unless your API has only open and non-sensitive data, support SSL, and consider even enforcing it (that is, redirect any API traffic on the non-SSL port to the SSL port). It makes other authentication schemes more secure, and keeps your user’s private API data from prying eyes – and it’s not all that hard to do.

Don’t roll your own!

If the above suggestions still don’t apply to you, then keep looking – between OAuth, OpenID, SAML, HTTP authentication, and WS-Security, there are a lot of authentication schemes, and each has its pros and cons.

So wrapping up API security in our series on 10 API roadmap considerations. Here are some suggested questions you may want to ask when putting together your API security roadmap:

How valuable is the data exposed by your API?

• If it’s not that valuable, or if you plan to make it as open as possible, is an API key enough to uniquely identify users?
• If it is valuable, can you reuse username and password scheme for each user?
• Are you using SSL? Many authentication schemes are vulnerable without it.

What other schemes and websites will your API and users want to integrate with?

• If your API will be called programmatically by other APIs, or if your API is linked to another web site that requires authentication, have you considered OAuth?
• If you have username/password authentication, have you considered OpenID?
• Can you make authorization decisions in a central place?

What other expectations might your customers have?

• If your customers are enterprise customers, would they feel better about SAML or X.509 certificates?
• Can you change or support more than one your authentication approach for diverse enterprise customers?
• Do you have an existing internal security infrastructure that you need your API to interact with?

Up next:  API Data Protection and thanks to Torbin H. for the photo.

Did Twitter check their API analytics? »

Last week Twitter added a new rate limit to the “verify_credentials” method on their API. As a result, a number of applications that expected to be able to call “verify_credentials” more than 15 times per hour stopped working. The regular Twitter rate limit is 150 requests/hour, so Twitter is essentially saying that “verify_credentials” can only be called once for every 10 other API method calls that are made.

According to Jesse Stay, Twitter made this change without any notice to their developer community because they “assumed (apparently incorrectly) that people are only using this method occasionally.”

Twitter does a great job with their API – they make it easy to use and handle huge volumes.  

But this is a great example of the need for API analytics.   It should be fast and easy for any API product or engineering manager to drill down into “how often is verify_credentials called,” or better yet, “how often does the average API user call verify_credentials,” or even, “who are the top users of this API method?”  

We all have Web analytics to help us make decisions for our websites - we need the same decision support for APIs.  

Do you need API keys?  API Identity vs. Authorization »

(This is part 3 in our series on "Is your API naked? 10 API Roadmap considerations")

We’ve seen very few API providers with a completely open API – almost all employ at least one of these:

• Identity - who is making an API request?
• Authentication - are they really are who they say they are?
• Authorization – are they allowed to do what they are trying to do?

Do you need them all?  Maybe not.  Some APIs only need only to establish identity and don’t really need to authenticate or authorize.

API Identity vs. Authentication - Compare Google Maps and Twitter  

Take Yahoo and Google maps – they are fairly open.  They want to know who you are but they aren’t concerned what address you are looking up. So they use an “API key” to establish identity, but don’t authenticate or authorize. So if you use someone else’s API key, it’s not good but not a serious security breach.

The API key lets them identify (most likely) who is making a API call so they can limit on the number of requests you can make. Identity is important here to keep service volume under control.

Then take Twitter’s API -  open for looking up public information about a user, but other operations require authentication. So Twitter supports both username/password authentication as well as OAuth. Twitter also has authorization checks in its code, so that you cannot “tweet” on behalf of another user without either their password or an OAuth key to their account. This is an example of an API that implements identify, authentication and authorization.

The “API Key” – do you need one? 

API keys originated with the first public web services, like Yahoo and Google APIs.  The developers wanted to have some way to establish identity without having the complexity of actually authenticating users with a password, so they came up with the “API key,” which is often a UUID or unique string. If the API key doesn’t grant access to very sensitive data, it might not be critical to keep secret, so this use of the API key is easy for the consumers of the API to use however they invoke the API.

An API key gives the API provider a way to (most of the time) know the identity of each caller to maintain a log and establish quotas by user (see the last section). 

Usernames and Passwords – again, see Twitter

With more sensitive data a simple, API key is not enough, unless you take measures to ensure users keep the key secret. An alternative is username/password authentication, like the authentication scheme supported by the vast majority of secure web sites. 

It’s easiest to use “HTTP Basic” authentication that most websites use. The advantage of using this technology is that nearly all clients and servers support it. There is no special processing required, as long as the caller takes reasonable precautions to keep the password secret.

Twitter simplifies things for their users by using usernames and passwords for API authentication.  Every time a user starts a Twitter client, it either prompts for the username and password to use, or it fetches them from the disk (where it is somehow scrambled or encrypted where possible). So here it makes a lot of sense to have the same username / password for the Twitter API that it used for the web site.

Usernames and passwords also work well for application-to-application communications. The trick - the password must be stored securely, and if it’s being used by a server, where do you store it?  If you are running an application server that uses a database, you already have solved this same problem, because the database usually requires a password too. Better application server platforms include a “credential mapper” that can be used to store such passwords relatively securely.

There is a lot we'd like to write about around security so we'll split this up into a couple entries.  Next time:  Session-based authentication, OAuth, SSL and WS-Security, and rolling your own.