API Best Practices Blog
Moving the needle: Example API metrics »
It's an old cliche, but it's been said that you can't move the needle if you can't *see* the needle. So frequently we're asked "what are good metrics to measure an API program?"
While individual metrics are important - it might be as much about the 'process around metrics.' Or..how metrics are evangelized and used to drive specific parts of the API product development pipeline. Specifically:
1. Get early buy-in on the 'top 3' - strong API product managers often focus in on 1-3 top level 'strategic' metrics and get early, wide agreement from all parts of the extended team - the sponsoring exec, PM, engineering, BD, and operations. If different stakeholders are measuring success with different metrics (say number of developer sign-ups vs. API traffic vs. revenue) this can pull resources in different directions.
2. Track against realistic projections. Set expectations early by modeling anticipated results and then track actuals against this estimate. For example, pick a 'comparable' or competitor's API to guess developer portal traffic, then model the expected developer sign-ups and conversions (for example, 10% of visitors might ask for a key, 20% of them might built an app, 10% of those apps might drive ongoing traffic, each of those apps might drive a certain volume of traffic, and so on... )
3. Publish a weekly dashboard, religiously. Proactively call out how product updates and community activities do or don't move the needle so you can quickly adjust tactics and think of new ideas that might move the needle.
4. Create a metrics 'pipeline' - How do different metrics diagnose how each stage of the customer conversion process is working? For example, developer portal traffic might be a good metric to measure the marketing guys. (that is, they might be responsible for getting developers *to* the portal.) But whether or not a developer converts to ask for a key and then converts into an active API user might be a measure of how effective the PM process is working to create a product that developers want to use. User experienced bugs can measure development and product QA effectiveness, and so on..
Here is an example of a metrics pipeline that we recently discussed with a customer.
| Category | Example Metric |
|---|---|
| Awareness (measure of marketing effectiveness) |
-Developer portal traffic: Unique users, page views, and engagement (PVs/UU) |
| Signups (measure of portal messaging effectiveness) |
-Registrations (developer keys issued) |
| Adoption (measure of product fit) |
-Active developers, partners -Applications (number, by app type, geo, partner 'tier') -App end users (such as mobile app users) -Traffic: volume and % API vs. non-API -Developer retention (active developers lost) |
| Quality (measure of dev process) |
-User experienced problems (errors returned) -Bugs reported -Critical situations (P1 bugs or blocking bugs) |
| Community (measure of customer sat) |
-Community members -Community forum activity and engagement -Number of very active members -Net promoter score |
| Financial (measure of business model fit) |
-Revenue -Cost of data served (if licensed) -Profit and margin -Market share |
(Thanks to seenoevil for the photo)
