Apigee on the Road »

Mobile Apps 101: Key patterns you need to know (video & slides) »

Thanks to all who participated in the Mobile Apps 101 Webinar last week about "designing apps that people want to use." The video and slides are below.

Check them out for an introduction to some key patterns for mobile app development - repeatable patterns that represent functionality for the front and back-end of mobile apps.  Thanks @edanuff, @gbrail, @timanglade.

We'd also love to hear your thoughts, insights, or questions over on the api-craft forum.

Six Design Patterns for Highly Successful Mobile Apps »

For years, building web apps has required server-side code and a database. That was great for browsers, but then mobile apps came along and changed everything. Today apps appear on any number of devices, including browsers, and the same app needs to run seamlessly on multiple devices. 

Apps are fundamentally different from their web application counterparts. Traditional web applications run on a monolithic, server-side stack, which delivers content in the form of web pages. For the most part, browsers simply display this content. When a user interacts with a web page, requests are sent to the server for additional content.

In contrast, mobile apps run on a client. Instead of deploying program logic to a back-end server, the logic is housed on the client. This fundamentally changes the dynamic between client and server, making the client no longer merely a display mechanism.

Additional considerations for the development of mobile apps come in the form of the resource constraints imposed by mobile platforms. Mobile apps need to be as lightweight and agile as possible. In other words, you simply wouldn’t run a LAMP stack on an iPhone.

Another challenge for Mobile app developers is dealing with the cumulative traffic that can be generated by even a modest number of clients.  For example, many mobile apps connect to the cloud regularly throughout the day to perform authentication, upload and download data, or to perform geo-queries against a user's current location.  This can result in what amounts to a distributed denial of service attack (DDoS) against an underpowered service architecture.

An added consideration is the data-warehousing infrastructure necessary to deal with all the information being collected. For mobile apps to deliver cutting edge, rich interaction on mobile devices they must be supported by equally rich services and data storage mechanisms.

Enter the cloud.

By hosting rich services in the cloud, client apps can deliver the interactivity that users desire while maintaining a lightweight footprint. With the emergence of mobile PaaS (Platform-as-a-Service) solutions, the days of app developers having to find another developer with a different skill set to help or spending half or more of their development time building services, may soon be gone.

Everything from storing users, groups, roles, single sign-on authentication, social interactions, feeds, queries, connections between users and objects can be provided by a PaaS solution. This leaves the developers to focus on writing cool apps and not on the server code.

So what services do mobile developers need?  We think these 6 patterns are a great start.

1. User Management
Who are you? Who do you know who has the same app? Who do you know outside the app?
One of the most fundamental needs of app developers is user management - straightforward ways to handle users’ (customers’) access control, groups, roles, subscriptions, upgrades, reminders, and so on across multiple devices and apps.

Internet ID and social connectivity also relate to user management. Developers are being rewarded for building identity into apps - from showing pictures of users to bridging to other identities like Twitter and Facebook.

2. Connected & Social Interactions
Who do you follow? Who are your friends? Who will you invite?

In-app and extra-app interactions are all core to mobile apps and devices. For example, an app might bridge out to do a notification or invite a user to join.

In the past, this would have required a generous helping of custom server-side code. Development platforms that provide activity streams, linked profiles, and so on facilitate the development of connected and social interactions.

3. Activity Streams
Activity streams are a kind of static query on moving data. They take connections and turn them into activities, adding a social layer to data and opening up real-time collaboration.

Popular activities vary depending on context but activity streams enable activities such as status updates, check-ins, comments, or other broadcasts to fellow users off of any object or off of multiple objects.

4. Sync Content & Data
Apps need to be able to share data with users, and import, export, and sync application data with third-party applications. The ability to sync data across devices allows developers to create apps, which provide consistency of experience. Think about your favorite “shopping list” or “to do” app that syncs across your laptop, your iPad, and your mobile phone!

5. Location, location, location
Geo-location or Location-Based Service (LBS) apps attach real-world locations to mobile devices and are one of the fastest growing types of apps. You can establish when and where you receive information, track your peers and friends in real-time by location and proximity, find and provide information about things and friends near you.

6. Analytics
App developers need to be able to monitor usage and analyze data to drive both app functionality and business intelligence. Important services include real-time activity processing, behavior tracking and targeting.

Analytics on individual apps are powerful and useful for decision-making about functionality, usage, troubleshooting problems, and so on.  But it’s a game changer when the insights from aggregated data - possibly from multiple apps a developer has created - allow developers to turn what they’re doing into a business and evolve it over time.

I need an API: Where do I start? »

One of the most important questions that enterprises ask us has changed from "Why APIs?" to "I know I need an API; where do I start?"

Start where you have the most amount of pain - where the business drivers are. If your pain point is in meeting demand for mobile and social apps, then start there - with an internal API strategy. If you need to innovate with partners to deliver on a backlog of business development opportunities, start with a partner API. If you need to inspire a broad community of app developers to innovate and create growth and new opportunities, start with an open API.

The lines between internal, partner, and open blur a little more every day as companies innovate at the edge of their businesses. And what starts as one flavor of API can become a different opportunity down the road. We've seen it a bunch of times - an API team learns from internal and partner projects, develops the know-how and courage to open an API to the world of innovative developers who can take the API and the business in creative and valuable directions. We've seen companies have huge success in the opposite direction - businesses starting with Open APIs and realizing that the bulk and the most valuable innovation is coming from partner and internal collaboration.

As long as you start where the business pain is -  you're starting in the right place.

Unlocking business opportunity with APIs and mobile strategies »

I sat down with Brian Gracely (@bgracely) and Christian Reilly (@reillyusa) of Bechtel recently to talk about the evolving API and app economy. We had a terrific discussion about the adoption of mobile devices and how open access to data is unlocking tremendous new business value. Check out the Cloudcast.net podcast.

Christian gives us terrific insight and first-hand knowledge of why it makes sense for a big traditional enterprise to use APIs.  It starts with mobile devices driving the strategy. Mobile devices have evolved from toys in the enterprise to real business tools for people at every level in today's enterprise. We see the same story play out for all kinds of enterprises - from large-scale construction, to retail, to healthcare, to Telco, and others. Data entry and collaboration on job sites and in offices around the globe are now driven by apps on smartphones and tablets.  This increases immediacy and accuracy of data capture and project management, reduces cost and risk, and improves compliance.

In a pre-API world, enterprises were mired in large applications that were bound behind physical and digital walls.  As mobility grew exponentially and with it different expectations from consumers took hold, these large, legacy, internal applications simply couldn't cut it from a usability or user acceptance point of view. An application accessed from a tablet or smartphone through a browser or remote terminal is not  an acceptable experience, as people get used to cool apps for all aspects of their lives on their mobile devices.

In our conversation, I think Christian aptly captured the new environment as one in which "people don't want apps - they want information."

This realization drives a whole different mindset. In addition to the demand for a different experience on mobile than on desktop, enterprises also have to think differently about how to provide access to the information people needed. APIs were the obvious solution. Enterprises can expose read/write access selectively with APIs and they don't have to replicate the entire app.

Adopting an API strategy allows enterprises to do things like focus their investment on making robust APIs, which makes it easy for developers to build apps. Enterprises can leverage the creativity of (in-house or external) developers to build light-weight UIs (which can be very simple interfaces) that allow users to get and put data from anywhere and from numerous devices. They no longer have to be sitting in an office close to the "enterprise applications" and systems of record like ERP or billing systems.

In this scenario, IT became responsible for exposing the business as a platform, driving a fundamental change in the speed at which business can be done. Instead of IT projects taking years, they were now taking months. As a result, enterprises experience higher level of fluidity and flexibility - much more rapid and experimental development cycles and take advantage of natural feedback loops. This all directly relates to an enterprise's revenue and profitability.

Simply, this means that the constant underlying platform is the API. Apps, while critical for the last mile, are a more fluid component, available for rapid change, augmentation with other apps, or disposal.

There are many examples of companies who understand how to apply and invest in APIs in order to support the type of consumer and partner engagement that's the mark of the economy we're seeing in front of us, and they are being directly rewarded in revenue and profit.

Usergrid is now part of Apigee »

I'm excited to announce that Usergrid is now part of Apigee.

Almost every mobile app needs back-end services to store and manage users, data and events. But powering mobile apps from the cloud is a hard problem, requiring a rich set of easy-to-use features for managing users, delivering applications objects and data, and analyzing usage and tracking key metrics.

Engage customers where they are - at the edge of your enterprise »

Wayne Gretzky has this great quote -
A good hockey player plays where the puck is. A great hockey player plays where the puck is going to be.

A few days ago I had a great conversation with a big retailer (and with many other customers since then) about engaging customers on the edge of the enterprise and think it's worth sharing.

Today our systems of record in enterprises are ERP and billing systems. That will continue to be the case but the problem is that all these systems of record represent where the puck has already been - what has already happened.

We'd like to predict where the puck is going to be. Today, lots of information can be gathered and analyzed quickly, and in real-time. So you can know what your customer is doing right now. Based on what they are doing now, or have just done, you can predict what they will do, or want to do next.

We're talking about figuring out how to play where the puck is going.
Imagine, you are a leading retailer. A customer walks into your retail location. If they check in (Foursquare, Facebook, Google+, . .  on their mobile device), and you have their credentials, you can know their profile and recent history.  

Where have they been? Did they just leave Walmart or Tiffany? Their spending patterns will obviously be different.

Can you know from their Twitter stream whether they were grocery shopping or anniversary shopping? Knowing this, and correlating it with what they've already bought from you, gives you a perspective and clues to their intent.

Imagine if you can make this information available to a sales person on the spot? Now you've provided that sales person a context in which to interact with the customer. They're equipped to provide a great and customized experience.

The only way to make this happen is if your enterprise is interacting with all of the different streams of data that are available and correlating this data with what you already have inside the enterprise.

Use it all - data in the core enterprise, from outside the enterprise and most importantly from the edge of the enterprise (mostly mobile apps) - to make context-sensitive interactions for customers.

That's what engaging customers on the edge is.

“Mobile APIs - Optimizing APIs for many devices” Webinar - video & slides »

Thanks to all who participated in yesterday's webinar - " Optimizing APIs for Many Devices"  with Sam @sramji, Brian @brianpagno, and Greg @gbrail.

Here are the video and slides. We'd love to hear more of your thoughts and questions; please join the conversation on the api-craft forum.

Not serving JSON AND JSONP? Then you’re doing it wrong! »

If you've used an API recently, you've probably seen that the popular APIs out there support JSON. JavaScript Object Notation is a standard defined a while back by Douglas Crockford from Yahoo. It uses a subset of the JavaScript syntax to simply and effectively describe an object.

In the last few years, JSON has taken its place alongside XML as the de facto way to describe API data. Today's leading APIs support JSON in addition to XML, and an increasing number support only JSON.

JSON is popular because it's simple. Programming-language objects map to and from JSON in a straightforward way that everyone can understand. You need a "JSON parser" to convert JSON into an object (unless you're working in JavaScript) but you don't need to know much about it other than how to make it go.

If you are thinking of building an API, JSON support is critical. Here's why:

JavaScript. JSON is JavaScript. That is, a "JSON object" is literally a small fragment of JavaScript that represents an object and its sub-objects. That means that creating a real JavaScript object based on some JSON text is simple and fast. Web programmers love JSON.

JSONP. This lets JavaScript running inside the browser invoke APIs that reside on a different host on the Internet. This doesn't sound like a big deal but it is actually huge because all browsers implement a "same-origin policy" that otherwise makes this impossible. JSONP is hard to implement, but libraries like jQuery make it easy for the client if the server already supports it.  If you're not serving JSON AND JSONP you're doing it wrong!

Smaller. Smaller is better, especially in the mobile environment, and since JSON doesn't "say" every field name twice like XML does, JSON output is a lot smaller than XML.

Less complicated. JSON is free of namespaces, attributes, multiple "text" nodes, and other complexities of XML. The result is that JSON parsers exist for every language, they're small, and they're fast. Furthermore, if you need to write your own, it's not complicated. The same goes for security -- all that's necessary to prove that a JSON document is valid JSON is a simple regular expression check, which is easily available in nearly every programming environment.

Tools. An increasing number of tools support JSON. JSON support is not ubiquitous yet, but at the rate JSON is gaining it will be soon.

Many APIs now support XML and JSON - like the Twitter API, where JSON is the default. Some APIs support only JSON (like Foursquare's V2 API).

But JSON isn't  for everything. Next up: Why XML isn't dead yet!

Using your existing systems to create new value with APIs »

Companies need to wring value out of their investments; find new ways to serve customers or create new value with their current systems, products, supply chains and partnerships. 

I keep going back to water analogies.  My house has a plumbing infrastructure that was paid for a long time ago.  I recently found new value in that infrastructure, an automatic sprinkler system.  Again the system reminds me of an API.  Through this one common interface I can connect many endpoints (sprinkler heads) and my yard has never looked better.  I have some endpoints that cover the lawn, some that concentrate on shrubs, others take care of the plant beds.  The point is that I tapped in to my existing water pipes and found new value in a 35 year old system. 

Can you do the same with your enterprise systems?

Let's take a common strategy like a mobile device strategy.  First we have to understand what information customers need when using a mobile device like a smartphone.  Certainly location and product come to mind.  Maybe information about their account or orders.  These little gems of information can be exposed through the API interface and carried to many different endpoints; today a smartphone, tomorrow a tablet PC or connected car.  Let's call these 'functional APIs" since they carry a function to the user through a mobile device. 

The next step to realizing a functional API is to map the data elements and functionality of the new API to their location in the existing systems.  These data elements may map to databases, ERP systems, other internal services, etc.  In fact, a single API may map to one or more internal systems.  Any holes that are discovered will have to be filled. 

For example, if you are designing an API and define features or data that aren't available through existing systems, you will have to provide that new system or new data set.  But, in the interest of embracing the legacy systems, you should be able to reuse the majority of what you have by extending it outward through the new abstract API.

Leveraging what you have through an API can reduce your development cost, give you speed to market and also give you more flexibility in the future to integrate with more devices and partners.  Leveraging my existing residential water system yielded a "yard of the month" award, perhaps this strategy can beautify your business as well.

Next we'll talk about the details of integrating with existing systems.   And ots on this in our recent whitepaper "APIs: A Profit Interface - using APIs to grow your business.

Do you need an SDK for your API? »

In our last couple API strategy workshops, we've had good discussions on a common question for API providers - do you need to offer an SDK (software dev kit) for your API?

In this context we defined an SDK as going beyond offering an API to include platform-specific code that developers use in their apps to invoke API operations, also including the source code, and documentation that developers might need. Here's what we came up with:

Reasons to consider an SDK:

Speed adoption on a specific platform - for example Objective C SDK for iPhone.  Lots of experienced developers are just starting off with objective C+ so an SDK might be helpful.

Simplify integration effort required to work with your API - If key use cases are complex or need to be complemented by standard on-client processing. 

An SDK can help reduce bad or inefficent code that might slow down service for everyone.

As a developer resource - Good SDKs are a forcing function to create good source code examples and documentation.  (See some good examples here and here. )

To market your API to a specific community - you upload the SDK to a samples or plug-in page on the platform's exisitng developer community.

Reasons you might not need an SDK:

If your API is designed for adoption -  standards-based, well documented, etc.. developers may be able to get rolling without a client SDK.

Resources - it's tough to do a SDK for each platform you target, you'll have to pick carefully. Also, it can be time consuming and expensive to create SDKs (tip: maybe packaging up internal samples or test cycles can kickstart things)

Maintenance and versioning - convincing clients to upgrade to the newest version can be rough. Also, capturing the "local flavor" of each language you support can be a challenge.

Complexity -On each platform there might be use cases you don't expect, like keeping application-level secrets off clients, debugging, etc.

More is not necessarily better - fewer, more well-documented, code-level samples can often be the best resource. Facebook provides Android, C#, iPhone, JavaScript, PHP, and Python libraries (all documented differently), but Twitter supplies none. Facebook has more resources but still struggles a bit at times.

Our view: whenever possible, forget the SDK, and spend the time making your API better and more useable, and give good API resources such as example code.  

Agree or disagree? 

Beyond the Browser: The API web »

Maybe it was because I was reading it on my new iPad, but really enjoyed this post by John Doerr and partners on TechCrunch last week.

Was struck by the phraseology 'beyond the browser' and that we're quickly moving " beyond web sites limited by browsers… to interactive, connected applications with incredible simplicity, speed, and fluidity.... that will transform everything." 

Couldn't agree more, especially when we consider the next 5-10 billion devices sold (via Mary Meeker's amazing Web 2.0 deck below, slide 32) will not be PCs with browsers but connected devices - from the iPad to your Xbox or Tivo that don't use browsers but instead connect to content providers with APIs.

We are moving beyond the browser to the "API web", where much of your company’s web traffic will come from a non-browser source.  Enabling the API web is our mission.

 

APIs for connected devices - the rush is on »

My favorite Superbowl ad was for Vizio HDTV - it was great to see about a dozen leading Web APIs showcased right up there with Beyonce.

Yesterday Michael Zimablist posted on the New York Times Bits blog about how the NYT’s content must now support a growing range of devices like the Vizio, from web-connected printers to mobile apps to the new iPad.

And the Wall Street Journal's Martin Peers asks if the iPad’s e-book store is the new model for the television industry, citing how Netflix is streaming to over 50 different devices to build new audience. Blockbuster is pursuing a similar strategy to deliver it’s movies to “nearly every connected device.”

So the rush is on… just like a decade ago when companies scrambled for a ‘www’ address, companies today are rushing to create APIs - making their data and services available to execute a multi-channel, multi-device strategy.

Next: what are some considerations for your API in supporting an ever-expanding number of devices?

Mobile patterns are different from Web patterns »

Mobile application patterns are different from web application patterns.  There are a consistent, discrete set of differences in how they access cloud services.  There are consistent reasons why they’re favored over websites as well, primarily based on implicit intent and purposive computing experience, but that’s a subject for a future blog entry. 

For now, let’s assume that like web applications, mobile applications use HTTP to access their services, but unlike old-school web applications, they use REST and SOAP as the basis of their service protocols.

Difference 1: Bandwidth is expensive

Bandwidth always costs two unique things in mobile applications – time and battery.  Jeffrey Sharkey has a great talk about battery usage and good citizenship.  In some areas and for some users, bandwidth also hits their data plan, which makes bandwidth cost real money as well 

Difference 2: Bandwidth is inconsistent

Disconnections are part of everyday life when using the mobile internet to access websites or cloud services. When your local cell area becomes overloaded with requests or your service loses track of where you are between towers, when you are in even a momentary cell shadow, your connection is gone.  If this is in the middle of a data connection, that connection is reset and has to start over.

Difference 3: Local processing matters

First, non-trivial requests for data from cloud services often results in large datasets being returned to the device.  These chunks can not only be hard to process, but may be more information than the user will bother to access.  A request that returns hundreds of row-equivalents worth of responses may be mostly wasted processing if the user is only going to glance at the first few displayed screens’ worth.

Second, local applications have differentiated access to devices – awareness of onboard camera, location, or other services.  They also have differentiated preferences about data.  For example, the iPhone operating system is fluent in XML processing, and many iPhone applications transmit XML dialects to their cloud services.  However, XML is more expensive to the iPhone than PLISTs (a JSON-like simple data format) – roughly 4-5 times more expensive in compute cycles.  Other mobile devices have their own variations based on operating system version and device services.

Difference 4: Welcome to the hit-driven app economy

Media has been a hit-driven economy for decades, with winners and losers being made and broken overnight based on the wisdom or madness of crowds.  With the fantastic potential of mobile applications and the inconsistent actual experiences, collaborative filtering and editorial selection are producing a hit-driven “Top 25” economy for mobile applications – the equivalent of being “above the fold” in a website.  It’s a steady climb to get in this elite area, but once your application gets to this point you might as well have been written up by Walt Mossberg or Slashdot – traffic, downloads, and usage of the cloud services backing your app will all surge dramatically.

Difference 5: Concurrent usage by millions of nomadic users

Just combining Difference 2 (inconsistent bandwidth) with the fact that most mobile connections use HTTP 1.0 means that many more connections are being made, dropped and suffer an expensive reset not just of application state but of the HTTP connection itself.  Adding Difference 4 (the hit-driven app economy) to this means that concurrency – including “shadow concurrency”, the load of the dropped and restarted connections – has an even bigger role in mobile applications than traditional web computing.

Solving for mobile application patterns in cloud computing

There are probably ways to solve each of these problems individually.  What we’ve seen with some of our key customers is that all of these can be solved by applying a cloud service controller to manage the connections between mobile applications and their cloud services.

With a cloud service controller in the middle of the application and cloud service interactions, they’ve done the following things:

• Compressed service request and response data by 6-10x
• Accelerated service response time 5-7x through intelligent caching
• Carved large service responses into chunks likely to be used by the mobile user
• Translated service responses into formats easily  processed by the mobile device
• Reduced total network airtime usage by 15-20x
• Reduced battery drain on the mobile device
• Reduced dropped connection experiences for the mobile application
• Scaled caching and response capacity dynamically to match growth and spikes in usage

In one example, a customer took the network request/response time from 17 seconds to 1 second, and took local processing on the mobile device from 17 seconds to one second.  This reduced total application response time from 34 seconds to 2 seconds – an acceptable, even exciting level of responsiveness for that application’s users.  This was all achieved in a few weeks without rewriting either the mobile application or the cloud service upon which the application depended.

They did this by taking our core product (Apigee Enterprise), writing policies that let it route, cache, accelerate, paginate, and format their cloud services.  Since Apigee Enterprise is available as an .ami (Amazon EC2’s virtual machine format)  we've deployed it as a cloud service that expands and contracts its use of computing resources to match the load.  This way they haven’t been caught unprepared when their app made the top 25 list on the iPhone App Store and their legions of new users had the same responsive application experience as the users who popularized it in the first place.  Finally, future devices and application platforms will be easier to support from a single cloud service through construction of new formatting or pagination policies that match the needs of those device platforms.

Mobile acceleration may seem like a standalone thing for Apigee, but really it’s an example of using policy to solve an application pattern challenge.  More on that – and policy-oriented programming – in a future blog entry.  There are many more domains of use for this approach in cloud computing.

Here's a video we posted today describing our Mobile App Acceleration service.

Mobile is the future of cloud, cloud is the future of mobile »

Mary Meeker’s Web 2.0 presentation made a strong case for the imminent boom on the mobile internet.  Some statistics that caught my attention:
·         Mobile internet users will exceed 500M human beings in early 2010
·         The mobile consumer device market will exceed 10B in the next 5 years
·         The current iPhone + iTouch user base is larger than Netscape’s base in 1999
·         More than 20% of the world will be on 3G networks by the close of 2010

This is a serious change in how people are using the Internet.  I don’t think of the Internet as synonymous with the Web; the Web is one pattern of applications, based on HTML and HTTP, that uses a common infrastructure – the Internet.  Both the iPhone and Android phones have a much higher share of Web and Internet usage than previous generations of mobile devices.  But are they really using the internet?  Or are they using the Cloud?
 
From my point of view, the Cloud is another pattern of applications – like the Web – that use a common infrastructure – the internet.  But just as Web application patterns are different from other Internet applications (like email, for example), Cloud application patterns have their own rich array of styles and requirements.  And what we’re seeing with mobile consumer devices is an explosion of applications that reside on the device and use the Cloud for access to remote logic and data.
 
(I promise not to capitalize web, internet, or cloud from here on out.)
 
So the future of the mobile internet is not the mobile web – resized, limited web pages rendered by a WAP server and discovered via Google and Bing – but the cloud.  What we’re seeing are the first generation of sites rebuilding their backends to be accessible via cloud APIs in order to give access to mobile applications that will drive business.  Some sites are on their second and third generation, having rebuilt their web presence to rely on the same cloud APIs that their mobile applications do.

Introducing Apigee: Freemium, Self-Service API Management »

   Today we are delighted to announce Apigee. 

Apigee is a website that provides analytics, protection and control for APIs.  If you offer an API you can use Apigee to understand usage, protect your app, and enforce API terms of use.   If you are a developer using APIs in a mashup, mobile or social app, you can use Apigee to get visibility into the actual service levels of the APIs you consume.   

Apigee is freemium and self-service.  You can start using it in minutes and Apigee's Basic service is free for under 10,000 requests per hour.  Apigee is built on the same Sonoa ServiceNet technology that our enterprise customers use for industrial-strength API management.  You can think of it as a freemium, simplified subset of policy framework available in Sonoa technology.

Beginning today we are offering Apigee in a private, invitation-only preview.  Anybody can sign up for an invitation here.   More details are on the Apigee blog.