What would you think of a retailer who kept some of their merchandise assortment in a locked backroom, inaccessible to potential customers?
And what would you think if that retailer’s customers were trying to pick the lock to that backroom to get access to that merchandise, to try it out and possibly buy it, or maybe feature it in their own store for sale on behalf of the retailer? Should the retailer ignore those customers or potential distributors? Put a more sophisticated lock on the door? Pursue legal means to get those customers to ‘cease and desist’? What if you are that retailer??
At Apigee, we are sometimes asked, “How do I know if I should open an API?” One sure sign you should have an API strategy is when well-intentioned developers are scraping your website to get at your data so they can do something interesting with it. That’s a clear indication you’ve got something of value locked away, and making that data easily accessible could unleash a torrent of innovation.
So when confronted with scraping activity on your website, what should you do?
1. Find out who the scrapers are and have a dialog with them!
Often they are your most rabid fans, customers or potential partners. Find out what they are trying to do and why. If the data they are scraping directly supports your core business, say a retailer’s item catalog or hotelier’s room information, you should consider making that data freely available, or even paying developers to take the data and generate customers with it. If the data does not directly support your core business, you may have a distinct data business that’s a derivative of what you do.
2. Take inventory of the data exposed on your website, and also take inventory of data you have that’s not exposed on your website.
Do you want that information in the hands of your customers or partners? How could it be used to build your brand, increase your customer engagement, further your company goals?
3. Find the Business Model
If it seems like there is an opportunity to extend your business or create a new business channel, get working on an API strategy immediately! There are many examples of companies that have built fantastic distribution channels or complementary businesses via APIs…. Facebook, Twitter, eBay, Salesforce, Netflix, and many more. Done right, an API lowers the barrier to entry for developers wanting to build services off of your data… just the opposite of the challenge scraping poses. While there is nefarious scraping out there, scraping might just be opportunity knocking.