We’re huge fans of Twitter – both as users and observers of the impact the Twitter API. Since it was opened, Twitter’s API has shown how providing access can unleash huge waves of innovation – giving rise to clients, data businesses and much more.

So it’s very sad to see the news today that the company will be cutting off their firehose partners. This will directly affect third party data re-sellers (DataSift already responded) and end customers. However, it is likely to affect not only them, but also be very negative to Twitter in the long run.

Cutting distribution

There are no doubt business drivers for the decision which are not visible on the outside. We also have no insight into the precise business relationship between Twitter and the two partners affected (perhaps the reason for the change lies there and not in a strategic decision – though this seems unlikely). However, whatever the reasons, from a platform perspective this looks to be a very ill judged move. Why?

Because Twitter just lost a critical innovation layer.

More specifically, Twitter’s current and future data customers just lost a lot of potential value. So did Twitter. To see why this is so means thinking about the platform potential Twitter has and how it is realized.

The move is basically signaling that innovation on the firehose will now be in hands of:

  1. Twitter itself: working closely with its customers.
  2. Customers: ingesting the data and working with it. (The customers being analytics providers and -presumably – individual businesses.)

While it makes sense at some level to want to be closer to the consumers of data (that’s valuable and laudable from a product perspective), removing other channels is an innovation bust. Twitter will no doubt do a great job on a range of use-cases but it’s severely damaging not to have a means to enable full firehose access for others. Twitter should really be expanding firehose access, not restricting it:

  • The Twitter and Gnip teams have awesome engineers (we know plenty!), and they’ll continue to produce excellent product and services. However, with all the will in the world, they will not be able to cover all use cases. Business constraints will push them to focus on the most lucrative and largest. Niche cases just wont be served.
  • Few organizations have the capacity to handle the full twitter firehose or even meaningful slices of it. This means a further a loss of value for the smaller use cases.
  • Many of the most interesting use-cases involve co-mingling (public) data from Twitter and other social sources: this is very hard to do well and just got a lot harder.

Although some of these losses may seem minor, given Twitter’s vast data flow it’s very likely that some of these niche uses could be hugely valuable in the future.

As in B2C client shutdowns, innovation will slump

With the decision to restrict access to client building in 2012, Twitter did gain more control over the client ecosystem. However, looking at the state of Twitter clients today there’s little new – it still feels like 2012. The Twitter apps are neat and slick, but there is nothing radical. Would there have been something radical if API access was open? We’ll never know.

In some senses, the firehose decision is worse than the client decision. Since with the client decision there was a least a means to retain access for high volume clients and a means to achieve that status.

The firehose decision appears to shut off all third party access, so the slump will probably be even bigger.

But Twitter owns the firehose – surely the grabbing more of the pie is better?

Yes, this might give Twitter a short term boost. It may even improve Twitter’s own data products (although it is hard to imagine they could not have innovated more without this move – the bottleneck is likely talented engineers, not customer access). However, it is economics 101 that the move will decrease the rate of growth of the overall amount of value created on the Twitter firehose. Less parties will now be building tools to realize that value. Less experimentation will take place (since there is no business model for gain). All that decreases value to customers.

Worse, it cuts out opportunities for market building. If increasing numbers of customers get value from the Twitter data stream via parties such as DataSift, the competitors of those companies will be willing to pay increasing amounts to have access to the same data – some of it direct from Twitter. Twitter makes money on both channels – from the resellers and from its own customer base.

This is a fundamental principle of platform economics: Grow the pie, don’t try to eat all of the pie.

The Negative Signalling of moving up the stack

Twitter’s press release names analytics partners as its customers for the firehose. Arguably, this is a retrenching of the boundaries and hence “not such a big deal”.

However, a secondary effect of this announcement could be to put exactly those customers on notice. Without a clear boundary on what Twitter sees as its domain, there is a risk that social media analytics will be the next area the company moves into. That fear itself has the potential to crimp innovation.

If this turns into a long term strategy for Twitter, the best investments one might make is into on premises / license data analysis software. Ultimately end customers will be having to deal with the firehose and build out their use-cases. Since this will be so hard to do, few people will attempt it – putting value created on the platform into reverse gear.

Platform Thinking

Building a platform ecosystem that genuinely works requires three types of parties – the platform provider (Twitter in this case), the recipients of the value (customers) and third parties (that innovate in and around the platform for the benefit of the customer). Anytime one of these is shut out, there is potential value left on the table – in Twitter’s case, it’s likely to be a great deal of value.

Twitter’s calculation may well be “we can make more money with direct control, since we own the asset that people are trying to access (tweets)”. That may well be a correct assessment in the short term – but it is fundamentally bad for customers: less innovation, less value delivered. In the long run that is opportunity lost for Twitter also.

We have no skin in the game that’s affected by this decision, but it’s sad to see from the one-time platform visionaries.

In 2008 we were saying “do as Twitter does” in 2015 we’re saying “absolutely don’t do as Twitter does”.

[UPDATE] a nice view from the trenches seeing the other side from Tyler Singletary at Klout (@harmophone): How To Be A Platform: Making Tough Partnership Choices.