This week, I attended the Velocity Conference, a conference focused on “building a faster, stronger web.” Many of the talks focused around gathering metrics, including RUM, synthetic performance data, and system health monitoring. The data points of interest go beyond simple web traffic and hit counts and focus on how users engage a site, how the experience and speed of the site affects conversions, and quickly diagnosing issues when they come up.[1]

Everybody Builds The Same Thing

“At [Acme], we built Widget X […],” was a phrase I heard innumerable times. Web development teams are faced with very similar problems, regardless of the market their business is in. They’re gathering data to show the business how visits convert to sales, as well as how the site performs across the combinations of browser, geography, and connectivity that constitutes their visitors.

In most cases, these tools are tightly coupled to business data, giving incredible visibility into how systems work, and how deployments impact site and business performance. Unfortunately, this tight coupling can make it difficult for others to benefit from this work.

It was discouraging to hear about talented teams doing the same work by building the same types of tools, mostly unbeknownst to each other. Of course, there is the egotistical issue of building the roundest wheel, but that didn’t appear to be the case. Instead, these teams weren’t in a position to share their work.

Give It Away

Aaron Kulick (@gofastweb) made a comment during his talk A Web Perf Dashboard: Up & Running in 90 Minutes that really resonated with me. While showing some screenshots of a dashboard used at WalmartLabs, he mentioned that he’d asked about sharing data, because it would be interesting to engineers, but wouldn’t mean much to competitors. Of course, the legal department had vehemently prohibited that. The good news was that the technology they’d used was available as open source; they’d simply stacked components together to build their monitoring.

Twitter has open-sourced a mountain of software developed by their engineering teams. Finagle is the foundation of their next-generation systems, and it’s available for anybody to build on. There are other companies building systems to accomplish the very same things Finagle does. The platform allows Twitter to build their systems more quickly, but isn’t directly related to the specific value they’re aiming to provide. Recognizing this, they’re generously giving it away.

Competitive Advantage

What about competitors? What’s to keep Facebook from coming along and “stealing” Twitter’s work by using Finagle in their own applications? If that were to happen, Facebook could successfully sidestep all of the effort required to build Finagle.

Yes, that’s a very narrow view of a bigger picture. It’s also a highly contrived example, because while their targets are similar, Twitter and Facebook seem to be aiming at different targets. If you’re going to make the argument, however, you should step back and remember that both companies are also heavily relying on the work of the other.[2] There is mutual benefit.

We All Move Forward Together

The effects of open source were plainly visible all week at Velocity. The web has a deep history built on the shoulders of open source: Apache httpd, PHP, Linux, MySQL, Python, Ruby, Perl. It’s currently moving forward quickly with Node.js, MongoDB, Scala, Redis, Tornado, Nginx, Varnish, Stud, HAProxy, and countless other open source projects graciously shared by their developers. These examples all highlight the incredible progress that has been possible because of open source.[3]

Did I mention Chrome, Firefox, and WebKit?

  1. One demo I saw showed the ability to diagnose a single customer’s issues on the site within seconds of an error, and backtrace their clicks on the site. ↩︎

  2. Twitter has used Cassandra, a project Facebook started. ↩︎

  3. As a cynical quip, consider a major alternative: IIS on Windows. Threads! ↩︎