Half a decade ago, Jonathan Heiliger compared the world of Internet data centers to Fight Club.
It was the spring of 2011, and the giants of the Internet—including Google, Amazon, and Microsoft—were erecting a new kind of data center. Their online empires had grown so large that they could no longer rely on typical hardware from the likes of Dell, HP, and IBM. They needed hardware that was cheaper, more streamlined, and more malleable. So, behind the scenes, they designed this hardware from scratch and had it manufactured through little-known companies in Asia.
This shadow hardware market was rarely discussed in public. Companies like Google saw their latest data center hardware as a competitive advantage best kept secret from rivals. But then Facebook tore off the veil. It open sourced its latest server and data center designs, freely sharing them with the world under the aegis of a new organization called the Open Compute Project. “It’s time to stop treating data center design like Fight Club and demystify the way these things are built,” said Heiliger, then the vice president of technical operations at Facebook.
Google was the first company to rethink data center design for the modern age.
With the Open Compute Project, Facebook aimed to create a whole community of companies that would freely share their data center designs, hoping to accelerate the evolution of Internet hardware and, thanks to the economies of scale, drive down the cost of this hardware. That, among other things, boosts the Facebook bottom line. It worked—in a very big way. Microsoft soon shared its designs too. Companies like HP and Quanta began selling this new breed of streamlined gear. And businesses as diverse as Rackspace and Goldman Sachs used this hardware to expand their own massive online operations. Even Apple—that bastion of secrecy—eventually joined the project.
Two big holdouts remained: Google and Amazon. But today, that number dropped to one. At the annual Open Compute Summit in San Jose, California, Google announced that it too has joined the project. And it’s already working with Facebook on a new piece of open source hardware.
The announcement reaffirms the power of Facebook’s big idea. Google was the first company to rethink data center design for the modern age. For years, its technology was well ahead of anyone else. And when Heiliger complained of that Fight Club mentality, Google was surely top of mind. But through the Open Compute Project, Facebook has pushed the rest of the industry forward. And market forces have pushed Google to share its secrets in new ways. “The community and Google have inched closer and closer together,” says Jason Taylor, Facebook’s vice president of infrastructure.
But Google’s move also points to other changes inside the big Internet players. That joint open source project from Google and Facebook relates to the rise of deep learning, an artificial intelligence technology that is rapidly reinventing so many parts of the modern world. Both companies see AI as a key part of their future, and both believe they’ll get there faster if they share and collaborate on some of the core technologies that drive these neural networks.
Happy to Share
Google’s Urs Hölzle—one of the company’s first employees and the engineer most responsible for what is probably the world’s largest and most advanced computer network—doesn’t see today’s announcement as a big change. He points out that Google has openly discussed its internal hardware designs in the past.
“I know that historically in the press, there has been a tendency to position it as Open Compute Project versus Google. But it has never been like that,” he says. “Over the past ten years, we have shared many, many things with the industry. This is the latest one.”