Facebook network manager: Just assume parts of network will fail

Experts in the computing industry for a while now have advised against using the old five-nines reliability standard that historically characterized the telecom industry, and Facebook (NASDAQ: FB) is no exception.

Omar Baldonado, who manages the network team at Facebook, told operators and vendors at the Open Networking Summit last week that the notion of trying to meet a 100 percent or other reliability metric is a "very, very difficult thing."

In his previous roles at vendors, he realized that making a 100 percent failproof network is a really tall order. "What we've focused on is finding the problems quickly and automatically resolving them quickly through software, so just assume that different parts of your network are going to fail," he said during his keynote.

"Your design should accommodate all the failures, right, because you're not going to build the perfect … you're not going to find the perfect switch that would never fail, the perfect line card or the perfect optic. It could be hardware or it could be software, whatever, something's gonna fail -- plan for that," and build systems that detect problems and try to remediate it as quickly as possible.

Earlier at the same conference, John Donovan, chief strategy officer and group president at AT&T Technology and Operations, was asked how AT&T considers companies like Google, Facebook and Amazon coming on the scene and into service provider territory.

"If Google Fiber isn't a competitor, I don't know what is," he said, noting that Amazon sells the network with what they do. "That's the world we're in today, as it gets to software, the barriers to entry lower, and so everybody potentially is a competitor. You named a few that we have to take very seriously. The only thing we can do to control our own fate is make this transformation as fast as possible and be a nimble company moving as quickly as possible," he said.

"The cultural change we're making as a company is as important as any investments we're making technology-wise," he added. "The pace at which we're accelerating is tremendous. Whoever comes into our space, we're just gonna have to compete with them as if they're world-class companies."

But, he said, "I can tell you right now, everything we do that is benchmarked for productivity and efficiency and investments, we look at software companies. We don't look at the other telecommunications providers as like-for-like competitors. We compare ourselves and what we want to be aspirationally to companies that were born in software, who only do software, and that's the core of what they do."

Related articles:
SK Telecom SVP: Future of SDN, NFV is mobile
SDN/NFV hit the road maps, ecosystem broadens - Year in Review
Why AT&T, Verizon, Ericsson and the rest of the industry is embracing SDN and NFV