It took Brooks Cappella about a nanosecond to notice that his BlackBerry wasn't working. He manages the technology operations at a New York hedge fund, Highland Financial, and he was wrestling with some problems on the company's servers when he stopped getting messages late on Apr. 17. Just what he needed to make a difficult day worse. 'Any downtime is something we notice right away,' he says. 'Not having the BlackBerry didn't help.'
Gulp. Surprise turned to disbelief as millions of hyperconnected professionals found themselves in the midst of a BlackBerry blackout that lasted from late in the day of Apr. 17 until the early hours of Apr. 18. As the network for the addictive e-mail device crashed, there were pangs of stomach-churning panic in some cases, simple loneliness in others. Research in Motion (RIMM), the Canadian company behind the BlackBerry, says the e-mail service was down for a little less than 12 hours.
Leaving customers guessing
Though service was restored relatively quickly, it may take a bit longer to regain full confidence in the BlackBerry network. RIM issued a very brief statement on the issue saying the interruption affected customers in North America and consisted of e-mail delivery being 'delayed or intermittent' during the incident. But it didn't address why the network failed or what the company plans to do to avoid similar problems in the future. Analysts and customers were left guessing. 'RIM has been rather uncommunicative today about all this,' says Todd Kort, analyst with market researcher Gartner Group (IT). 'They haven't said anything more than, 'Yes there is a problem and everything has been fixed'.'
This is hardly the first time RIM has faced these kinds of issues. The BlackBerry service experienced outages of varying severity in 2001, 2003, 2005, and 2006. RIM addressed the issue in its most recent annual report, saying that 'continuity of operations planning' had been a priority during the 2005 fiscal year and that the company had added 'additional internal redundancy' in order to improve reliability of its network.
Network overload‾
So why more outages‾ RIM's rapid growth may have contributed to the problems. Richard Doherty, research director of technology consultant Envisioneering Group, suspects a network overload of some kind was behind the outage, prompted in part by a spike in network traffic related to the users seeking news about the Virginia Tech shooting and compounded by RIM's stunning subscriber growth. The company added more than a million subscribers in its most recently reported quarter, bringing its worldwide subscriber base to about 8 million.
'Networks can only take so much at once,' he says. 'If 3% of all the people with telephones were to pick up their receivers at the same time, the phone system would collapse. When you build a network you have to plan for peak use.'
The center of the problem may have been at RIM's network operations center, or NOC, in Waterloo, Ontario. Jack Gold, an analyst and consultant based in Northborough, Mass., says that would explain why BlackBerry service on multiple wireless networks all stopped functioning. The devices are available from wireless carriers such as AT&T (T), T-Mobile, Sprint (S), and Verizon Wireless, a joint venture of Verizon (VZ) and Vodafone (VOD).
Rivals rush in
Additionally, the service failure seemed to be more acute among users of RIM BlackBerry Enterprise Server (BES), and less so among users of its BlackBerry Internet Service, says Kort, the Gartner analyst. BES users are classic, corporate BlackBerry users whose companies run BlackBerry server software on their own internal servers. BlackBerry Internet Service (BIS) users tend to be the ones who buy their own devices, and use them primarily to access their personal e-mail accounts. 'BIS users seem to have been unaffected or at least less affected than the BES users,' Kort says.
Competitors will find it hard to resist pouncing on the problems of RIM, the market leader in wireless e-mail. Motorola (MOT), for instance, has been pushing hard to gain ground in the market, and in January acquired Good Technology, which offers services and software similar to RIM's. 'I wouldn't be surprised at all to see Motorola try to capitalize on this,' says analyst Sean Ryan of IDC.
Microsoft (MSFT) didn't hesitate. The software giant, which sells its own wireless e-mail technology, reached out to the media after the BlackBerry outage to say that such crashes don't happen with the company's wireless e-mail. The reason: Microsoft's wireless service runs on the same servers as the company's traditional Exchange e-mail"”and doesn't require a separate network like RIM's.
'A short-term annoyance'
Rob Sanderson, an analyst with American Technology Research in San Francisco, said the outage would likely have little effect on RIM's business prospects. 'In the past this sort of thing has been more a short-term annoyance than having any sustainable business impact,' he said. 'We don't yet know the cause, but most popular services suffer some periods of growing pain. I think more important is how frequent are interruptions and how quickly service is restored. RIM seems to be pretty reliable and responsive.'
If there was one lucky draw for RIM, the outage took place during a period of relatively light e-mail use. The first word of the service break hit the news wires at about 8 p.m. EDT, and was restored before 7 a.m. Greg Tozian, vice-president at Internet advertising firm Overland Agency in Portland, Ore., said he noticed that several e-mails he'd sent from the device were marked with a red 'X' indicating they hadn't been sent. 'Then there were scads of e-mail coming in all at once at 6:44 this morning,' he says. 'Luckily, it wasn't during business hours when I was trying to get something done.'
Perhaps a BlackBerry blackout then is not always such a terrible thing, if it happens at night and only very rarely. The outage no doubt led to dinner conversations marked by fewer distractions from vibrating electronic devices and more opportunities for engaging in the world beyond the tiny electronic screen.
Perhaps there was more time for pondering larger philosophical issues about the human relationship with technology. Consider this posting on the Wall Street blog Dealbreaker.com: 'If a banker sends a BlackBerry and no-one reads it, does he really exist‾'
Hesseldahl is a reporter at BusinessWeek.com in New York. With Cliff Edwards in Silicon Valley.
Copyright 2000-2007 by The McGraw-Hill Companies Inc. All rights reserved.