RIM's BlackBerry service outage

blackberry outageResearch In Motion's (NASDAQ:RIMM) global BlackBerry service outage in October stands out as the biggest wireless flub of the year precisely because of the elements involved: RIM, BlackBerry, service outage. Those terms normally do not go together, and RIM has built its reputation on being a reliable and secure way for enterprises and governments to send and receive data. RIM's reputation took a major hit from the outage--the largest in its history--and the company is still working to repair the damage. 

The outage started Monday, Oct. 10, with customers in the Europe, Middle East and Africa experiencing delays in service. By Tuesday, Oct. 11, RIM had identified a core switch failure within RIM's infrastructure as the cause of the outage, and since its backup had failed the service was still not restored.

RIM launched a website page to update customers about the outage, but there was still no word from RIM co-CEOs Jim Balsillie or Mike Lazaridis. On Wednesday, customers in the Americas began experiencing service delays. As RIM worked through the issue, it posted a note from RIM CIO Robin Bienfait, who gave an update on the efforts to restore service. As Wednesday night approached, some service was restored though RIM continued to clear a backlog of messages. 

It was not until Thursday morning, four full days after the outage started that Lazaridis appeared in a video message expressing his apologies for the outage. "You expect better from us, and I expect better form us," he said. On Thursday service was restored globally.

Although RIM's system is designed to failover to a back-up switch, it did not work as previously tested, creating a backlog of data. This caused RIM to "throttle" service in the affected regions, which then created more disruptions in regions not previously affected, including North America.  In a conference call that Thursday, Lazaridis noted that the company had an 18-month record of "99.997 percent" service reliability prior to the outage.

RIM sought to make amends in the aftermath, by offering a selection of premium mobile applications for free as a token of its appreciation for customers' patience during the outage. RIM's enterprise customers were also offered one month of free technical support. Balsillie said the company is considering compensating wireless carriers as a result of the outage. The company also formed an internal "SWAT team" to look into the cause of the outage.

What made the outage so devastating was that it cut at the heart of RIM's identity as a company. The way in which the company struggled to respond--particularly Balsillie and Lazaridis' decision to remain silent for days while the outage was occuring--is emblematic of the company's struggles to keep pace with Apple (NASDAQ:AAPL) and Google (NASDAQ:GOOG) in smartphones. The fact that the outage happened days ahead of the iPhone 4S launch didn't exactly help. All of that combined to make the outage this year's biggest turkey. 


RIM's BlackBerry service outage