HLR-driven outages causing network headaches

Recent mobile service interruptions might be a sign of things to come as networks succumb to traffic overloads that disrupt signaling, or they might just reflect the intricacies that can sometimes trip up engineers conducting network tweaks and upgrades. Alternatively, the outages could be a due to a combination of both situations.

On different dates this month, France Telecom's Orange network and Telefonica's O2 network in the United Kingdom both suffered day-long outages. Those network crashes, as well as three outages suffered by Verizon Wireless (NYSE:VZ) in December 2011, were all due to issues involving the home location register (HLR), according to a GigaOM article.

The HLR is the central database that stores details of each mobile customer's identity and service profile as well as tracks each device's location for call and message delivery.

What separated Verizon's network crash from those of Orange and O2 is that Verizon's outage impacted its new LTE network while the European networks that went down were aging 2G and 3G networks.

Some contend the outages were caused by massive volumes of signaling traffic to which the networks are being subjected by smartphones and other mobile computing devices. "Constant network chatter from smartphones and their applications are overwhelming network cores. On normal days they can handle that traffic, but even a small glitch throws everything out of whack," said GigaOM.

One common feature of the recent network outages appears to be botched network upgrades, which caused those disastrous glitches. In December, Verizon attributed its 2011 outages to "growing pains" within its IP Multimedia Subsystem architecture as it deployed LTE.

In testimony last week before France's National Assembly, France Telecom CEO Stephane Richard blamed Orange's recent outage on a software bug in Alcatel-Lucent (NASDAQ: ALU) equipment. He said the software had undergone an update roughly 48 hours before the 10-hour outage, according to Reuters.

"Orange customers who kept their phones turned on and didn't move between cells remained unaffected for longest, as no changes were made to their entries in the HLR. But even those customers were affected in the end, as a build-up of error messages caused the HLR to saturate," reported ComputerWorld UK.

As for O2's network crash, the issue may have been related to the transition of subscribers' details to Ericsson's (NASDAQ:ERIC) Centralized User Database, which disappeared during the process leaving handsets unable to authenticate their users, according to The Register.

The network failure at O2 raised concerns over whether UK mobile networks will be able to handle the crush of traffic expected once the 2012 Olympic Games begin in London later this month.

"The huge influx of visitors to London ahead of the games will cause network traffic spikes, putting pressure on the UK's mobile networks, which already have a poor reputation compared to others in Western Europe. While UK mobile operators claim to be prepared, they have not yet given indication of the scale of their plans," said Steven Hartley, practice leader, Ovum Telecoms Strategy.

T-Mobile USA also suffered a data outage on its HSPA and HSPA+ networks earlier this month, which some observers have suggested could be related to the operator's $4 billion network modernization and LTE deployment effort, which involves intensive spectrum refarming. The operator has not publicly cited a specific cause for the problem.

For more:
- see this GigaOM article
- see this ComputerWorld UK article
- see this Information Age article
- see the The Register article
- see this Reuters article

Related articles:
O2UK hints at user compensation following 24-hour network collapse
FT Orange network outage could cost €20M in repairs and customer compensation
T-Mobile restores service after data outage
UPDATED: Verizon restores service after LTE network falters again
Verizon: LTE outages a result of network's 'growing pains'