Super Bowl XLVIII: How did Tier 1 wireless carriers' networks hold up?

Super Bowl XLVIII just wrapped on Sunday, with the Seattle Seahawks demolishing the Denver Broncos 43-8 at MetLife Stadium in East Rutherford, N.J. Now that the game is over, wireless carriers are boasting about the traffic they saw at the venue and how their networks held up at the game, which more than 85,000 people attended.

In a company blog post, Verizon said it recorded record volumes of data two hours before kickoff. The carrier notched 800 percent more data connections during this weekend's game than during the busiest hour at last year's Super Bowl in New Orleans. Further, Verizon said that during the halftime show with Bruno Mars and the Red Hot Chili Peppers, Verizon used more data in MetLife stadium during a single hour than at any previous Super Bowl. Verizon said that the total customer in-stadium data usage on its network was 1.9 terabytes, which it said was a Super Bowl record.

As for AT&T, John Donovan said AT&T's total data usage in the stadium on its network was more than 624 GB, which is the highest data usage AT&T had ever seen from a one-day sporting event it has measured. Further, AT&T said traffic peaked at 5-6 p.m. Eastern Time, which was the hour directly before kick-off, and during that hour 119 GB crossed its in-stadium network. That peak data usage is up 52 percent from the 78 GB during the busiest hour at last year's Super Bowl.

Sprint said in a statement that its network "performed exceptionally well during the Super Bowl on Sunday, besting our competition in voice performance and providing consistent, solid data performance. Sprint saw 4G LTE data traffic increases over a typical game day of 83% and 150% for download and upload speeds respectively. And, Sprint was using a single channel of LTE, so about half of what our competitors are using today. With the deployment of Sprint Spark, Sprint will be able to aggregate multiple channels of LTE to boost throughput speeds that will match and ultimately exceed competitor speeds."

T-Mobile said that compared to the New Orleans Super Bowl in 2013, data traffic increased by approximately 700 percent during the busiest hour for its network, but during that period it still saw peak download speeds reach over 60 Mbps. Additionally, T-Mobile said that compared to what it typically sees at MetLife Stadium, voice traffic during the Super Bowl increased by approximately 70 percent.

However, FierceWireless worked with Nexgen Wireless, an independent, third-party telecommunications software and engineering services vendor, to see how the carriers' networks performed at the event.

Nexgen's competitive benchmark studies focus on in-building traffic at major events. During the Super Bowl, Nexgen tested the carriers' 3G voice and LTE data quality. The data was collected for each network using the vendor's Accuver XCAL-M software loaded on the Samsung S III, with a handset for each carrier's network. This information then was uploaded to a server for Nexgen's team in Chicago to process and compile using Nexgen's proprietary tools.

One very important caveat: Nexgen said its XCAL-M software was not ready to support T-Mobile's handsets and file structure. Thus, T-Mobile's network performance was not tested.

Each testing sequence for each carrier was conducted at six locations throughout the stadium. Six tests were done. The first was conducted 45 minutes prior to kickoff, the second was at kickoff, the third was at approximately 7:30 p.m. ET, the fourth was at halftime in the game, the fifth was 45 minutes after the start of the third quarter, or 9 p.m. ET, and the sixth was at approximately 9:45 p.m. ET.

Nexgen measured average downlink throughput, average uplink throughput, ping latency, voice call success rates ((total attempts minus failures)/total attempts), and voice call drop rates ((total successful calls minus drops)/total successful calls).

For downloads, the tests used a 20 MB file, and for uploads the tests used a 10 MB file. To measure latency, Nexgen conducted a total of 100 ping tests with packet size of 32 bytes for each sequence. A short voice call test consisted of 10 short calls (approximately 20 seconds in length each) and  long call test consisted of three long calls (approximately 2 minutes in length each).


The raw data:

 

AVG Downlink Throughput (Mbps)

AVG Uplink Throughput (Mbps)

Ping Latency (ms)

Voice Success %

Voice Drops %

ATT

5.31

2.897

143.82

80.00%

0.00%

Verizon

2.08

1.365

166.02

90.00%

0.00%

Sprint

1.15

0.075

215.43

97.00%

0.00%

Nexgen's tests found that AT&T's network was the clear leader in data downlink and uplink throughput as well as latency. On all three metrics, AT&T outclassed Verizon Wireless, which is the official wireless provider of the National Football League. However, the New York metropolitan area, where the game was held, is one of Verizon's largest markets, and its network could have been carrying more traffic, which might have contributed to the results.

Verizon spokesman Tom Pica offered another explanation: He told FierceWireless that "47 percent of our customers' 4G LTE usage at MetLife Sunday was on our new AWS spectrum. One big miss in the third-party NextGen test was use of a device (Galaxy S3) that is not compatible with our newly deployed AWS spectrum, which has increased 4G LTE data capacity by 300 percent in the NY market, which includes MetLife Stadium." (Around 20 percent of Verizon's smartphone base can access AWS spectrum for LTE today but nearly all new devices going forward will have the capability.) Click here for more details from Verizon on its Super Bowl performance.

Sprint badly lagged on those three metrics, an indication that its network is still behind Verizon and AT&T when it comes to deliver LTE data. However, Sprint's network performed the best in terms of voice call success rate, with a 97 percent success rate, compared to 90 percent for Verizon and 80 percent for AT&T.

One thing that should bring cheer to all three carriers: There were no voice call drops for any of the carriers. The carriers networks were "very stable after the calls were successfully originated," according to Nexgen.

The carriers took great pains before the game to note the improvements they were making to their networks. As CNET noted, Verizon Wireless (NYSE:VZ) began prepping its network 18 months before the game and deployed a Distributed Antenna System with 552 antennas to help increase network capacity four times at MetLife Stadium. AT&T Mobility (NYSE:T) said it launched a DAS system at the stadium with more than 500 antennas and 6.4 miles of cable to triple the capacity of the coverage that MetLife had during the 2013 regular football season. Sprint (NYSE:S) said it began its working to improve its network at the stadium as far back as late 2012 and increased voice capacity by 53 percent and data capacity by 52 percent by enhancing its existing cell sites and adding temporary sites, including five cell sites on wheels (COWs) in three corners of the stadium parking lot. And T-Mobile US (NYSE:TMUS) said it increased the capacity of both its HSPA+ and LTE networks by 15 times in an upgrade that started in November 2013, including via more backhaul, a DAS system and COWs.

Article updated Feb. 4 at 1 p.m. ET with comments from Sprint and T-Mobile.

Super Bowl XLVIII: How did Tier 1 wireless carriers' networks hold up?
Read more on