As featured on TM Forum's the Insider blog
I have to admit to being one of the unconvinced that big data was anything more than just a fancy name for data analytics and business intelligence (BI) or a great excuse for suppliers to sell more software and boxes. However, after two days at TM Forum’s Big Data Analytics Summit in Amsterdam I have been shocked into a new sense of respect for this fascinating, yet critical discipline.
I am not planning to sing the praises of big data here but to try my best to encapsulate the key points I picked up, and there is no way I will deep dive into any of the technicalities as there are far better qualified people to do that. For the layman, which I freely admit to being, big data seems to have come about simply because we are now collecting, processing, storing and utilizing data at a ridiculously gargantuan rate, and that legacy BI technology simply can’t cope.
Every business, especially CSPs, have data being created and stored all over their networks and beyond into the cloud. It is no longer viable to collect all that disparate data, ‘normalize’ it and store it one place specifically for analysis and reporting purposes. Sure, it’s fine for historic data but these days we are talking about real-time analysis of a customer, a connection, a handset, a cell site, a router, a switch, etc. The list is endless.
My understanding is that big data, and the technologies it embodies, make it possible to access data from anywhere, in real time and generate output for a specific query in the blink of an eye. This is the sort of analysis that Google does when you search, and what Amazon does when you browse, and what mobile operators are hoping to do when offering customers new services, and also to predict when they are going to churn or need a new handset. It’s the same thinking around pushing targeted ads to mobile customers and being able to guarantee to advertisers that they have been delivered and read.
I have been hearing about customer profiling for years but this traditionally grouped like-minded types together using medians or averages to determine where they fit. As pronounced by one speaker, you can never deliver ‘average’ for a one-to-one customer experience. But it has been simply too difficult and required too much processing power to be able to do this for individuals. Not any more, big data is promising and actually delivering this ‘power of one’ today.
However, not every one of the 150+ attendees believed all they were hearing. The skeptics were not necessarily vocal, but certainly cautious. A number spoke of big data’s current position in the famous Gartner hype curve. Most thought it positioned at the peak of inflated expectations phase, about to slide into the trough of disillusionment. One speaker described the trough more like a very deep ravine but with short distance across to the plateau of productivity. Amazing how graphic techies can get sometimes.
One hosting and data center operator admitted to NOT using big data because she felt imminent breakthroughs in Scalable SQL technology would be more viable.
The most common word used throughout the event was ‘Hadoop’ (an open source software project that enables the distributed processing of large data sets across clusters of commodity servers) that seemed to be utilized by almost everyone in some fashion. For sheer shock value, the Google presentation demonstrated the massive processing power being offered to governments and corporations and just how effective big data was in being used to do queries in seconds that would have taken days only a few years back. The fact that the whole internet can today be indexed in 30 minutes, a process that used to take four days, is mind-boggling.
There was no shortage of case studies by leading operators as well as eBay, Google, Microsoft, Verizon, Telefónica and CapGemini, to name a few. Needless to say, I have now become a ‘believer’ and will be keeping a close eye on advances in big data and how it can be best applied to the telecom industry.