BT is turning to big data analytics to cut down on unnecessary engineer dispatches, improve broadband performance and reduce the occurrence of nuisance calls.
The telecoms giant runs a Cloudera enterprise data hub powered by Apache Hadoop that allows it to quickly analyse data from millions of users.
Data stored in Hadoop gives BT the ability to analyse network characteristics to identify whether slow internet speeds are caused by a network or customer issue. BT can then evaluate whether an engineer is likely able to repair the problem. The company claims that this enhanced understanding of performance problems has saved the company millions of pounds by reducing the number of unnecessary technician dispatches.
It has also helped them create new services for customers, such as "View My Engineer", the SMS and email alerting system that lets customers track the location of Openreach engineers. The company next plans to add predictive analytics to improve vehicle maintenance.
"We think it's been about a times ten return on big data across a whole range of different projects," says Jason Perkins, head of business insights and analytics architecture at BT.
Those projects range from analysing how regional weather variations affect infrastructure performance to identifying the reasons for spikes in call centre volume and reducing nuisance calls.
"We were able to use a big data approach and machine learning to filter out those calls, which were becoming a major problem," says Perkins.
"It's dramatically reduced the number of nuisance calls and freed up bandwidth, which is really important for us as well. It's had a great impact for the customer as well as they're getting a lot less noise in their lives."
The Cloudera platform
The Cloudera enterprise data hub (EDH) is a commercial platform for big data management that provides a unified view of customer data stored in a Hadoop environment.
BT’s previous extract, transform, and load (ETL) processes were built on a traditional relational database that was struggling to process BT’s vast volumes of customer data spread across different siloes.
The distributed nature of Hadoop lends itself to this kind of data problem, and helped BT process five times the volume of data in a third of the time that it could on the old system. The result was a return on investment of between 200 and 250 percent within one year of the deployment.
"We found £1 billion of extra revenue by getting our data in order,” Perkins claimed. “That's a lot of change to find down the back of the sofa."
The hub forms part of a multi-purpose analytics architecture at BT that also includes a data lake, an enterprise data warehouse that supports business intelligence and marketing, and an analytics sandbox where they can test new ideas. There are currently 50 such experiments running in the sandbox, which can quickly be retired if they fail or put into production if they succeed.
BT ensures the data is used responsibility through usage policies and techniques such as encryption capabilities, differential privacies that edit the raw data, and digital watermarking to trace any data that comes out of the BT platform.
“The big data platform is enabling us to bring the data together, but also add the smarts on top of it, whether it be machine learning, AI, the self-service capabilities, whether that's data prep or data visualisation. It's allowing us to give that information to our users so they can make much more informed decisions.”
BT also investigated data services offered by Hortonworks and MapR, before plumping for Cloudera.
"We thought Cloudera was a leader," says Perkins. "We were not just looking at that time, but we were trying to look forward, which is always hard, but we thought they had the best mix.
"They had some great use cases where other companies who we admired were using them, which gave us the confidence that they'd be a great partner for us, and we had confidence in their plans going forward.
"When you're going for what was at the time quite a small vendor, you're always worried about mergers, acquisitions, what's going to happen to this company when we've got all our data invested in it. Those were important considerations at the time."
Perkins now wants to integrate sensors in its fleet of vehicles that can support predictive analytics to identify vehicle failings and exploit new capabilities such as streaming analytics, machine learning and iterative processing.
"The whole point of having all this data is to make the customers' lives better," he says. "If you can understand it, we'll be able to get away from customers having to ring us up to tell us about their problems. We'll know about them, we'll be fixing them before they tell us about them. That's the ambition."