Utilities giant Centrica turned to the NoSQL database from Couchbase three years ago to consolidate customer information in one place for field agents to access via a mobile app, but the experience wasn't smooth sailing.
Speaking during Couchbase’s Connect EU conference in London today Alan Fairhurst, head of integration delivery for digital technology services at Centrica spoke about how developers at the utilities company got to grips with the NoSQL database and it's proprietary N1QL query language, which was designed to extend SQL queries to work on JSON documents
First though it’s important to understand what Centrica was trying to achieve, namely giving its 50,000 global field workers across the British Gas, Bord Gais, Dyno and Direct Energy brands, to name just a few, access to customer information from a bespoke mobile app.
The challenge for Centrica developers was the vast array of backend systems this information was stored in, from an array of vendors both old (SAP, Oracle) and new (Salesforce).
The first step, starting three years ago, was to bring all of this customer information from these systems into a Hadoop data lake. Loading that into Couchbase using a Spark loader allowed the app developers to gain a single view of the customer. By exposing this information via an API gateway, Centrica was able to develop an iOS app for its field workers.
Now, the customer information app is used daily by between 55% and 75% of field engineers so that they can see what a customer’s previous interactions with the company has been, what other products they hold and if they must consider any vulnerabilities, amongst other information.
The process to get to this point wasn’t without its challenges though, ranging from security concerns to a lack of NoSQL expertise in house.
Fairhurst admits that the security team at Centrica has “not always been the most cooperative. So we partnered with them and made them part of the team and had business buy in to build the roadmap,” he said.
Also, the software engineers at Centrica were relational database experts. That didn’t stop them from jumping in and starting to work with the N1QL query language though. This is where the problems begun.
“They struggled with key lookups and were under tremendous pressure,” Fairhurst admits.
The resulting queries were “horrifying”, according to Fairhurst. “We had nested queries, few matching indexes and ended up with a complex solution and terrible performance. Couchbase went from hero to zero rapidly,” he said.
The solution was to swallow their pride and reach out to the Couchbase professional services team, who came in to work directly with the development team to coach them and start to move to key value lookups wherever possible and allow the developer team time to test and learn from their experience.
Then, when it comes to N1QL, Fairhurst recommended using it like a surgical knife: “only when needed”.
In terms of other lessons learned, Fairhurst urged customers to be on the latest version of Couchbase wherever possible, ensure your developers have some experience with a NoSQL datastore and to not run Couchbase as one big cluster but break it down for different use cases.