Sweating hardware assets at Experian with SUSE Enterprise Storage

When Experian’s Business Information (BI) team overseeing infrastructure and IT functions saw the customers’ demand for better and more comprehensive data insights grow at an unprecedented rate, the company required a better storage solution that would enable them to maintain the same performance level. Implementing the SUSE Enterprise Storage solution gave Experian a starting platform for seamless capacity and performance growth that will enable future infrastructure and data projects without the company having to worry about individual servers hitting capacity.

The Problem

As a company facing increasing customer demands for better and more comprehensive insights, Experian began incorporating new data feeds into their core databases, enabling them to provide more in-depth insights and analytical tools for their clients. Experian went from producing a few gigabytes a month to processing hundreds of gigabytes an hour. This deep dive into big data analytics, however, came with limitations – how and where would Experian store larger data-sets while maintaining the same level of performance?

From the start, Experian had great success running ZFS as a primary storage platform, providing the flexibility to alternate between performance and capacity growth, depending on the storage medium. The platform enabled them to adapt to changing customer and business needs by seamlessly shifting between the two priorities.

But Experian’s pace of growth highlighted several weaknesses: First off, standalone NASes platforms were insufficient, becoming unwieldy and extremely time-consuming to manage. Shuffling data stores between devices took days to complete, causing disruptions during switchovers. The second challenge was a lack of high availability – Experian had developed robust business continuity and disaster recovery abilities, but in the process, had given up a certain degree of automation and responsiveness. Their systems could not accommodate the customer demand for 24/7 real-time access to data created by the advent of APIs and the digitalization of the economy. Experian’s third and greatest challenge was in replicating data. Data would often fluctuate and wind up asynchronous, creating a precarious balance – if anything started to lag, the potential for disruption and data loss was huge.

Experian had implemented another solution exclusively in their storage environment that had proven to be rock solid and equally flexible. While the team was happy with its performance, the new platform failed to fully address the true performance issue and devices and controller cards would still occasionally stall. As a company in the business of providing quick data access, the lag time raised serious concerns and presented obstacles in meeting client and business needs.

The Solution

Experian only saw one real short-term solution and moved to running ZFS on SUSE Linux Enterprise. This switch saved Experian time to find a more durable resolution, but was also fraught with limitations. Experian spent a number of weeks trying to find a permanent solution that would protect both their existing investment and future budget. To fix the limitation issue, Experian temporarily added another layer above their existing estate that would manage the distribution and replication of data.

As Experian was preparing to purchase the software and hardware needed to provide a more long-term solution, they come across SUSE’s new product offering – SUSE Enterprise Storage, version 3. Based on an open source project called Ceph, SUSE Enterprise Storage offered everything Experian needed with file and block storage and snapshots to run well on their existing HPE DL380 platform. SUSE was already Experian’s operating system of choice for a few years, proving to be reliable, fast and flexible. SUSE support teams were also responsive and reliable – this new solution offered the perfect product to meet Experian’s need.

The Outcome Experian’s initial SES build was modest, based around four DL380s for OSDs and four blades as MONs. Added to that were two gateway servers to provide block storage access from VMWare and Windows clients. SUSE Enterprise Storage’s performance met and exceeded Experian’s expectations – even being a cross site, real-life IOPS easily go into thousands. The benefit to Software-defined storage is that it allows clients to abstract problems away from hardware and to eliminate the issue of individual servers hitting capacity. By adding more disks to make space for more data and adding another server when access has slowed down without having to pinpoint exactly where they need to go, capacity planning is much less of a headache for Experian. Software-defined storage also enables Experian to sweat their server hardware for longer, making budgeting and capacity planning easier.

While SES doesn’t replace the flash-based storage Experian uses for databases, having a metro-area cluster means that business continuity is taken care of. Experian ended up with is a modern storage solution on modern hardware that gives the company a starting platform for both seamless capacity and performance growth that enables future infrastructure and data projects