At many enterprises, analytics initiatives present more upside than risk. They layer onto rather than disrupt the core business.

When your core business is analytics, such initiatives pose an exponentially higher risk. You are playing with the crown jewels.

The following case study is therefore instructive because it examines how a decades-old data and analysis firm, with everything to lose, is embracing a revolutionary cloud-first approach. The smartest minds in the organization have carefully weighed their options at each stage, balancing the low risk of the status quo with the upside of change. Data leaders and practitioners can learn from their story.

This firm, which we’ll call “Assessments Unlimited,” provides vertical-focused offerings that include the following:

• Financial services analytics that addresses regulatory compliance, product and pricing decisions, and risk assessments for fraud, credit, and collections

• Property and casualty insurance risk, for example, covering weather disasters, claims, fraud, compliance, and IoT vehicle tracking

• Decision support for the energy and mining industries, including supply chain management, geopolitical risk, and operational risk

The Starting Point

For years Assessments Unlimited has notched steady growth with its industry domain expertise, rich vertical-focused datasets (now exceeding 10 PB), new services and deepening integration with customer workflows. Their strategy is to continuously develop new services, build and acquire new data sets and partner with complementary data provider and technology platforms.

But its legacy data architecture hasn’t changed much and was becoming an anchor. 

Assessments Unlimited runs most of its analytics workloads on a consolidated, multi-tenant DB2 z/OS system that resides on monolithic on-premises hardware. Nearly 20 billion SQL statements are executed weekly on this mainframe environment, as well as Oracle and SQL Server, to serve the three lines of business. The legacy code base comprises tens of millions of lines of code.  Multi-year license agreements are costly, as are ongoing MIPS charges. Software and analytics product rollouts are slow due to interdependent services and shared platform resources.

This year their CTO embarked on a data modernization initiative that will refactor the architecture and reinvent how they operate. Their objectives are to:

• Diversify the platforms underlying their services

• Reduce time to market for new offerings

• Improve cost and transparency, and shift CAPEX to OPEX

If all goes well, in 3-5 years all applications and IT services will reside on a more economic, managed public cloud platform with a strong open source component. Data will be available on a real-time basis and new technologies will be rapidly accessible. Analytics workloads will consume processing and storage resources that are fit for purpose. Development and release cycles will shrink thanks to elastic resources and a simpler code base.

The Target Environment

As a first step, Assessments Unlimited has decided to replace its mainframe system with AWS Postgres. Like other financial-services/analytics firms such as Capital One, BMLL Technologies and RavenPack, Assessments Limited has selected the AWS Postgres open source object RBDMS to improve performance, availability, scalability and cost. Over time they expect to adopt the Amazon S3 data lake platform to aggregate, transform and perform discovery on a wide variety of data types. They also will adopt Amazon Redshift and Snowflake to support targeted data warehouse reporting and analytics. 

This new data platform is intended to run existing workloads more efficiently and support new revenue-generating initiatives such as machine learning, which ranks alongside the cloud migration, Lean Six Sigma and “global talent optimization” as a C-level initiative for operational excellence. Assessments Unlimited is using machine learning to model oil and gas exploration investments, identify cyber-security risks and improve the accuracy of insurance claims analysis. These initiatives require a more efficient and flexible data architecture.

Getting from Here to There

Assessments Unlimited has defined several key requirements to minimize risk during its multi-year architectural transition.

• Application migrations to the cloud will be phased. For each phase, Assessments Unlimited will isolate to the degree possible an inter-related set of processes and applications. They will first migrate read-only applications as they have the fewest inter-dependencies. Still, certain services will be compromised during each phase, given the linkages that have built up over years of running a largely monolithic platform.

• Business disruptions will be minimized. For example, to achieve zero-downtime migrations, Assessments Unlimited is using change data capture to store incremental updates during batch migration loads. When the batch is complete, incremental updates are applied to the target and users can non-disruptively switch to the fully synchronized new cloud platform.

• During migrations, only mission-critical data access is permitted, to avoid unnecessary resource consumption. This means alerting users in advance of specific services and activities to avoid during off-hours migration windows.

• Replication of unchanged datasets is to be eliminated wherever possible. This means replacing batch replication with change data capture, reducing production impact, transmission bandwidth costs and processing efficiency.

• The process will be automated wherever possible to reduce effort and the risk of error. This applies to tasks such as data replication, target data structure creation, and data transformation.

• Training is core to each phase. Assessments Unlimited will train IT staff and Line of Business users on the new platform and services with each incremental cloud rollout.

• Sensitive data will be encrypted and tokenized. For example, Assessments Unlimited is encrypting data as it travels over the Wide Area Network according to the AES 256 symmetric-key standard.

The architectural transition is underway. Assessments Unlimited is currently migrating its mainframe data to AWS Postgres. They plan to add Oracle and SQL Server as sources and S3, Amazon Redshift and Snowflake as targets. At each step, they will measure progress and adjust the plan based on lessons learned.  

The scope and risk of this cloud transition is a testament to just how mission-critical data modernization has become. Its success will be a barometer for other Fortune 2000 companies as to whether the promise exceeds the peril.