Data Quality in the Cloud: Ensuring Data Value as You Move to the Cloud
Data management has become increasingly complex over recent years, along with the variety of data types, databases, deployment platforms, and data use cases. Enterprise data that was once kept centrally only in on-premises databases is now widely distributed across networks and stored in multiple locations in the cloud as well as on premises. Managing data across cloud, multi-cloud, and cloud-hybrid platforms brings new challenges and calls for new approaches to data management. The same technical innovations that changed data management are also driving changes to data quality management (DQM). The trend toward data silos, the move to self-service data and analysis, big data analytics, and artificial intelligence and machine learning (AI/ML) all require that we rethink data quality management.
We can no longer afford for data quality to be an afterthought. Profiling, assessment, and cleansing late in the data lifecycle doesn’t work in today’s high-speed data world where real-time data, multiple use cases, and self-service data analysis are the norm. DQM must be woven into the analytics lifecycle and the information supply chain, and that is not practical with a wide range of disparate tools and a steep learning curve. A small set of familiar and stable tools that work across on-premises, hybrid, and cloud environments is the new necessity for DQM.
More data, more kinds of data, more consumers, more use cases, and all of it moving faster. The stakes for data quality are higher than ever. DQM practices of the past, with multiple data quality applications to manage multiple data silos, is unsustainable. And combining DQM silos with data silos only compounds the difficulties. The risks inherent in poor data quality— and the consequences of working with quality-deficient data—can be quite severe. Data has become a critical business asset. DQM must become a critical business process.