Our world is increasingly data-driven, given the expanding palette of tools and technologies that enable all members of an organization to more easily manipulate and analyze data originating both within and external to the organization. Advanced analytics tools allow users to run a variety of statistical and machine learning algorithms, while end-user visualization tools help users create “data stories.” Environments that integrate the R language allow researchers having little or no formal training in software development to develop programs that perform analysis using a variety of data sets managed within the organization. Some tools provide a drag and drop user interface that even further reduces the need for programming skills.

The result is that growing “data awareness” improves the fidelity and value of analytical models that can eventually be directly integrated into operational processes. Yet enabling non-IT staff to perform analytics on an organizations data coupled with the competitive nature of business environments has started a vicious cycle, in which representatives of the different business functions have an ever-growing appetite for more data to be delivered at an accelerating pace.

The accelerating speed of business only exacerbates the challenge. Traditional siloed data management creates obstructions that can shackle the business and result in diminishing competitiveness. Multiple commerce channels (ranging from electronic Point of Sale transactions at brick and mortar stores to 24/7 eCommerce activity) create greater volumes of customer transactions, and moments lost in presenting the right offers to each customer lead to missed opportunities. To maintain a competitive edge, many organizations need to perform real time analysis of information and data flowing from their systems.

These increased pressures often result in organizational contention by placing significant stress on those tasked with managing and provisioning the data. The increased demand for better/faster data increases the workload and responsibilities for the data owners and data steward responsible for ensuring that corporate data assets are discoverable, available, and accessible while guarding against inappropriate access.

These types of challenges will not be easily overcome as long as organizations do not revisit their data architecture and consider updating their data strategy in the context of business-driven data governance. Acknowledging some key challenges will help in motivating how a future data strategy is better able to address immediate business needs:

• Data organization: Because of the rapid increase of data assets to be subjected to analysis, it is difficult for data stewards to adequately capture and manage the lifecycle associated with information. That suggests that the data strategy should establish methods and practices for logical semantic organization of data artifacts. Collect the metadata that supplements searchability and discoverability of the information contained within each data asset and store those assets in a way that simplifies finding and accessing the data.
• Streamlined data flow: Delays in availability of and restrictions imposed on use of the data limit what the business groups might be able to achieve with the data they ultimately acquire from IT. The data strategy should incorporate ways of documenting information flows and determining where potential bottlenecks might limit data utilization.
• Data quality and validation: Traditional data governance expects data stewards to profile and cleanse data before those data sets are made available. Broader data utilization complicates this because different users may have different rules to be applied regarding data standards and quality.
• Data protection: Because different users have different access rights, a one-size-fits-all approach to data security and protection may not be sufficient. In a modern data strategy, rules associated with the appropriate transformations for masking and de-identification must be defined and applied to ensure that the organization meets the security and data protection compliance requirements.
• Data transformation: Again, to satisfy the different user needs, each data consumer should have the ability to adopt and apply transformations to enable analysis. That suggests that the data strategy should incorporate a means for freeing the users to define their own data transformation scripts yet ensure that none of the data preparation processes allow for conflicting results.

An organization’s marketing and strategy groups need to be able to think out of the box, and work with their organization’s data in an unhampered way. Fortunately, there are emerging technology innovations that can help address some of these challenges, ranging from data governance tools, data catalogs, collaboration environments, and even clever adaptation of cloud computing and storage platforms. In upcoming posts we will explore ways that these and other technologies can augment the enterprise platform to enhance data awareness and improve data utilization.