The Rise and Fall of the Data Professional Class?

The Rise and Fall of the Data Professional Class?

We hear a lot these days about the new data professional class, with large corporations rushing to add chief data officers and other data specialists to their C-suite list. Wall Street firms are setting up operations in Silicon Valley to secure the talent they feel is needed to compete in the market. The data professional is being extolled in universities and the mainstream press as the career of the future. But the emergence of a new professional class of data experts, complete with their own jargon and associations, comes with challenges of a different sort. Let’s step back a moment for some perspective.

Back when I became a data professional, data was largely one responsibility among many facing computer programmers and systems analysts. My first position was in what was then called Electronic Data Processing (EDP). At that time, data was not yet the exclusive purview of the specialist.

As corporations began to view data as “information” and as a corporate asset, they began to develop more specialized roles to organize and manage data in a more systematic fashion. A new class of data professionals emerged with titles that included the data architect, data analyst, data modeler, database administrator (DBA), and data steward. The data warehouse, meanwhile, emerged as centralized approach to corporate data management.

A potential downside of specialization: a diminution of perspective and understanding. Goodbye data ‘Renaissance Man’. Before the advent of the data specialist, business and technology executives had broad responsibilities that cut across many domains. One domain was data, and how data fit into the larger business picture. One consequence of professional classes can be the development of jargon that may be impenetrable to the lay person or outsider, and establishment of new dogmas that serve the status quo and self-preservation, often at the expense of innovation, creativity, and discovery – the very cornerstones of data insight.

But there are signs that the pendulum is swinging back. Cutting against the grain of a growing data professional class today is the trend toward ‘democratization of data’. Organizations have struggled for decades with the challenge of how to make data accessible to business decision makers in a timely and effective fashion. In the 1980’s, it was Executive Information Systems (EIS); in the 1990’s and 2000’s, Business Intelligence (BI). Big Data approaches arrived with the promise of delivering data into the hands of business analysts and business decision-makers quickly. With this trend, responsibility for data is moving outwards, away from the data specialist, and into the hands of laymen. Power over data is shifting from the technologist to the business person.

Those organizations that are proving most successful in extracting insights from their data foster strong business and technology cooperation and coordination. Working together in partnership toward a common outcome, these organizations offer a prescription for success in realizing data value.

As data proliferates, and as the demand for data increases to match this proliferation, business people will demand greater access and control over the data they need to make business decisions. Over time, data processes will become embedded within mainstream business processes, and data professionals will progressively be absorbed into the business mainstream.

Edward Gibbon, in The History of the Decline and Fall of the Roman Empire, wrote of “the vicissitudes offortune, which spares neither man nor the proudest of his works, which buries empires and cities in acommon grave.” Will the data professional class also become a relic of the past?

This article first appeared in The Wall Street Journal on September 22, 2015.

Randy Bean

Randy Bean is CEO and managing partner of NewVantage Partners, a leading provider of data management and analytics-driven strategic consulting services to Fortune 1000 firms, and a leader in Big...

More About Randy Bean