There is a revolution going on in the world of business intelligence (BI). The balance of power has shifted from the information technology (IT) department to business units. Meanwhile, big data and advanced analytics have grabbed the limelight and soaked up discretionary spending for information management initiatives.

Given these tectonic shifts, some say BI is “old school” and data warehouses are data  “dead ends.” But don’t listen to these new age critics; BI leaders still have a critical role to play. But rest assured, it is a new role, and one that BI leaders may not be accustomed to playing.

The New Role for BI Leaders. To survive, if not thrive, in the new world order, BI leaders need to do three things:

• Acknowledge that the data warehouse is no longer the center of the information universe.
• Partner with information delivery peers across the organization to create a data analytics center of excellence.
• Design an analytical ecosystem that unites the people, processes, and technologies involved in turning data into insights and action.

These three tasks are not for the timid. They require strong leadership and communications skills and a vision for how data and information can transform an organization. The new BI leader straddles the worlds of business and technology and proactively knits them together into a coherent whole.

1. Center of the Universe

For the past 20 years, the only way to deliver information to decision makers was by building a data warehouse and disseminating BI tools to business users. Today, the data warehouse is just one of many data structures in a decision support ecosystem. And that’s a good thing.

Truth be told, we asked the data warehouse to do too much, and it groaned under the strain. A data warehouse is a repository of certified, trustworthy data designed to answer predefined questions via interactive reports and dashboards and support ad hoc dimensional analysis. It’s not optimized to support unfettered exploration and discovery outside of the dimensions and facts contained in its models.

Because of the recent influx of new big data analytics technologies, a data warehouse no longer has to be the alpha and omega of decision support. It can assume its proper role in the information pantheon, which is to support the core reporting and analysis requirements of business managers and executives and their business analysts.

2. Partner, Partner, Partner

In the new world order, the BI team is just one of many that delivers information to business users. To succeed in this environment, BI leaders need to proactively identify all relevant players, coordinate their activities, and facilitate the creation of a shared vision of how to use data and analytics to drive the business. The new BI leader needs to partner with:

• Head of advanced analytics. This person oversees a team of data scientists, data analysts, and statisticians. The team is a heavy user of data that often creates its own data environment, usually in Hadoop, replicating much data that already exists in the data warehouse.
• Business unit heads. They hold the purse strings for all new projects and pay an allocation for shared services that support the BI and DW functions.
• Business unit CIOs. Larger companies typically have CIOs dedicated to each business unit who try to bridge the proverbial business-technology gap. They review and approve new technology initiatives for the business unit.
• Functional analytics managers. The managers of financial analysts, pricing analysts, operations analysts, and so on are key players. Their analysts are the heavy users of data and analytics and have an intimate knowledge of business needs and requirements.
• Chief data officers. Many large companies and government agencies hire chief data officers to comply with new governance, security and privacy regulations. CDOs and their teams of data stewards control the dissemination and usage of critical information.
• Hadoop administrators. Typically in IT, these individuals create the next generation of data infrastructure using Hadoop, NoSQL, in-memory and other new technologies. These folks are the point people for new technology investments.
• Security officers. These individuals often have to review and sign off on new data-intensive technology investments to ensure proper data security and privacy.
• IT managers. These folks oversee the database, network and storage systems administrators who make a data center function. They often determine the schedule and process for moving new applications into production.
• Auditors. In a heavy regulated industry, auditors often trigger new technology investments or prompt organizations to rearchitect existing environments to optimize business benefits and minimize risks.

Center of Excellence. As an advocate for the enterprise use of information, the new BI leader needs to take the lead and bring together the above players to chart a shared information strategy for the company. In the process, the new BI leader spearheads the creation of a center of excellence devoted to data and analytics. This is no small task and requires people with exceptional vision and leadership skills.

3. Design an Analytical Ecosystem

One of the first tasks of the new data analytics center of excellence is to create a data ecosystem that unites the technologies and processes that turn data into insights and action.

This data ecosystem is much more complex than a decade ago when the data warehouse was the only game in town. Today, we have Hadoop to collect and analyze multi-structured data; in-memory databases and appliances to support high-performance queries; NoSQL databases to serve up operational data or analyze entity relationships; data integration technologies to support streaming, replication, virtualization, and bulk loading of data; analytic sandboxes to support ad hoc analysis; self-service tools to empower business analysts to access, integrate, analyze, visualize and report on data.

In short, our BI environment has become a lot richer and more functional, yet a lot more complex. It takes much discussion and planning to craft an appropriate data analytics environment.  Below is an example of analytic ecosystem that I show to many clients, some of whom adopt it wholesale as a framework for guiding the development of their next generation data environment.

Conceptual Analytic Ecosystem

The ecosystem depicts four intelligences—business intelligence, continuous intelligence, analytic intelligence and content intelligence—each with distinct architectures, tools, and development environments. The framework is divided into top-down and bottom-up BI environments, each of which is geared to casual and power users respectively. Each box can be implemented using any technology either as a stand-alone environment or collectively with two or more boxes in a shared server environment.

Hadoopis increasingly used as the staging area for landing new data, especiallymulti-structured data, as well as a hub for disseminating data to downstreamsystems or virtual views to support ad hocanalytics. It’s also used as a low cost archive for storing data that no longerfits cost-effectively in the data warehouse.

The data warehouse is represented by the “business intelligence” box. Although it straddles top-down and bottom-up environments, it is not the center of the new analytical ecosystem. The data warehouse is now just one of many component parts.

Continuous intelligence systems may run inside Hadoop or Spark or specialized complex event processing, streaming or analytical applications. If business requirements require near real-time (i.e. 15 minute updates) versus real-time or continuous updates of thousands of events per second, then a data warehouse with micro-batch updates or trickle feeds may suffice.

Analytical systems are sandboxes that range from views inside Hadoop to partitions inside the data warehouse to analytical appliances or in-memory machines running on Hadoop or relational databases.  

Summary

To stay relevant in a fast-changing business and data environment, BI leaders need to recognize that the data warehouse is no longer the center of the data analytics universe. Consequently, they need to reach out to other data analytics players in the organization and partner with them to create the analytical ecosystem of the future. This encompasses both a new center of excellence that formalizes relationships among all players and a data architecture that stitches together all the tools and technologies required to meet the full range of data analytics requirements.