The revolution in big data analytics has some traditional business intelligence (BI) teams questioning their role, if not their existence. In the age of self-service, BI teams need to be more than just a development shop that supplies data and applications; they need to serve as a combination coach, guide, and service provider.

Monochrome to Kodachrome

When television entered American homes en masse in the 1950s, the television sets were huge, screens tiny, and displays black and white. By the 1960s, color broadcasts supplanted monochrome ones, kicking off a second media revolution.

The same is happening in business intelligence: we are moving from a monochrome world of data delivery to a kodachrome one. The data warehouse has given way to a big data ecosystem and self-service BI has supplanted IT-driven applications. These fundamental changes, along with a host of new faster, better analytical technologies, have changed the look, feel and functionality of BI in a dramatic fashion. (See figure 1.)

Figure 1. The New World of BI

Given these new radical changes, how has the role of the BI team changed? What is left for a BI team to do if business users can buy, download, install, and build their own queries, dashboards and reports? The short answer: plenty. Here are three things that BI teams must do to succeed in the kodachrome world of data analytics.

1.    Deliver Trustworthy Data

In some ways, the role of the BI team has not changed at all. Its primary task today—as it has been since the dawn of analytic computing—is to deliver a trustworthy repository of clean, integrated, business-oriented data. Whether you call this a data warehouse or big data ecosystem, doesn’t really matter. Whether you source data from SAP and Oracle or Twitter and Facebook, is not relevant.

No matter what architecture or type of data, every organization needs skilled data professionals who can architect, build, and maintain a repository of trustworthy information that business users can use to devise plans, troubleshoot issues, and make decisions. In fact, the demand for skilled data architects and data integration specialists is higher than ever. It’s just that these folks need to upgrade their technical skills to deal with Hadoop, Hive, and other open source technologies. If they do so, they can also upgrade their titles to “data engineer” or “data analyst” or even “data scientist.”

2.    Support Business Analysts

Secondly, BI teams need to shift their focus from meeting the needs of casual users to meeting the needs of power users. On this score, BI teams have failed miserably in the recent past. Since it’s nearly impossible to gather requirements from business analysts, most BI teams punt and simply deliver dumps of data to business analysts upon request. This manual, non-automated process is laborious and time consuming and does nothing to free business analysts from reliance on the BI team.

Analytic Sandboxes. A better approach is for the BI team to build analytical sandboxes where analysts can mix corporate data with their own local data under the supervision of the BI team. Here, analysts get the data they need when they need it, and the BI team gets to observe what the analysts do and adapt the data warehouse to better meet their needs. There are many types of analytic sandboxes. Some exist as managed partitions inside a data warehouse; others as analytic appliances running replicas of the data warehouse; while newer ones run as Hive or columnar tables inside Hadoop.

Data Catalog. Just as important, the BI team needs to create and maintain a catalog of the data warehouse and other relevant data sources so business analysts can find data they need without assistance. More than a dictionary, a data catalog allows business analysts to mark tables and fields with tags, ratings, and comments so other analysts can learn from their work. This creates a living document that supports bottom-up data governance.

Just-in-time Training. The BI team also needs to train business analysts how to use the data catalog, navigate the data warehouse and use self-service BI tools. Ironically, empowering business analysts to be self-sufficient with data and BI tools requires a lot of hand holding. BI teams can host internal user groups, serve up lunch and learn sessions, and provide formal training classes. Of course, the best training happens spontaneously when BI professionals co-locate with business analysts.

Production Reporting. Finally, BI teams need to convert ad hoc reports produced by analysts into production reports with standard tools and data and the requisite scalability, security, and accuracy. Today, this hand-off rarely happens unless BI professionals sit side by side with business analysts and work on the same team. Co-location as part of a federated organizational structure pays many dividends.

3.    Foster Relationships Not Requirements

The final and most important role of the new BI team is to foster a tight working relationship with business units. A federated BI team that embeds BI developers into business units while maintaining centralized oversight is a critical first step, as mentioned above.

Requirements Analysts. It’s also critical to convert the BI requirements analysts from order takers to consultants. Too many requirements analysts enter business units stone cold—without business contacts or context. Without prior knowledge of the business unit’s processes, politics and people, these analysts can only take good notes and translate business needs into reasonable development specifications.  Usually, something gets lost in the translation, frustrating both business users and the BI team.

Relationship Managers. A better approach is to turn business requirements analysts into relationship managers who spend most of their time in one or two business units, attending strategy meetings and talking to business users. By immersing themselves in an organization, relationship managers can proactively suggest technical solutions to business issues without conducting formal requirements. When BI professionals proactively suggest pragmatic and affordable solutions that can be deployed quickly, business people see the BI team as a partner to their success.

Summary

In the new kodachrome world of BI, the role of the BI team has changed. Rather than serve as the only conduit to data and applications, BI teams have three fundamental tasks: 1) deliver trustworthy, integrated data 2) support business analysts with sandboxes, data catalogs, training, and production reporting and 3) develop close working partnerships by co-locating BI developers and relationship managers in the business units they serve.