The market for business intelligence (BI) tools is crowded. There are many technologies and products clamoring for customer attention. Figure 1 below shows ten categories of BI technology that exist in the market today, most of which overlap to some degree. Organizations need each of these technologies to deliver a comprehensive set of BI services. (See the appendix for a description of all ten categories.)

However, organizations don’t need to purchase ten different tools from ten different vendors. Many vendors offer suites of BI tools that span multiple technologies, although most BI vendors only offer one or two technologies. Thus, it is possible for a company to standardize on a single large BI vendor whose toolset encompasses all nine technology categories. But this may not be the best strategy since an “all-in-one” solution does not necessarily deliver “best of breed” products.

Figure 1 - The BI Tools Landscape

Top Down versus Bottom Up

The size of the ovals in figure 1 indicates the degree to which each technology category spans a range of characteristics in three dimensions: functionality, scope, and types of users. The horizontal dotted line refers to the demarcation between “top down” and “bottom up” BI.

Top down. The top-down world of BI delivers reports and dashboards to casual users (e.g. executives, managers, and front-line workers) who use information to do their jobs. Eighty percent of the time, these users simply want to monitor business processes and metrics for which they are held accountable. An interactive dashboard is the ideal information interface for these users because it enables them to quickly view the status of their processes and drill into more detail if needed. The top-down world is generally driven by IT which adheres to a systematic process of gathering requirements, creating data models, and delivering data warehousing-centric applications that are designed to support on-going needs.

Bottom up. In contrast, bottom-up BI is designed to meet the needs of power users (e.g. business analysts, statisticians, and data scientists) who are hired to analyze information on a daily basis. These users explore data in an ad hoc fashion and then manipulate that data with analytical algorithms to answer new questions that aren’t addressed in the standard reports and dashboards of the top-down world. The bottom-up world is generally driven by business analysts who piece together data from a variety of sources, both inside and outside the organization, using low-cost analytical tools to meet immediate or pressing needs. 

Two Goliaths

Out of the nine categories of BI technologies, two stand out as the most dominant: relational OLAP (ROLAP) and visual discovery. These two technologies—which come from ends of the BI spectrum—offer greater scope and functionality and meet the needs of more types of users than the other technologies. ROLAP hails from the top down, IT-driven, enterprise-oriented world of data warehouses and data marts, while visual discovery technology hails from the bottom-up, analyst-driven, departmental-oriented world of spreadsheets and analytical models. These two technologies are now on a crash course in the battle for supremacy in the BI market.

Visual discovery. ROLAP tools have a long history in the BI market, but visual discovery tools come from aggressive upstarts that harbor not-so-secret ambitions to dominate the BI market. Led by vendors such as Tableau, QlikTech, and Spotfire, this new category of BI tools has sparked a swift rearguard action among traditional enterprise BI vendors which have all shipped visual discovery products in the past 12 months to protect their flanks from the nimble invaders. (See my latest report, “Visual Discovery Tools: Market Segmentation and Product Positioning.”)

Two years ago, the visual discovery oval was only half as big as it is today because the tools were primarily used by individual analysts for personal use and build departmental dashboards. But because of its ease of use, low cost, and visual interactivity, visual discovery technology has spread quickly. It has already subsumed the desktop analysis category and most of the multidimensional OLAP category (except write-backs and complex dimensional calculations), and is now threatening to move into the domain of ROLAP tools as well as almost every other category of BI tools. (See figure 1.)  

Self-service BI

The market for visual discovery is expanding rapidly because the technology makes triply good on the promise of self-service BI, which has been a BI holy grail for nearly 20 years. The tools not only empower business analysts to explore and analyze data without boundaries, but also create interactive dashboards for departmental colleagues. This dual function gives the tools a stealth opening into the world of top-down BI. Moreover, visual discovery dashboards empower casual users to do more with their sandboxed data than just view it: they can navigate and modify the data, and collaborate with others, among other things. So visual discovery tools pack a powerful wallop: they support individual analysis, dashboard authoring, and dashboard analysis.

You would think that the visual discovery vendors would be content with their triple play, but they aren’t. They are moving rapidly into enterprise BI markets long dominated by ROLAP and reporting tools, pulled by their biggest customers who want to standardize on the tools throughout their organizations. Visual discovery tools also are impinging on other adjacent BI categories, such as data mining, big data analytics, and operational reporting.

A Path to the Future

Given the market landscape described above, what should a BI vendor do to grow market share? The natural evolution of BI products is to migrate from desktop to departmental to enterprise segments; in essence, from the lower-left side of the quadrant chart to the upper right. (See figure 2.) Even ROLAP tools started as desktop products (remember MicroStrategy in the early 1990s??) and then moved up the evolutionary ladder from departmental to enterprise solutions. For better or worse, visual discovery tools are tracing the same path. (Sometimes I wish departmental BI vendors would remain content as departmental players instead of taking the risky journey into the enterprise.) Departmental vendors that make a successful transition figure out how to balance the very characteristics that made them successful with new enterprise features.

Figure 2. Evolution of BI Products

Segmenting the market. However, it’s impossible for one product to appeal to both desktop, departmental, and enterprise users. The requirements conflict. Departmental users want BI tools that are easy to use, fast, quick to deploy and low cost. Enterprise users want scalability, reliability, security, and reusability. The key is not to meet the needs of both markets in one product but to create separate products, one for each market that is packaged, branded, and priced distinctly. It’s also important that vendors provide a seamless migration path from one product to the other, not just for the software, but for data and metadata as well.

Marketers need to have a clear picture of the target customer in each market and the words and features that resonate with each, such as those mentioned above. They need to remember that departmental customers buy software to satisfy an immediate need. Generally, they are business people trying to solve a specific business problem. They don’t want or need to concern themselves with architecture and other long-term systems requirements. They don’t care who supports them, IT or a third party, as long as they get answers to issues fast and at an affordable price.

In contrast, enterprise customers buy software to address long-term needs. As such, the IT department plays a major role in the buying decision since they need to scout out whether the software aligns with the company’s existing infrastructure and architectural requirements. On the business side, the buyers want to know that the vendor is a leader in the industry and has a vision for the future. They want to see the vendor “check off all the boxes” on a request for proposal for features and functions that may or may not ever need. In other words, they want the BI vendor to show how they support all nine categories and delivery channels (e.g. mobile, cloud, Web, desktop) as well as the latest technologies (e.g. Hadoop, social media, analytics). This is a tall order for BI vendors and sucks up a lot of research and development, especially for departmental vendors leaping into the enterprise market.

Today’s batch of BI players generally adhere to these market principles. Yet they are still vulnerable to competition from visual discovery competitors because of the self-service triple play described above. Although top-down BI products provide self-service BI, the IT department needs to first create a semantic layer or populate the mashboard library with predefined objects before business users can author or interact with the software in a self-service manner. Trying to push down self-service from the top-down world is a lot harder than pushing self-service from the bottom up.

Market Vectors  

Given the breadth of the BI market and its current velocity, vendors need to follow one of three market vectors, while not losing sight of the others. These vectors are operational analytics, enterprise dashboards, and big data analytics. (See figure 3.) It will be hard to dominate all three vectors, given the competition and speed of product evolution in each sector.

Figure 3. Emerging BI Vectors

1. Enterprise dashboards. This represents the sweet spot of the BI market because enterprise dashboards meet the information needs of the greatest number of business users—casual users who simply want to monitor their areas of responsibility. These dashboards can be simple, if departmental in scope, or complex, if enterprise in scope. The degree of difficulty largely has to do with the degree of data governance required to deliver trustworthy data and the challenge of delivering adequate performance as the number of users and volume of data and metrics in each dashboard increase.

2. Real-time analytics. Products in this fast-growing sector monitor and analyze data in motion versus data at rest—in other words, they apply rules and calculations to real-time event streams, such as those emanating from Web and systems logs, sensor networks, digital devices, and commercial machinery.  This sector sits at the intersection of analytical intelligence (e.g. data mining) and continuous intelligence (complex event processing). (See my BI Framework 2020 for a description of these intelligences.) This is a small market but one that is finally gaining traction with customers.

3. Big data analytics. Big data analytics is another hot market, fueled by interest in Hadoop and NoSQL technologies. This space is wide open right now and vendors from three distinct markets—database, BI/visualization, and analytics—are battling for hegemony. My bet is on products that combine all three capabilities in a single integrated toolset that exploits the latest advances in CPU, memory, visualization, and parallelization and can be purchased along with the hardware (e.g. appliance or cloud.) This aligns with the macro trend in the high tech industry toward convergence. During the 1980s and 1990s, the industry broke apart computing monoliths (e.g. mainframes) into discreet commercial components, and today, it is bringing them back together again. We’re going full circle.

Departmental strategies. Desktop and departmental BI vendors will need to align closely with one of these vectors. They simply don’t have the resources to play in all of them, especially if the aspire to move into the enterprise space, which will suck up valuable resources beefing up their metadata, security, clustering, and other enterprise capabilities. For instance, it would make sense for QlikTech to track the enterprise dashboards vector, while Tableau pursues big data analytics, and Spotfire chases operational analytics. 

Enterprise strategies. Enterprise BI players will need to play in all three, but specialize in one. They will also need to offer desktop and departmental tools to round out their portfolios and provide an attractive and affordable on-ramp for future enterprise customers. Obviously, the challenge for enterprise BI vendors is how provide compelling product value and deliver clear and persuasive marketing messages while chasing all three vectors and downstream markets. This will not be easy. It will take a clever marketing officer with a deep understanding of customers, markets, and messages to thread this needle.

Summary

The BI market is complex and evolving. Among the nine distinct categories of BI technologies, two have the largest footprint in customer accounts—ROLAP and visual discovery—and both are contending for supremacy in the BI market. ROLAP is the preeminent top-down technology, while visual discovery is the preeminent bottom-up technology. Both make good on the promise of self-service BI, but visual discovery offers greater versatility by empowering both casual and power users to analyze data and author dashboards.

But no technology or product can thrive without people who understand the current dynamics of the BI market and know how to position their products accordingly. Desktop and departmental vendors need to bridge the chasm between departmental and enterprise solutions while pursuing one of three emerging BI vectors. Enterprise players need to span all BI technologies and emerging vectors while retaining clear, crisp messaging for each.

APPENDIX: THE BI TOOLS FRAMEWORK

The BI Tools Framework (see figure 1) positions categories of BI tools by functionality, scope of deployment and type of intelligence (top-down versus bottom-up). To deliver a comprehensive set of BI capabilities, organizations need to implement BI products or suites that span all dimensions of the BI Tools Framework.

Here are definitions of the key terms in the framework:

Top-down intelligence: Top-down intelligence consists of reports and dashboards that answer predefined questions or monitor business processes using metrics aligned with strategic objectives. Top-down tools query data sets with predefined schema, including data warehouses and data marts. Top-down tools are installed, designed and maintained by the IT department but operated by power users and super users.

Bottom-up intelligence: Bottom-up intelligence consists of ad hoc analysis and mining tools that answer unanticipated questions arising from new and changing business conditions. Bottom-up tools query any type of data in any application or system and use flexible schema and rich visualizations to effectively manipulate and analyze data. Bottom-up tools are used by business analysts, statisticians and data scientists with minimal or no IT assistance.

Top-Down Business Intelligence Tools

Enterprise reporting: Also called production or managed reporting, enterprise reporting tools enable professional report developers to create pixel-perfect and complex reports and distribute them to large numbers of users on a scheduled basis.

Operational reports: Unlike ROLAP tools, which connect to data via specialized schema and architected metadata, operational reports and dashboards tools connect directly to source systems that run the business. Also called data mashups, these tools are used to create operational or executive dashboards that monitor near-real-time business activity across multiple applications and systems. Some higher-end operational dashboards monitor real-time event streams.

Ad hoc reports: These tools enable motivated business users, that is, power users, to create ad hoc reports by selecting data elements (metrics, dimensions, attributes) from a scrubbed list of data objects (semantic layer) and then format the result set.

Relational OLAP (ROLAP): Relational OLAP tools provide a dimensionalized view of data held in relational databases, using a combination of specialized schema and end-user metadata, that is, a semantic layer. IT developers use ROLAP tools to create interactive reports and dashboards that run against large data warehouses. Because ROLAP tools span reporting, dashboarding and analysis, they are ideal for delivering layered enterprise dashboards that provide three-click access to any data from a top-level view, assuming adequate performance.

Mashboards: Mashboards are ad hoc dashboards that users createby dragging and dropping predefined content (charts, tables, controls) from a library onto a dashboard canvas without coding or scripting. Objects automatically link and filter each other using common keys.  

Bottom-Up Business Intelligence Tools

Desktop analysis: Epitomized by Microsoft Excel (and now PowerPivot), desktop analysis tools are designed for business analysts who want free-form SQL access to any data and the flexibility of a desktop tool to merge, integrate and model data to answer any question they need to answer.

Visual discovery: Visual discovery tools are self-service, in-memory analysis tools that enable business users to access and analyze data visually at the speed of thought with minimal or no IT assistance and then share the results of their discoveries with colleagues, usually in the form of an interactive dashboard.

Multidimensional OLAP: These tools store data in a specialized multidimensional format and either precalculate or dynamically calculate data values at the intersection of every level in the dimension hierarchy, providing fast query performance. Designed for business analysts, the tools require IT professionals to set up, design, load and manage the databases. They are constrained in size because of the explosion of data caused by the calculation of dimensional values.

Data mining workbench: These tools provide a drag-and-drop development environment for statisticians who need to design, build and manage analytical models, either individually or in a team.

Big data analytics platforms: These are highly scalable analytical systems that run on grids of computers, including Hadoop and MPP database appliances, and contain libraries of analytical functions that enable statisticians and data scientists to explore, manipulate and model large volumes of data without having to move the data to a workstation or departmental server. They also support analytical sandboxes that enable analysts to upload private data and mix it with corporate data to conduct analyses as well as strong visualization capabilities.