This is the first in a four-part series on enterprise business intelligence.

The battle for supremacy in the business intelligence (BI) is over: visual discovery tools have won.  Riding the coattails of upstart Tableau with its current $6 billion market cap, visual discovery technology now dominates the BI landscape. (See market segmentation below.)

As evidence, every BI vendor has introduced a visual discovery tool in the past three years. And many top BI vendors now trumpet visual discovery as their flagship BI product. For example, IBM Watson, SAP Lumira, Qlik Sense, and SAS Visual Analytics now take center stage at their respective companies, overshadowing their BI forebears.

Visual discovery tools have skyrocketed in popularity because they meet the needs of both business analysts and departmental users. As self-service tools, they enable business analysts to connect, blend, visualize and analyze data without IT assistance. In addition, business analysts or BI developers can create attractive interactive dashboards for departmental users. Consequently, business units no longer need to wait for corporate BI teams to meet their needs—they get the data they want, when they want it, and how they want it.

In other words, visual discovery tools span both top-down and bottom-up BI or multiple modes of BI, as Gartner calls it, meeting a large swath of business requirements. (See figure 1 below.) The only thing these tools don’t do is pixel-perfect reporting and workflow-driven analytic applications, but expect those capabilities soon.

Scaling the Enterprise

With victory in hand, now the hard part begins.

Although visual discovery tools excel at self-service, they generally lack enterprise features. Fortunately, late entrants in the visual discovery space recognize this gap and have architected products that promise to deliver both self-service and enterprise BI features. (See my next blog, “Part II - Ten Features of a Modern Enterprise BI Tool.”)

As such, visual discovery tools are following the natural trajectory of all dominant BI technologies: they evolve from desktop to departmental to enterprise products. (See figure 1.) OLAP tools followed this trajectory in the 1990s and 2000s, and now visual discovery tools are following suit, eclipsing the primacy of OLAP tools in the process. Fueling this ascendancy into the enterprise are large customers on one hand and capital investors on the other.

Figure 1. BI Market Analysis - 2015

Since I introduced this chart in 2011 to explain BI market segments, the visual discovery oval has dramatically increased in size, swallowing up nearby market segments and making significant incursions on every other segment. Like OLAP technology before it, visual discovery tools are following the trajectory marked by the blue arrows, rising from desktop to departmental to enterprise class technology. The question is whether visual discovery tools can balance the diametric needs of casual and power users as well as departmental and corporate groups.

Crossing the Chasm

Desktop to Department. The path from a desktop to departmental class tool is straight and smooth. Desktop BI vendors add a server—either on premise or in the cloud—so analysts using desktop tools can publish reports and dashboards for departmental colleagues to consume. The output remains live, giving departmental users a rich, interactive, analytical experience. And if they need a new view, they simply walk down the hall and cajole the analyst to create it.

Department to Enterprise. But moving from a departmental to enterprise class product is a tortuous journey and monumental challenge. Many once formidable BI products have died unseemly deaths along the way. Although departmental products only need to meet the needs of one group, an enterprise product must satisfy the demands of many, each with a different view of the world and different language and metrics to describe it.

Moreover, they’ll need to support a panoply of business requirements—from reporting, dashboards and analytics to alerting, bursting, and collaboration. And they’ll need to scale to support thousands of concurrent users; support enterprise-scale administration, management, monitoring, and security requirements; and adhere to stringent SLAs for performance, availability, reliability, and security, and application integration—in other words, meet the needs of both users and corporate IT.

Product Makeover. Meeting these types of enterprise requirements often requires departmental products to undergo wholesale makeovers. They often must re-architect both server and administrative environments, creating upgrade and migration headaches for loyal customers. Aggravating matters further, many use venture funds or IPO windfalls to acquire new technologies rather than build functionality organically, potentially creating an architectural mishmash that further frustrates long-time loyal customers.

The Long Trail

For visual discovery tools to succeed in the long term, they’ll need to balance diametric needs of speed and standards, flexibility and control, self service and governance. Finding a suitable middle ground between the long-term needs of the organization and the immediate needs of individual business users is not easy. There are tradeoffs with every solution. But this is the challenge of delivering BI in the 21st century.

The last generation of market-leading BI tools over-emphasized enterprise features and consequently fell into disrepute; they became easy targets for visual discovery tools in the battle for BI supremacy. Now that the war is over, vendors now face the real battle of turning visual discovery tools into enterprise products that balance the need for both freedom and control.

Read - Part II - Ten Characteristics of a Modern BI Tool