How many times have you heard a statistic such as “42% of respondents rate their Business Intelligence (BI) program as moderately successful” or “more than 50% of all Business Intelligence projects fail”? When I read these types of statistics I often wonder what’s behind these numbers. I’d love to be able to drill down directly to the respondents and ask them how they defined success or failure when answering the question. I recently met with a company that asked if I could come in and help make their Business Intelligence program more successful. Naturally, my first question was to ask them how they define success. Each person in the room defined success a bit differently and the meeting turned into a healthy discussion on what constitutes a successful Business Intelligenceprogram.

One of the Business Intelligence industry’s big challenges is that the definition of success has remained relatively ambiguous, and in some cases, left in the hands of the tool vendors. The industry has done a good job at identifying the general contributors of success, such as better access to data, number of users, increased quality of data, better decisions and cost savings. The challenge is that these contributors in themselves don’t necessarily determine the success or failure of a Business Intelligenceprogram, and are rarely measured. For example, how many Business Intelligence programs are considered a failure because they were 3 months late delivering the initial data warehouse or have less than 5% active users in their company.

As a starting point for discussion, I am going to make a definitive, and likely controversial statement of what defines success. In my experience, there are two components that determine the success of a Business Intelligence program; reality and perception. Reality can drive perception, and unfortunately, perception can sometimes drive reality.

Reality

From a reality perspective, the success or failure of a Business Intelligence program can only be measured by its Return on Investment (ROI). Why? Because companies are in the business to make money and if they don’t, they will ultimately fail. A Business Intelligence program is an investment that a company makes in hopes of achieving a positive return. With that in mind, I contend that a Business Intelligence program that achieves a negative ROI over time is considered a failure. Ongoing ROI should be calculated each year, and it takes into account all increased revenues, decreased costs as well as capital and operational investments. In a recent TDWI keynote, Ken Rudin, who is the Head of Analytics at Facebook, made the case that the goal of BI should be about impact, not simply enabling insight. Insight in itself can’t be measured, but impact can.

Perception

So how does perception drive reality? If the target users, whether in the business or IT, perceive the data to be inaccurate, incomplete, unreliable and/or inaccessible, and the tools offered are too complex, exhibit poor performance, lack required capabilities, or are incompatible with how the users work, they will look to other solutions for their reporting and analyses. In many cases, this is more of a perception than reality. However, if the analyses that lead to increased revenue or decreased cost are not attributable to the Business Intelligenceprogram then it will likely be considered a failure. To avoid this situation, better training, marketing, communication and governance are required.

A Story from the Trenches

A few years back I was the Director of Data Warehousing for a large manufacturing company and each month the CIO used to ask me how many active users we had on thedata warehouse. He believed that the program was failing because less than 40% or our targeted constituents were using the data warehouse. Then our team embarked on a project sponsored by the Vice President of International Sales to identify high growth customers and create a set of targeted analyses and reports for his field representatives. Ultimately, he attributed 3% of the growth, or $24M that year in international sales to the work of the Data Warehouse team. Our CIO stopped asking how many active users we had and starting focusing on the ROI generated by the Business Intelligence program.

This article was first published on September 10, 2013 on datasourceconsulting.com