Read - How to Succeed with Self-Service Analytics, Part I: Pitfalls and Paradoxes

Read - How to Succeed with Self-Service Analytics, Part II: Know Thy Customer

Read - How to Succeed With Self-Service Analytics, Part III: Trust But Verify New Reports

Having the right standards and governance processes is critical to self-service analytics, as we discovered in the prior article. But just as important as governance processes are the teams and people that execute those processes. In other words, self-service analytics requires a carefully crafted federated organizational model with strong oversight and distributed development.

Self-service analytics requires the right people in the right roles doing the right things throughout an organization. This enables organizations to “push down” development and support from the corporate team to embedded local developers and business users. But this push-down strategy only works if there is also a well-planned and highly choreographed process for promoting and prioritizing requests at the corporate level that are either too small or too big for local resources to handle.

It also requires cross-functional oversight of the enterprise program to ensure that all voices are recognized and heard, all needs are scoped and prioritized, and everyone uses the same standards and governance processes. As you can see, self-service analytics has a lot of moving parts—it requires a skilled leader with the skills of a circus juggler to keep all the plates spinning without any crashing to the ground.

Oversight Committees

At many companies, each business unit or department acts as its own arbiter of data and technology standards and is responsible for communicating those standards, approaches, and practices to workers and educating them. Not surprisingly, this decentralized approach creates data chaos and silos that undermine data consistency and process effectiveness at the enterprise level. Once disconnected teams experience enough data pain, they recognize the need for centralized governance.

Data Analytics Council. To tame the chaos, representatives from each department join forces—often at the request of the head of the corporate data analytics (CDA) team. This grassroots group forms the core of a Data Analytics Council—an enterprise oversight committee that serves as a board of directors for data analytics. It manages one or more of the formal processes listed in table 1, with direction and support from a chief data officer or VP of data analytics.

Table 1. Governance Processes Required to Support Self-Service Analytics: 

The Data Analytics Council usually consists of a working committee composed of analytics managers from each business unit and an executive committee consisting of business sponsors. The working committee is the workhorse of the council since it’s comprised of analytics managers who experience the “pain of bad data” every day. (See figure 1.)

Figure 1. Data Analytics Council – Composition and Responsibilities 

A working committee may have subcommittees that tackle various governance processes, such as reviewing and approving data definitions, prioritizing projects, governing reports, governing analytics, setting tools standards, and creating a data literacy or training program, among other things. Sometimes, independent groups manage these processes, such as a data governance committee or prioritization board. For example, the Data Analytics Council may define terms critical for analytical processes and pass them to the enterprise Data Governance Committee for consideration.

Informal Processes 

To take root and flourish, self-service analytics also requires informal processes. Here, knowledge trickles down, while requests trickle up.

Trickle Down. The CDA team trains and coaches embedded data analysts residing in business units. (In the ideal scenario, it also hires and evaluates the performance of those data analysts even though they report to a business unit head. See “The Modern Data Analytics Organization: Federating the Center of Excellence.”) To support the embedded analysts, the corporate team may hold office hours for one-on-one meetings. The corporate team may also run data labs where analysts can work on their code with the help of a corporate specialist. The corporate team may also help organize a community of practice for embedded data analysts to network on a regular basis.

From there, knowledge continues to trickle down in a step-ladder manner. Embedded data analysts build local solutions (i.e., dashboards) for their department and coach data-savvy business users to customize existing reports (i.e., data explorers). In turn, data explorers coach data consumers on how to use their data analytics tools and gain more value. (See Figure 2.)

Figure 2. In a Federated Organization, Support Trickles Down and Requests Trickle Up

Trickle Up. While coaching and support trickle-down, requests trickle up. Data consumers send their questions about the data or requests for new features to data explorers, who address them if possible. In turn, the data explorers relay their questions and requests to data analysts—their local data experts—who may build a quick solution, or prototype a more complex one and show it to specialists on the CDA team. (See figure 2.)

At the same time, business users submit trouble tickets to a data analytics help desk using a form that triages the request. The help desk is designed to handle break-fix issues and small or simple requests. It is staffed by entry-level managers or support specialists. Larger requests or enhancements are forwarded to the corporate prioritization committee via a formal submission process. (See below.)

Project Management 

A data analytics prioritization committee is the focal point for handling larger requests that exceed a certain threshold, say projects exceeding $100,000 or three months in length. The committee scopes, triages, and ranks requests, and maps them to the available development capacity of the corporate development team. Typically, a project manager works with business leads to triage requests, estimating complexity, duration, and skill requirements. A robust triage process enables the prioritization committee to know exactly how many requests it can fulfill in the quarterly development cycle.

Request Pathways

Project requests may come from departments, the corporate PMO, and the CDA team itself. The Help Desk may also forward requests that are beyond its ken to handle.

Departmental Requests. Within each department, an analytics manager coordinates a team of data analysts. The manager scopes, consolidates, and prioritizes requests from business unit leaders, managers, and staff and assigns them to the appropriate analyst. However, if the request is too large or complex for the local data analyst team, the manager submits the request to the data analytics prioritization committee. (See figure 3.)

 Figure 3. Project Request Pathways and Prioritization Process

Each departmental analyst is aligned with a specific function or team but may also work on requests in other functions, depending on availability, skill level, and domain knowledge. Data analysts are divided into job classification tiers based on their ability, experience, and domain knowledge. Departments will have a different mix of analysts based on their data analytics maturity.

Corporate PMO Requests. The prioritization committee also fields requests from the enterprise PMO and the CDA team. The PMO specifies data analytics work in large enterprise IT projects, such as the implementation of a new enterprise resource planning application or a cloud migration project. This work might entail building event streaming pipelines or moving a data warehouse to a public cloud platform.

CDA Requests. The CDA team submits projects that involve enhancing the data infrastructure, whether adding new sources to the data warehouse, automating data pipelines or building subject-area models to support specific departments or corporate processes. Often these “internal” requests get crowded out by urgent corporate PMO and departmental requests, sacrificing the long-term health of the organization’s data infrastructure. To avoid this problem, CDA organizations may dedicate a team to handle this work.

Resource Allocation

To make this top-down request model work, it’s critical that the CDA team and embedded data analyst teams—allocate a specific percentage of their time to each request pathway. (See tables 2 and 3.)

Table 2. Embedded Analyst Team

This manner of allocating resources requires that each team to accurately estimates its development capacity. Teams that apply DataOps practices know how to do this. (See “Best Practices in DataOps: How to Create Robust Data Pipelines.”) Without an accurate estimate, teams fall behind schedule and create a perpetual backlog.

Developer Allocation. It is also helpful if individual developers—both corporate and embedded—allocate a fixed percentage of their time to various request pathways, depending on their skills and assignments.

For example, at the departmental level, embedded analysts might reserve 60% of their time to work on departmental projects, 10% for help desk tickets, and 10% for corporate data infrastructure projects that affect the department. In addition, to encourage innovation, many companies allow developers—both corporate and embedded—to spend one day a week (20%) working on self-directed projects that will benefit the company and advance their skills.

At the corporate level, developers are more likely to be assigned to teams dedicated to different request pathways, with the option to rotate teams after one or two years. For instance, the CDA might create a dedicated help desk to address small requests, an operational reporting team to handle operational requests, a cross-functional “SWAT” team to build departmental solutions quickly, and an architecture team to extend the data infrastructure.

Table 3. CDA Team

Summary 

Self-service analytics requires a federated, business-driven organization led by strong and enlightened leaders who excel at communicating across departmental and corporate boundaries. Ultimately, it’s an exercise in push-down development that offloads a significant degree of analytics and support work from a corporate team to local data analysts.

But a federated model requires significant coordination, cooperation, and planning among corporate and departmental teams. The CDA team needs a robust team of data and analytics specialists to assist local teams, build enterprise applications and data infrastructure, and maintain a help desk. It also requires strong oversight committees, including a prioritization process for managing top-down project requests.

Read - How to Succeed with Self-Service Analytics, Part V: Tools and Technologies