ABSTRACT: Data catalogs are red hot but organizations are struggling to implement them and gain widespread adoption. That’s according to preliminary results from an Eckerson Group survey on data catalogs.

Data catalogs are red hot but organizations are struggling to implement them and gain widespread adoption. That’s according to preliminary results from an Eckerson Group survey on data catalogs. (Make sure your voice is heard—take the 5-minute survey now.) 

Data catalogs sit at the intersection of self-service analytics, data governance, and metadata management. They are a key ingredient of a successful data & analytics program. They help business users discover data, facilitate self-service, and improve data quality and governance. 

The interest in data catalogs is high. Almost all respondents (94%) plan to deploy a catalog: half are either fully deployed (10%) or partially deployed (41%) while 42% are planning to deploy. Only 6% have no plans, citing lack of budget or research to make a commitment. (See figure 1.)

Figure 1. To what degree has your organization implemented a data catalog?

What’s driving the interest in data catalogs? The biggest reason respondents cited was to “support data discovery” (72%) and “improve users’ ability to find and access data” (66%) which are essentially equivalent. These were followed by “improve data curation and governance (68%) and “support our self-service initiative” (60%).

Figure 2. What motivated your organization to purchase a data catalog?

But results of early adopters of data catalogs is uneven. Although almost a third (32%) of respondents said their data catalog has met their expectations to a “very high” or “high” degree, 43% said it only met expectations to a “moderate” degree, while more than a quarter (26%) said it met expectations to a “fair degree” (20%) or “low degree” (6%).

There are several challenges plaguing data catalog implementations. Chief among them are tool complexity, cited by 43% of respondents, lack of user adoption (39%), and lack of integration with other systems and tools (39%), among other things.

Figure 3. What challenges has your organization encountered when deploying a data catalog?

Best Practices

Respondents provided advice to other companies embarking on a data catalog project. One said, “Start small!”. Another said, “Using API-only data access, the catalog should be the gate through which all access goes. Then it must be accurate, inclusive of all data, and users have no choice but to use it.” Similarly, another respondent said, “Core catalog functions (classification, lineage) MUST be integral to data ingestion, engineering and analytics development. We cannot ask engineers to "fill out extra forms" to support the catalog. One either feeds the catalog directly from the engineering artifacts, or builds the engineering directly from catalog artifacts.”

From a technical perspective, one respondent said, “Model critical data so that you have a data blueprint with descriptions and good data names. You should be able to generate a data catalog from the contents of the data model.” Conversely, another touted organizational strategy: “Install a data office to support key-users in using the data catalog and take ownership. Train the trainer.”

Take the Survey

Before our survey closes later this month, please tell us your experience with a data catalog by taking our survey. You will be asked the questions above plus others on tooling. Make your voice heard! 

We will publish a final report on data catalogs, including the full survey results, in mid-December.