Eckerson Group Predictions for 2020 in Review

Here at Eckerson Group, we like to mark the end of one year by making predictions about the next. Little did we realize, as we sat down to this task in late 2019, just how unpredictable 2020 was going to be. Yet, despite the unexpected nature of this past year, it’s worth taking a look back to see how we did.

Last year, we made nine predictions, sourced from across our team of consultants and analysts. You can check out the full article here. Below you can see how our calls shook out.

Prediction #1: Data Exchanges Are the Next Big Thing

Data Exchanges are here! In March, as the pandemic spread and most of the US and Europe went into lockdown mode, Dawex, one of the leading data exchange vendors, made headlines by creating a pro bono Covid-19 data exchange to facilitate data sharing between hospitals, government agencies, private companies, and research institutions. In another major stride for data exchanges, the European public sector, famously cautious about data privacy, adopted a secure data exchange powered by Equinix. Throughout the year, major organizations like S3 Partners and Dow Jones made their data available on exchanges, and, this fall, the data exchange companies Harbr and InfoSum raised $38.5 million and $15.1 million respectively in Series A investment rounds. In October, Wayne Eckerson even published a report exploring the growing trend: "The Rise of Data Exchanges: Frictionless Integration of Third-Party Data".

Prediction #2: 2020 is the Year of the Graph

A lot of progress was made in the graph technology sector, but it didn’t quite achieve the widespread adoption we had called for. One big step was the March announcement by industry leader Neo4J of a business intelligence (BI) connector for their graph database. Being able to use a graph database as a BI data source goes a long way in lowering the bar for using graph in an enterprise setting. Similarly, the fall launch of GraphUI, a visual interface that layers on several of the foremost graph databases, shows that graph technology is rapidly becoming more accessible. According to a Dataversity survey, however, graph usage in 2020 lingered at around 15% of the organizations polled, approximately the same as in 2019. Nevertheless, nearly 20% said they were planning to adopt graph in the next two years.

Prediction #3: The U.S. Government Will Craft Federal   Data Privacy Regulations 

Although the Consumer Online Privacy Rights Act looked poised to start the ball rolling on federal data privacy regulations in late 2019, it never even made it out of committee. At the state level, more bills were introduced to address consumer data privacy than in any previous year, but very few passed as the coronavirus pandemic shifted legislative priorities.   


Prediction #4: The Hottest Job in 2020? Ontologist 

Last year, we pointed out that many data-driven companies like Amazon, Google, Facebook, and Nike were starting to post jobs looking for ontologists with skills in semantic technologies. This trend did not really expand beyond these select few companies in 2020, however. Some of the role’s loss of momentum can likely be explained by the Covid-19 induced hiring freezes and limited budgets for creating new positions.


Prediction #5: Data Warehousing Becomes a Strategic Service

The trend of Data Warehousing as a Service (DWaaS) continued in 2020. Most true DWaaS outfits focus on a single industry with vendors in the real estate, pharmacy, manufacturing, and financial industries among others. These companies provide functioning data warehouses straight out of the box. At the same time, more general solutions like Snowflake, Domo, and Infor Birst, come with industry models and canned dashboards to deliver an experience that is increasingly close to a DWaaS. Among these, Snowflake’s massive IPO in September showed investors’ continued high expectations for this model, especially as more companies move to the cloud.

Prediction #6 - Lean, Agile and Product Management Approaches Go Mainstream

Lean and agile approaches to data product development, especially DataOps, gained momentum this year. Thanks to savvy content marketing by vendors, the term “DataOps” is now fully in the zeitgeist and more companies, particularly at the Fortune 2000 level are turning to the methodology to speed development times while improving data quality. On the vendor side, the last year has seen a proliferation in the number of companies offering DataOps platforms that lower the bar for organizations to implement the approach. In fact, we recently published a report looking at four leading approaches to that space.

Prediction #7 - Data Vault Reaches a Tipping Point

In our experience, a plurality of new data warehouse modeling projects seem to be using Data Vault modeling techniques. We have seen data architects getting a lot of pressure from upper management to move to the Data Vault methodology and data warehouse design evangelists at vendors pushing Data Vault to clients. It certainly has gone mainstream and may yet prove to be as significant an advancement in modeling as dimensional modeling 20 years ago.

Prediction #8 - Streaming Databases Will Become   Popular 

Modern data applications increasingly rely on streaming technologies like Kafka to enable real-time insights, a trend that only grew during 2020. Players like Confluent with their ksqlDB are now making queryable data streams that can act, in some respects, like a database. This normalization and integration of streaming into the rest of the data lifecycle is probably best exemplified by TIBCO’s September announcement of “Hyperconverged Analytics,” which integrates its streaming platform with its data science and visual analytics products. For more on streaming and real-time analytics, read Kevin Petrie’s ebook, out December 15th, Continuous Intelligence: Controlling Operations with Real-Time Context and Analytics.

Prediction #9 - Time-Series Analytics Automates Operational Insights

Time-series analytics are paving the way for a new approach to tracking business metrics. Now, instead of simply analyzing the snapshots provided by dashboards, companies are relying on business monitoring tools that use AI to raise alerts about anomalous behavior, find correlations between metrics, and evaluate the root causes of irregularities. This shift in approach is reflected in the focus of traditional BI vendors including ThoughtSpot, Yellowfin, and Qlik, all of which now incorporate business monitoring capabilities into their platforms. At the same time, dedicated start-ups such as Sisu, Anodot, and Outlier have built products focused exclusively on those functionalities. Wayne Eckerson addressed this topic further in his June report "Business Monitoring Systems: Using Machine Learning to Analyze Business Metrics".

In year when it sometimes feels like nothing went as initially planned, I think we can be pretty satisfied that 6.5 of our predictions came to fruition. This has, in spite of everything, been a remarkable year in the data analytics space and the progress made by practitioners and vendors has been astounding. To see where Eckerson Group thinks trends are headed for 2021, check out our new slate of predictions, here

Joe Hilleary

Joe Hilleary is a writer, researcher, and data enthusiast. He believes that we are living through a pivotal moment in the evolution of data technology and is dedicated to...

More About Joe Hilleary