Our experts in data analytics, data management, and data science took a few moments last week to discuss the top six trends that will dominate the headlines in 2018. Our predictions range from the emergence of report catalogs and converged data analytics platforms to self-driving data, data science appliances, and the first fines issued for GDPR violations. Experts categorized each of their predictions as “incubating”, “emerging”, or “going mainstream.”

Here are our top six predictions:

• Self-Driving Data Emerges 
• Report Catalogs Debut 
• GDPR Claims Its First Victims
• Universal Semantic Layer for Data and Analytics Appears 
• Data Engineering Surges
• Data Science Teams Dominate 

In a field as dynamic as data analytics, it was hard for our experts to settle on just 10 key trends. So, we also provided a bonus trends to whet your appetite for new year.

Bonus Predictions 

• Analysts Get Classified and Certified
• Analytics and Data Platforms Converge
• The Power User Triple Play 
• Virtual Assistants Infiltrate Business Operations
• CIO’s Lose Budget Dollars 
• CEO’s Go Mobile and Get Data Savvy 
• Data Architecture Heats Up
• Data Science Becomes Purpose Built

All together, 2018 looks like another action-packed year, full of new technologies, techniques, and trends. Enjoy!

Top Six Predictions for 2018

1. Self-Driving Data is Explored (incubating)

A few forward-thinking vendors begin to explore the concept of self-driving data. Similar to a self-driving car that knows its destination and can navigate a course and avoid collisions, self-driving data knows its destination, can navigate data pipelines, and deliver data wherever it’s needed. We have the technology to make this happen — machine learning, artificial intelligence, data and process encapsulation, service-oriented architecture, and micro-services. They need to be applied by pipeline management tools to build smart data ecosystems. (Dave Wells)

2. The Report Catalog Debuts (Incubating)

If 2017 was the year of the data catalog, 2018 is the year of the report catalog. Think about it. Organizations have hundreds, if not thousands of reports, generated by multiple analytical tools, making it nearly impossible for business people to find the right report. A report catalog will enable users to search, profile, view, rate, compare, and annotate reports from any system in real time. In addition, they can bookmark their favorite reports to create a personalized information workflow. BI managers might be the biggest beneficiaries of report catalogs: they can monitor report usage, audit changes, compare differences, and clean out unused reports. (Wayne Eckerson)

3. GDPR Claims Its First Victims (emerging)

The General Data Protection Regulation (GDPR) regulates the processing of European citizens’ personal data. It applies to organizations both inside and outside the European Union (EU). The regulation goes into effect on May 25, 2018, and many organizations are ill-prepared to meet the deadline. In 2018, expect to see penalties for non-compliance, especially among U.S. companies which are generally less aware than their European counterparts about the complexity and gravity of the new regulations. Violations carry fines of up to $20 million or four percent of annual turnover. (Henry Eckerson)

4. Universal Semantic Layer for Data and Analytics Appears (incubating)

In 2018, expect at least one data analytics vendor to open source its middleware (i.e. services layer) and encourage competitors and others to plug into it. The data analytics middleware will have a robust SQL and programmatic interface (API) that serves as a universal semantic layer for both analytics capabilities on the front-end and data assembly, integration, and preparation functions on the back-end. Customers have long wanted vendor-agnostic middleware for data analytics, and the time is right for one or more courageous vendors to step forward and open up the crown jewels. (Wayne Eckerson)

 5. Data Engineering Surges (emerging)

Data Engineering becomes the new in-demand skill as companies seek top talent to design and build data pipelines and data services. Data engineering will be recognized as a multi-disciplinary skill that spans database architecture, microservices architecture, data modeling, programming, Hadoop technologies, NoSQL databases, and data stream processing. (Dave Wells)

6. Data Science Teams Dominate (emerging)

The two scariest numbers for a data scientist? 80 and 50. 80% of a data scientist’s time is spent preparing data and 50% of all predictive models are never used by business. The solution to these problems is to use multi-disciplinary agile teams to replace lone-wolf data scientists. The teams include data engineers, data stewards, data scientists, programmers, business analysts, optimization experts and business end users. Data scientists will be just one part of this larger team. This approach will eliminate the need to find and recruit superstar data scientists. (Steve Smith)

BONUS PREDICTIONS

1. Analysts Get Classified and Certified

What’s the most abused title in large organizations? Analyst. Does anybody know what an analyst is or does? Many analysts are glorified report developers or requirements gatherers or systems administrators. In the world of data analytics, the title “analyst” should mean something. In 2018, Eckerson Group will launch a formal classification of analyst roles, qualifications, and responsibilities. We are seeking input now and help creating a body of knowledge and certification tests. Let us know if you are interested in contributing! (Wayne Eckerson)

2. Analytic and Data Platforms Converge (Emerging)

In 2017 saw the emergence of modern analytics platforms that integrate the full slate of business intelligence functionality and data platforms that integrate the complete set of data integration capabilities. These platforms provide a common set of services (i.e. metadata, security, APIs, collaboration, and semantics) across all functional capabilities. In 2018, we’ll see these two platforms converge into a single data analytics platform running a single set of unified services. Indeed, some BI tools (Qlik, SiSense, Dundas) already offer full-stack data analytics platforms while larger software vendors have all the components. The 2018 version of data analytics platforms will be more scalable, open, and enterprise centric. And this will lead to the next trend. (Wayne Eckerson)

3. The Power User Triple Play (Emerging)

There is a new self-service platform emerging for power users. The platform integrates a data catalog, data preparation tool, and data visualization tool. Some platforms will come with their own proprietary modules, while others will provide robust interfaces to pure play products in each area, or a combination of open and proprietary products. Already, many BI tools offer visual discovery and data preparation functionality, with cataloging on its way.  (Wayne Eckerson)

4. Virtual Assistants Infiltrate Business Operations- Emerging

Expect to see an increasing number virtual assistants that augment jobs, enabling employees to do higher value tasks or take on more work. Chatbots and virtual assistants to help customers with simple tasks is standard these days, but virtual assistants that help employees do their jobs are less common. Here are a few current examples: Conversica offers a virtual sales assistant to qualify leads, x.ai offers 'Amy', takes over email conversations to schedule meetings, and Zirtual offers virtual assistants for a number of tasks, including research, tracking expenses, and managing social media. As natural language generation (NLG) and natural language processing (NLP) mature, more virtual assistants with increasingly vocation-specific skill sets will emerge. (Henry Eckerson) 

5. CIO’s Lose Budget Dollars (mainstream)

CIOs might see their budget cut in the next few years, not because there is less technology, but because the business will buy it themselves. Marketing, sales, finance, and other departments will deploy technology, tools, and applications, such as cloud solutions, that IT previously implemented. CIOs can halt the outflow by partnering with the business to solve problems and anticipating business needs. CIOs should let the business steer the discussion around technology and not become (or continue to be) a roadblock to progress. (Dewayne Washington)

6. CEO’s Go Mobile and Get Data Savvy (mainstream)

Gone are the days when secretaries print out email messages for top executives. Get ready for tech savvy CEOs who want analytics on their smart phones and tablets. CEOs will no longer call for insight and information because they can access it themselves on a mobile device no matter where they are. CEOs will drive analytic adoption by calling their direct reports about the data in their mobile dashboards. (Dewayne Washington)

7. Data Architecture Heats Up (going mainstream)

With continuing evolution of data management and confusion about data topology and data flow, data architecture becomes a critical skill. Data architects are in demand but the role of architect shifts, giving less attention to modeling and schema and much more attention to topology of data stores, data flow standards and constraints, data management frameworks, and data pipeline templates and conventions. Data architecture is less concerned with how data is stored and more concerned with how it is moved and how it is used. (Dave Wells)

8. Data Science Becomes Purpose Built (emerging)

Data science is complicated, but so are smart phones and we all know how to use them. The reason smart phones are easy to use is because their complexity is packaged as an appliance with limited but highly focused functionality. Purpose-built data science applications will be similarly successful. These might include predicting churn in the mobile industry, detecting fraud in retail industry, performing location selection for fast-food restaurants. The alternate, “Swiss army knife”, approach of data science workbenches will still be absolutely needed for the really hard and really new problems. And superstar data scientists will still grab the headlines for breakthroughs. But the bulk of the ROI from data science will accrue to firms that deploying purpose-built data science applications. (Steve Smith)