Edge analytics in the Internet of Things

The Internet of Things (IoT) plays an essential role in the ongoing digitization of today’s world by bridging the gap between physical and virtual entities. IoT can thereby be defined as “a global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies” [1].

Analytics in IoT ­— or the Analytics of Things — became a hot topic recently, as sensors and smart objects increase the data pool for analytics tremendously and enable completely new scenarios like tracking physical objects more precisely than ever. However, IoT also poses many challenges to existing business intelligence and analytics (BIA) environments. Firstly, manifold heterogeneous and distributed data sources in IoT networks need to be integrated. Moreover, data in IoT networks is often unstructured (e.g. image and video data) and sometimes erroneous (e.g. inaccurate sensor data). Lastly, data sets are often very large and fast, e.g. continuous data streams coming from sensors.

Consequently, one major issue is the transfer of large data sets from IoT sources to an analytics platform. The transmission of large volumes of raw data increases latency and is sometimes not even feasible due to poor or underpowered network connections. A solution to these issues is to process and analyze data directly at or near the source ­­– also known as edge analytics.

The idea behind edge analytics is to push analytic functions to source systems and thereby reduce data transmission volumes by transferring only aggregated results to a host application. By way of illustration, modern jet engines generate up to a terabyte of data per flight [2]. Given this fact, it is reasonable to transfer average values or outliers rather than the entire stream of data. In addition, these aggregate values can be processed and stored at the device and transferred at a later time, e.g. when the device has better network connection.

Besides reducing the amount of transferred data, edge analytics also enables new features like anonymize data before it actually leaves its source, also known as tokenization. This addresses privacy concerns and supports analytics scenarios in which external parties can access data without violating privacy rights.

However, one major problem is that the light-weight environment of many edge devices is usually not designed to store large data sets and much less to conduct analytics on these large data sets. This is why many IoT components use IoT gateways to process their data and transfer it to the cloud [3]. Moreover, the distributed character of edge analytics also increase complexity in BIA environments and poses new challenges to data integration.

Architectural approach to edge analytics

Though the basic concept of edge analytics is very straight-forward and doesn’t seem that complicated, no reference architectures and standards have been established yet. Thus, this section introduces a way of looking at edge analytics based that is part of our research at the University of Stuttgart [4].

Obviously, IoT objects that are capable of edge analytics need some kind of component that provides data management, data transformation and query capabilities – or as we call it, an analytical atom. Consequently, the goal of an analytical atom is to turn an IoT data source into some kind of self-contained mini data warehouse (DW).

As Figure 1 illustrates, analytical atoms can be placed at various locations. Each IoT object or even each sensor can have its own atom. For instance, an engine, a car or a plane can constitute one or more analytical atoms that stores and processes certain parts of their data and transfers results to an overarching (cloud) platform. Furthermore, additional components can be placed at an intermediate level (e.g. at a cell tower [5]) to combine various atoms and aggregate their results before forwarding them, similar to current IoT gateways. To stick with chemical metaphors, we name these components in our approach analytical molecules.

Figure 1. An edge analytics architecture based on analytical atoms

More interesting than the high-level perspective is the microarchitecture within an analytical atom (c.f. Figure 2). According to the idea of a mini DW, it seems natural that an atom also comprises typical DW layers:

• A data management layer that stores raw data coming from the source system,
• an analytical layer that transforms and processes data,
• and an access/presentation layer that takes care of tasks like role management and other preceding tasks.

Moreover, an overarching metadata management system is required to track data about source systems, data provenance, transformations and many other additional information. And certainly a low-level interface to connect to the source system and a high-level interface to communicate with subordinate systems, other atoms or analytical molecules is essential. Despite the low- and high-level interfaces the depicted components are optional and their concrete arrangement can vary depending on the use case and the surrounding environment. For instance, it is possible that the data management is partly cloud-based and only a simple serving database is hold locally to answer latency critical requests.

Figure 2. Microarchitecture of an analytical atom

With this architectural approach, an analytical atom can constitute a broad range of functionality from rather simple use cases where for instance only average values are calculated and stored, up to complex structures similar to a full-fledged data warehouse. For example, a sensor within a machine can have a light-weight analytical atom attached that simply screens a data stream and stores outliers in a very simple key-value store, whereas a car for instance can have a more sophisticated analytical atom embedded that saves all incoming data in an local big data store and allows advanced time series analysis on it.

Summary

This article illustrated the underlying idea of edge analytics and briefly introduced analytical atoms as an architectural approach to it. The idea of analytical atoms is part of an ongoing research at the University of Stuttgart and is not fully matured yet. Currently, we refine the microarchitecture with a prototype and evaluate concepts dealing with the question how analytical atoms and molecules can be integrated to enable new analytics scenarios within the IoT. The overarching vision that we pursue, is an environment with standardized analytical atoms that can be combined ad-hoc to support arbitrary analytics scenarios. However, there is still a long way to go.

Further reading

[1] Internet of Things Global Standards Initiative
http://www.itu.int/en/ITU-T/gsi/iot/

[2] Palmer, D.: The future is here today: How GE is using the Internet of Things, big data and robotics to power its business
http://www.computing.co.uk/ctg/feature/2399216/the-future-is-here-today-how-ge-is-using-the-internet-of-things-big-data-and-robotics-to-power-its-business

[3] Treadway, J.: Using an IoT gateway to connect the "Things" to the cloud http://internetofthingsagenda.techtarget.com/feature/Using-an-IoT-gateway-to-connect-the-Things-to-the-cloud

[4] Baars, H., Ereth, J.: From Data Warehouses to Analytical Atoms – The Internet of Things as a Centrifugal Force in Business Intelligence and Analytics https://www.researchgate.net/publication/301298501_From_Data_Warehouses_to_Analytical_Atoms_-_The_Internet_of_Things_as_a_Centrifugal_Force_in_Business_Intelligence_and_Analytics

[5] Satyanarayanan, M. et al.: Edge Analytics in the Internet of Things
 https://www.cs.cmu.edu/~satya/docdir/satya-edge2015.pdf