Streaming analytics, as the name suggests, is the study of data “in flight,” meaning programmatic calculations that are performed on data as it is created (typically while it resides in memory) and before it is stored. Organizations across industries analyze data in flight to increase revenue, reduce cost and control risk. They apply streaming analytics to data from many types of events – credit card swipes, website clicks, the turn of machinery gears – anything that emits a data signal.

Streaming in Real-Time Analytics: Analytics architectures that are built for specific use cases that require real-time data processing. Streaming for real-time analytics mostly reflects the streaming pipeline displayed in Figure 6. Such pipelines usually comprise an edge analytics component that pre-processes data near the source and then sends data via stream transportation (i.e., messaging) to downstream analytics systems that conduct complex event processing or other stream analytics to extract insights. Real-time analytics can mostly be found in scenarios with low-latency requirements and where immediate insights are key. Common use cases are the following: Data Exploration and Data Governance.

Streaming analytics, in contrast, is performed on data in motion. As soon as a source system generates an event, that event is “streamed” into one or more downstream systems. The analysis, in this case, is done programmatically on the fly as the events stream by. The streaming analytics program evaluates individual events or collections of events, looking for noteworthy anomalies, patterns, or correlations. When it finds something of interest, it triggers an alert or updates a dashboard, or takes some other prescribed action. With streaming analytics, insights and actions happen in real time—within minutes, seconds, or even milliseconds. Figure 2 provides a diagram of streaming analytics. Streaming analytics often assesses and compares collections of events called “windows.” Analysts and data scientists configure event windows by various methods. Tumbling windows, for example, collect fixed numbers or time periods of events sequentially in a stream. Sliding windows do the same, except that they overlap, while session windows collect events that are divided by pauses in the data stream. Figure 3 provides examples of event windows. Streaming analysis might include Artificial Intelligence (AI), which refers to logical tasks such as natural language processing that normally require human cognition. A common component of AI is machine learning (ML) software, which teaches itself data patterns after being “trained” to do so on historical data. Data teams can apply ML models to streaming data, then adjust them based on results, either periodically or automatically on an ongoing basis.