In streaming-data systems, like applications that stream telemetry from industrial assets or IoT, people encounter problems related to flow control, memory management, and poor performance. The Akka Streams API addresses these issues and it naturally models the common patterns found with streaming telemetry, through simple, composable, and high-level constructs. This talk will provide a motivating example for using Akka Streams and demonstrate a number of these common patterns, in support of building reliable and resilient systems for streaming telemetry at scale.
This is an introductory talk and it assumes very little background knowledge. Some experience with Scala, Akka, or a systems programming language is helpful, but is not necessary. Someone new to Scala, or even unfamiliar with Akka, should benefit from this talk. Someone who has intermediate to advanced experience building streaming-data applications should also appreciate this talk, especially if they are unfamiliar with the Akka Streams API.
This talk aims to demonstrate the unique challenges of streaming telemetry, especially when the stream of data is unbounded and not directly controlled by the server, typical in industrial settings, or IoT. These challenges include decomposing messages, batching messages within near real-time constraints, rate-limiting requests, managing concurrency and periodic events, and flow control, among others. These challenges emerge as common patterns and while they are conceptually easy to address, implementations become difficult in traditional multi-threaded, systems-programming languages. Many people turn to Akka and the actor model to address these challenges, however, when building applications for streaming data, the actor model alone is not sufficient, and people encounter the same implementation challenges. This talk aims to show how the Akka Streams API models the common patterns found with streaming data and makes building reliable and resilient applications quite straightforward, without the need to implement a lot of code.
This talk will incorporate elements of these two blog posts of mine:
It will also incorporate some of the introductory material from this talk of mine at Reactive Summit 2017:
Colin Breck has almost two decades of experience in developing time-series infrastructures for the monitoring and control of industrial applications. At Tesla, he works on distributed systems for the monitoring, aggregation, and control of distributed, renewable-energy assets. Previously, he worked on the PI System at OSIsoft, including the time-series database and publish-subscribe infrastructure. He writes monthly at [blog.colinbreck.com](http://blog.colinbreck.com/).