Apache Spark aims to solve the problem of working with large scale distributed data—and with access to over 500 million leaked passwords we have a lot of data to dig through.
Advancements in the API make running Spark with Scala, Python, or even SQL smoother and faster than ever. This talk will introduce you to Spark and the new way to run queries on structured, distributed data by looking at breached credentials. We'll walk through how to get started with Spark and discuss the tradeoffs for using different abstractions provided by the framework. With the help of live code, we'll find patterns in the password data and look at how you can encourage your users to be more secure. You will see how easy and fast it is to both explore and process data using Spark SQL and leave with the tools to get started with your own distributed data...and a password manager.
Kelley has worked in a variety of engineering roles, ranging from trading live cattle derivatives to building production data pipelines in Scala. She spends a lot of time thinking about how to make technical concepts accessible to new audiences. In her spare time, Kelley spends a lot of time cooking and greatly enjoys reorganizing her tiny kitchen to accommodate completely necessary small appliance purchases.