Back To Schedule
Wednesday, October 28 • 3:40pm - 4:20pm
This is Sparkhara: OpenStack Log Processing in Real-time Using Spark on Sahara

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

OpenStack service controllers produce large amounts of log data, and processing these logs can be a time consuming and difficult task. Data processing applications excel at automating these types of tasks and providing new windows into the underlying trends in the data produced.

We will discuss how to configure your stack to produce log data that can be consumed and analyzed in real-time by Spark applications running on the OpenStack Data processing service (sahara). These techniques can be used to inspect the current status and health of your services, decreasing the time spent between problem detection and solution. This also creates an exciting platform for developers to explore new transformations of log data that will unlock endless possibilities such as failure analysis, performance enhancement, and fraudulent activity detection.

This talk will cover a broad range of OpenStack and Apache technologies used in the development of this solution. We will discuss the following as they pertain to the greater problem space:

  • Logging service data to Zaqar and Manila

  • Launching Spark clusters with Sahara

  • Visualizing and experimenting on data with Zeppelin

  • Creating Spark jobs to deploy through Sahara

  • Storing processed data to Trove databases

There are many paths to achieving improved log processing in OpenStack. Attendees should expect to learn about how they can improve the stability and performance of their stacks through the advanced techniques discussed in this session.

avatar for Nikita Konovalov

Nikita Konovalov

Software Engineer, Mirantis
Nikita has been working with OpenStack Data Processing (Sahara) from the early days of the project. He has been implementing the initial version of Sahara UI which then has been accepted to main OpenStack Dashboard codebase. He added support for Sahara benchmarking for Rally project... Read More →
avatar for Michael McCune

Michael McCune

Principal Software Engineer, Red Hat
Michael McCune is a software developer creating open source infrastructure and applications for cloud platforms. He has a passion for problem solving and team building, and a lifelong love of music, food, and culture.
avatar for Chad Roberts

Chad Roberts

Senior Software Engineer, Red Hat
Chad Roberts has been involved with Openstack Sahara since the Icehouse release. In addition to work on Sahara, he has also contributed to the Horizon dashboard.

Wednesday October 28, 2015 3:40pm - 4:20pm JST
Expo Hall (Marketplace Theater)

Attendees (0)