OpenStack Mitaka Design Summit

Monitoring and log management are two closely related siblings, necessary to guarantee the stability and availability of an OpenStack system. Monasca is a scalable, multi-tenant, high performance, fault-tolerant OpenStack Monitoring as a Service (MONaaS) solution described at https://wiki.openstack.org/wiki/Monasca. In this session we present our work on integrating additional log management functionality into Monasca, to bring the two siblings together in one tool - tailored for OpenStack'ers.

The technical basis of our work consists of Elasticsearch, Logstash and Kibana, aka ELK. The ELK stack has probably become the most widely used open source tool for centralized log management. It also finds frequent adoption among OpenStack users, as demonstrated in various sessions at previous summits. The goal of our work in the Monasca project is to:

1. give operators a unified turn-key solution to manage all logs from their multiple OpenStack systems;
2. provide OpenStack tenants a service that allows for managing logs from their applications running on OpenStack - accessible from Horizon.

As part of this talk we will dive into our architecture to show how to meet our primary design goals: scalability, multi-tenancy, resilience and extensibility. One important aspect is our integration with Monasca that uses common components, such as Kafka, and common architectural patterns. Another highlight that we will point out is our newly implemented REST API, that facilitates Logging as a Service in the spirit of Loggly. Here, authentication happens via Keystone, which extends the existing ELK stack by adding multi-tenancy, applying the tenant model of OpenStack.

Finally, we will give a demo of our solution.

Keywords: monasca, elk stack, elasticsearch, fujitsu, hp, logging, log management, monitoring