OpenStack Mitaka Design Summit

With the ever increasing adoption of Openstack in Enterprises and transforming from Sandbox research projects into viable Product Grade deployments the ask for architecting the deployments properly is fast becoming a necessity. This session would would walk through all the required tasks to be considered by an Openstack Architect while designing an Enterprise Openstack Cloud .

Openstack Design and Architecture :
Process of Designing and Planning a Minimum Viable Platform to start off an Openstack Architecture. A Cloud Discovery Workshop Template used by the Red Hat Cloud Practice will be discussed. This session helps prepare the architecture team and customers to collaborate on the solution architecture. Some topics to be considered for Openstack Architecture Design Solution Architectures based on Cloud Discovery Worksop .

Openstack Base Deployment :
         Control Plane Recommendations .
         Compute Recommendations .
         Networking Recommendations .
         Failure Domain Recommendations .
         Availability Recommendations

Openstack Services - Deploying and Extending OpenStack Core Services:
 The core components will be covered in addition to tech preview components. . Horizon . Keystone . Nova . Neutron . Cinder . AMQP . Glance . Heat . Swift . Ceilometer . Sahara . Trove . Tempest . Rally

Operating and Managing Openstack:
We will dive into how to operate and manage an OpenStack cloud. . Ceilometer . Nagios or other Montoring Tools . Database Patching . OS Patching . Maintenance Tasks . Hypervisor Maintenance . Controller Maintenance

"Rule The Stack" is a community competition held at the Summit that challenges OpenStack wizards to build a complete OpenStack infrastructure with compute, storage, and networking. Participants compete on speed, accuracy and sophistication of their deployment.

In this session, the winners of the previous 3 Rule the Stack competitions will reveal the tips and tricks they used to reliably provision OpenStack in just minutes. Learn how these techniques extend beyond the competition and can be adapted to be used in your own data center.  If you are interested in trying your hand in this year's competition, we will provide a USB key for you to give it a shot.

Think you have what it takes to "Rule the Stack"?  Join us to learn from past champions and discuss new tricks!

We had presented our cloud design at OpenStack Paris Summit (http://bit.ly/1DbJPUO) and started the operation after the conference. In this talk, we are going to share some important lessons and processes learned after the one year of OpenStack operation. This talk will help people who just want to start OpenStack operation or think of an operation by a small number of people.

1.   Team Building
It is essential to organize DevOps team to keep up with an active OpenStack development. We created DevOps team from scratch. We share the process of the team building and each member's skill doing DevOps.

2.   Monitoring System
Monitoring is important to keep the system stable. We share items we are currently monitoring (about 60,000 items) and show some important items to prevent service disruption. Alos, we share some custom scripts for OpenStack health check (e.g. RabbitMQ, MySQL and OpenStack services).

3.   Log Analytics
Logs (e.g. OpenStack debug log, Syslog, Auth log and Operation log) give you very important information and we can find potential problems/risks by analyzing those logs. We are getting more than 40GB logs a day and it is difficult to find important information among them. We demonstrate our Elasticsearch based log analytics/visualization tool to sort out useful information.

4.   Continuous Integration
Once you start a cloud service, it is difficult to stop the service though there are many necessary updates. We have updated the environment more than 100 times without downtime. We demonstrate Neutron Agent update that is one of the most difficult part of current OpenStack. We also share CI/CD tools and own tools used for system validation after updates.

5.   Daily Operation
We share our daily works.

1. Tools help you to monitor the system efficiently 


2. Tools help you to check security alert 


3. Issue tracking and management 


4. Tools and procedures used for emergency operation (remote operation tools)

Thanks to the community, it becomes easier to deploy OpenStack by many tools(e.g. Juju, RDO and Fuel); however, there is still less information about keep running/updating OpenStack without downtime. We are going to share our experiences and own tools developed through the private cloud operation. Also, we share future challenges to make OpenStack operation more efficient. Today, there are still some manual operations but our goal is to help OpenStack operators sleep better by automating most of the operations.

Your OpenStack cloud has been deployed on a Long Term Service release such as Ubuntu 14.04 using the Icehouse release. Now that Icehouse is out of support from core OpenStack developers and moved into the hands of the Distros its time to start planning the move from Icehouse to Kilo, Juno, Liberty or beyond. Upgrading a live cloud is no trivial task and this talk aims to walk you through the dos, don'ts, and how tos for planning and implementing your cloud upgrade.

Automated deployment has become an important differentiator for OpenStack based products. This presentation highlights similarities and differences, across several OpenStack vendors.

To automate deployment is an absolute necessity in running an OpenStack environment, but which approach should you choose? Some vendors want you to deploy with Puppet, others with Chef, still others with Juju. Some major public OpenStack providers manage their infrastructure with Ansible or Ironic. In this presentation, you'll learn about a variety of automated deployment approaches for OpenStack and their pros and cons.

We will be covering the RHEL OSP Director (TripleO), SUSE Cloud (Chef/Crowbar), Ubuntu OpenStack (Juju), and Rackspace (Ansible).

High availability of the cloud and applications is a requirement for enterprises. There are three fundamental levels at which “high availability” (HA) can be provided in an Openstack-based cloud:

• Controller and core-services level


• Virtual machine (VM) level


• Application level

In this talk, join engineers from ZeroStack Inc. as we take a deep-dive into clustering, high availability of services and VMs. Based on our experience with building and deploying a highly-reliable, scale on demand Openstack based private cloud solution, we provide a case study with design insights, lessons learned and best practices.

In the first part of this talk, we describe and demonstrate how various services that constitute an Openstack controller can be created and managed in a highly available way to provide Reliable Openstack:

• Stateless services: Nova, Neutron, Cinder, Glance, Keystone, Heat


• Stateful services: MySQL, Message queues

In the second part of the talk, we switch gears to talk about high-availability at the VM-level. In the basic case, we discuss how Openstack notifications can be used to detect  faults of compute instances in order to restart them. In the more advanced context of a remote controller, connectivity cannot be used as a proxy for liveness - we discuss how VM-level HA can be still be achieved to run applications reliably.

Finally, in the third part of our talk, we explain how high availability may be achieved at the application level in Openstack-based clouds. Modern applications are increasingly designed in a distributed manner to survive failures of individual nodes. In addition, we explain how concepts of “affinity” and “anti-affinity” can be used along with the notion of host-aggregates to provide tolerance to failures.

Designing a Private OpenStack cloud is complex. It can take months of design and test effort to optimize the hardware and software components. What hardware configuration works best for your management/control plane? What is the “best” hardware configuration for a given workload? What workloads will you initially run (e.g., low latency, compute intensive, high transaction, etc.)? Do you know what workloads will be running in your cloud three months after deployment?

This presentation will cover best practices gleaned across server and services teams as well as the HP Helion Public Cloud in designing and running hardware stacks in OpenStack clouds.  It will explore the best practices and set-ups for control planes optimized for the different OpenStack services, compute configurations that factor in key workload considerations, and the storage and networking architectures that make a difference in how your cloud will handle data when using object and block storage.

Provisioning servers has long been service providers’ bread and butter, but in recent years customers’ demand for higher-performing bare-metal instances has increased dramatically, along with their interest in integrating bare-metal infrastructure with virtualized public clouds. This shift in the types of infrastructure customers are demanding is motivating service providers to create more efficient ways of both provisioning bare-metal infrastructure and integrating it into their customers’ overall infrastructure mix, and OpenStack has recently emerged as a key platform in making both of these developments possible. 

Focusing on the creation of a hybridized, OpenStack-based bare-metal and virtual cloud infrastructure, this technical talk will look at the challenges encountered as well as the solutions developed to successfully deliver bare-metal instances with Ironic and hybrid bare-metal and virtual infrastructure. Specific topics that will be covered include:

• Networking model and automation


• Deployment of cells


• Integration with 3rd party software


• Security

In this presentation, we will tell the stories of 3 top-tier telcos' journeys to OpenStack. We will discuss the opportunities (private cloud, public cloud, NFV), and challenges each telco faced, lessons learned, and current status of their OpenStack project.

Cloud solutions offer vastly improved agility and lower TCO over legacy datacenter solutions. But it can very challenging to move monolithic financial applications from traditional IT infrastructure to cloud-based environments, and to recreate them as distributed, service-oriented, “cloud native” applications with open source tools, new approaches to network communications, high availability, integrity, security, and auditability.


 


ShenZhen Securities Clearing Corporation .Ltd (SSCC) is building an IaaS, PaaS, and SaaS platform for their financial customers. They selected OpenStack as a go-forward IaaS solution, but then faced challenges in building a financial platform based on fully open source software. Mirantis worked with SSCC to fill the gap between OpenStack’s capabilities and feature-set, and their requirements for application management and delivery, and cloud application security.


 


In this session, we’ll share our experience in taking advantage of OpenStack’s flexibility


to design a scalable, reliable financial industry cloud for ShenZhen Securities Clearing Corporation, Ltd. We’ll discuss using OpenStack to provide:


 

• High application availability and reliability in a distributed application architecture with no critical single points of failure





• Robust data integrity and security with new storage architectures and multi-site backup protocols





• New approaches to network architecture and management, access and security, and communications made possible with open SDN

 


Attendees at this session will leave with a much-clearer grasp of financial IT requirements, legacy application debt, and will have in hand a clear, tested pathway to forward migration of critical financial apps onto OpenStack.


 

OpenStack is getting big in the enterprise, which is traditionally very Microsoft centeric. This session will show you everything you need to know about Windows in OpenStack!

To begin with we will show how to provision Windows images for OpenStack, including Windows Server 2012 R2, Windows 7, 8.1 and the brand new Windows Server 2016 Nano Server for KVM, Hyper-V and ESXi Nova hosts.

Next, we will show how to deploy Windows workloads with Active Directory, SQL Server, SharePoint, Exchange using Heat templates, Juju, Puppet and more.

Last but not least, we'll talk about Active Directory integration in Keystone, Hyper-V deployment and Windows bare metal support in Ironic and MaaS. 

The session will give you a comprehensive view on how well OpenStack and Windows can be integrated, along with a great interoperability story with Linux workloads.

OpenVSwitch recently added Open Virtual Networking (OVN) to OVS. We'll cover that and other recent changes to OpenVSwitch. Emphasis on Kilo and Liberty and changes in OVS and OVN.

This talk will cover the pros and cons of four different OpenStack deployment mechanisms. Puppet, Chef, Ansible, and Salt for OpenStack all claim to make it much easier to configure and maintain hundreds of OpenStack deployment resources. With the advent of large-scale, highly available OpenStack deployments spread across multiple global regions, the choice of which deployment methodology to use has become more and more relevant. 

Beyond the initial day-one deployment, when it comes to the day-two and beyond questions of updating and upgrading existing OpenStack deployments, it becomes all the more important choose the right tool. 

Come join the Bluebox and IBM team to discuss the pros and cons of these approaches. We look at each of these four tools in depth, explore their design and function, and determine which scores higher than others to address your particular deployment needs. 

Visualizing OpenStack data is one of the most powerful ways to make sense of what's going on with cloud-based resources. The OpenStack Horizon dashboard provides a number of built in visualizations, but how do we move beyond the features that are provided by Horizon to develop deeper or richer visualizations?

This presentation is based on our experience of developing the open source project Goldstone, and will discuss some of the challenges of making sense of OpenStack data, as well as demo some of the capabilities of D3.js.

One of the solutions our team has come up with for moving beyond Horizon was to develop software with an api-driven architecture that uncouples the data visualizations and the underlying data. Our client is built upon Backbone, a popular JavaScript framework, that sends ajax requests to the server component which is installed alongside an OpenStack cloud. One of our main challenges has been determining what data to gather, and how to make it relevant to an OpenStack operator. In addition to gathering metrics, we’ve developed a resource graph with the goal of finding the upstream source of problems in your cloud.

The software client we’re developing is built entirely in JavaScript, taking advantage of the Node.js JavaScript developer ecosystem tools such as Grunt, Karma, and Sass. And for the visualizations, we use D3.js, a low-level, highly customizable library for creating flexible, data-driven visualizations.

This presentation will include a walk-through of our learnings from developing Goldstone, a summary of some of the challenges of visualizing OpenStack our team has overcome, as well as a demo of some of the capabilities of D3.js.

Zaqar is the messaging and notifications service for OpenStack that allow users to build scalable, reliable and high-performing applications. It has a REST API that can be used by developers to transport messages between different components in their cloud applications through different communication patterns, such as producer/consumer or publisher/subscriber.

In Zaqar's v1, only a WSGI controller is available as a transport solution. This controller is enough for most of the use cases required by users like, for example, task distribution, broadcasting and point to point messaging.

But when the storage and recovery of messages is incremented to rates over 10.000 messages per second -- in cases like activity tracking in a website or log aggregation -- this transport is very inefficient.

To accomplish an adequate performance for this type of loads is necessary the implementation of a streaming interface. That is, long living connections with minimum overhead in which the messages can be freely pushed.

The websocket driver, the latest addition to the Zaqar project, aims to cover this use cases.

Attendees will gain insight into Zaqar's architecture before and after the API refactoring and the new driver addition, how we implemented the driver,  information about Websocket and why we choose it as a transport solution, along with some uses cases and benchmarks.

Getting your cloud images right is critical to the success of any cloud deployment. Getting all of the basics in place from security settings to patch levels will greatly streamline your use cases down the line. Doing this by hand can be complicated and filled with potential missteps.  The key to mitigating this is automating the process with a tool that is both reproducible and testable.

Enter Packer!

Packer is a tool developed by HashiCorp (the people behind Vagrant) to help you create identical cloud images for a variety of different environments. It also allows you to create image templates that are easy to version control and understand what happens during the image creation process.

In this talk, we will show you how to get up and running with Packer and get your images right from the start. We will cover the following areas:

• Packer Installation -  We will show you how to get Packer up and running quickly


• Preparing OpenStack - We will review what is needed on your OpenStack cloud to get started.  


• Building your first Packer Template - We will create and review the options in a Packer Template


• Testing - No code is complete without it.


• Expanding your Template with provisioners - We will review the options for iteration, once your base template is in place.

Packer has many options and will work with OpenStack and many other cloud solutions, both public and private. Join us to see how this tool really takes a lot of the pain out of image building images.

Monitoring and log management are two closely related siblings, necessary to guarantee the stability and availability of an OpenStack system. Monasca is a scalable, multi-tenant, high performance, fault-tolerant OpenStack Monitoring as a Service (MONaaS) solution described at https://wiki.openstack.org/wiki/Monasca. In this session we present our work on integrating additional log management functionality into Monasca, to bring the two siblings together in one tool - tailored for OpenStack'ers.

The technical basis of our work consists of Elasticsearch, Logstash and Kibana, aka ELK. The ELK stack has probably become the most widely used open source tool for centralized log management. It also finds frequent adoption among OpenStack users, as demonstrated in various sessions at previous summits. The goal of our work in the Monasca project is to:

1. give operators a unified turn-key solution to manage all logs from their multiple OpenStack systems;
2. provide OpenStack tenants a service that allows for managing logs from their applications running on OpenStack - accessible from Horizon.

As part of this talk we will dive into our architecture to show how to meet our primary design goals: scalability, multi-tenancy, resilience and extensibility. One important aspect is our integration with Monasca that uses common components, such as Kafka, and common architectural patterns. Another highlight that we will point out is our newly implemented REST API, that facilitates Logging as a Service in the spirit of Loggly. Here, authentication happens via Keystone, which extends the existing ELK stack by adding multi-tenancy, applying the tenant model of OpenStack.

Finally, we will give a demo of our solution.

Keywords: monasca, elk stack, elasticsearch, fujitsu, hp, logging, log management, monitoring

Ceph is a fully open source distributed object store, network block device, and file system designed for reliability, performance, and scalability from terabytes to exabytes. Ceph utilizes a novel placement algorithm (CRUSH), active storage nodes, and peer-to-peer gossip protocols to avoid the scalability and reliability problems associated with centralized controllers and lookup tables.

The community has been very active integrating Ceph into OpenStack. This is getting better each release.

With Juno we reached a critical step in terms of feature, robustness and stability. Kilo is even better, and Liberty is promising but we are not done yet.

In this session, Sebastien Han and Josh Durgin from Red Hat will describe the current state of the integration of Ceph into OpenStack and where are we heading to in terms of roadmap. They will go through all the OpenStack projects.

One of the most recent addition to Sahara (OpenStack Data-Processing-as-a-Service component) was the Storm plugin. Up to Juno, Openstack Sahara included only batch processing alternatives and Storm is one of the most popular open-source tools for real-time data analytics and stream processing. 

Implementing real-time data processing differs from Hadoop and other batch processing approaches because the data cannot be stored and then processed. The processing takes place while the data traverses the system. As a consequence sub-second processing latencies can be achieved. Therefore, this plugin enables new types of applications to be executed in Sahara. 

Real-time data processing is increasingly popular. Many such applications are now in our daily routine. One example, is online data summarization, where a high volume data feed (e.g., logs from a large cluster or sensor network) is summarized to let only relevant events be stored in a database. Other common examples, with varying performance and scalability requirements, include fraud detection, trend topics, and high frequency trading.

In this talk, we will present the Storm plugin for Sahara and guide the user through the essential steps to setup a scalable real-time data processing application. We will also share our plans for improving real-time data processing in OpenStack.

The work of running OpenStack has been getting easier; unfortunately, it's still just as hard to operate the underlying physical data center.  In this presentation, we're going to talk about how to use open source tools build a consistent and repeatable underlay for your OpenStack infrastructure using OpenCrowbar and OpenContrails.

Most importantly, we'll talk about how to TEAR IT ALL DOWN AND START OVER.

The RackN and Juniper teams have been collaborating to create an automated ready state environment that is hardware,  distribution and devops tooling agnostic with both physical and network topologies automatically configured for OpenStack environments using open source technologies.

In this session, we'll start with some first principles on physical Ops to ensure OpenStack success then we'll show how to duplicate this experience with your own infrastructure.