OpenStack Mitaka Design Summit

As OpenStack itself has continued to evolve it's governance model, Neutron has done the same. In a world where we're all operating under "The Big Tent", Neutron has carved out it's own effort dubbed "The Neutron Stadium." During Liberty we've added a lot of new things into the project, from APIs to plugin and driver backends to new services.

This talk will cover a background on Neutron, how it's evolved as the OpenStack world has changed, what the Stadium has brought inside, and where things are headed. You will leave with a clear understanding of how Neutron is a platform, what has changed and what the future holds in this area.

OVN is a new network virtualization project that brings virtual networking to the Open vSwitch user community.  OVN includes logical switches and routers, security groups, and L2/L3/L4 ACLs, implemented on top of a tunnel-based overlay network.  For physical-logical network integration, OVN implements software gateways, as well as supports hardware gateways from a variety of vendors.

The OVN architecture simplifies the current OVS integration within Neutron by providing a virtual networking abstraction.  OVN provides Neutron with improved dataplane performance through shortcut, distributed logical L3 processing and in-kernel based security groups, without running special OpenStack agents on hypervisors.

In this presentation, we will provide an update on OVN's progress and provide a demo.  The demo will show an OpenStack-driven OVN deployment connecting containers and hypervisors.  We will demonstrate newly added features such as security groups and logical L3.  We will also discuss how users can start using OVN in their own deployments.

The success of containers through Docker has created a new operational flow and ecosystem far more reaching than anyone initially expected. Both as a project and a model that unifies, improves, and enforces consistent deployments, while empowering application with near-native performance in a highly-decoupled microservice approach, Docker has created a movement for the next wave of the Internet.

In doing so, numerous projects & products have spiraled out to capture a piece of the industry that facilities the adoption and usage of containers even further. Lost in all the noise of announcements, blog posts and articles is the core of what we all as technologists really care about: what are my actual viable options and what should I know about them?

In this talk, I will walk through the R&D work and market analysis that I’ve done in the container ecosystem and educate you on what technologies are worth tracking and incorporating into your stack including, but not limited to: Kubernetes, Swarm, CoreOS, Flocker and Hyper.

DVR (Distributed Virtual Router) was first implemented at Juno. Virtual routers needed to work on Network Node, but, with DVR, they can work on Compute Nodes.  Since Network Node runs virtual routers and some other networking functions for all VMs, it was a single point of failure and can be a performance bottleneck.  If you need a highly available OpenStack system, you needed a cluster of Network Node. And if you need to scale out the networking, you needed to add more Network Nodes. 

With DVR, those issues can be addressed. At Juno, virtual routers were distributed to Compute Nodes, but DHCP and SNAT stayed put on Network Node due to some technical challenges. We still need Network Node for them, meaning we also need a cluster for HA and more Network Nodes for scaling out the system. Distributing them, we can complete DVR and we no longer need Network Node at all. We submitted a RFE for DHCP and SNAT distribution and worked on them, targeting for Liberty[1][2]. In this session, we will explain what were the technical challenges and how we solved them. Also explain how we can use the completed DVR and what the benefits are for us.

[1] https://bugs.launchpad.net/neutron/+bug/1468236
[2] https://bugs.launchpad.net/neutron/+bug/1467471

The technology industry has been abuzz about cloud workload containerization since the open source Docker project became a phenomenon in early 2014. 

Meanwhile, an OpenStack Containers Team was formed and the Magnum project launched to provide users with a convenient Containers-as-a-Service solution for OpenStack environments. 

As the potential of both technologies emerged, many wanted to see shared governance over the baseline container specification and runtime technology to ensure an open cloud ecosystem. 

This past June, a new group was formed with a goal of creating open, industry standards around container formats and runtimes, called the Open Container Initiative (http://www.opencontainers.org).

So how will OpenStack Magnum influence - and be influenced by - the new OCI group? Why is the OCI under the stewardship of the Linux Foundation? What is the scope of the OCI effort? What project goals and/or principles will guide their work? 

Attend this session to learn the following: 

• A brief history of the open container ecosystem and the major benefits that containerization provides
• An overview of the Magnum CaaS plugin architecture and design goals
• Insider details on the the progress of the Linux Foundation Open Container Initiative (and the related Cloud Native Computing Foundation)
• What it all means for deploying container orchestration engines on your cloud with OpenStack Magnum

Neutron has established a simple, and now broadly accepted, northbound API for network virtualization. In this talk, we will explore the use of Neutron for supporting multi-host and multi-tenant networking for Docker containers as well as OpenStack VMs. In particular, we use the new Docker plugin architecture along with the recently announced networking framework, libnetwork. We show how the libnetwork remote driver can be used to utilize Neutron as the networking backend and will discuss pros and cons of using Neutron in this role.

We will discuss performance comparisons between the use of Neutron with that of the built-in overlay driver, as well as other network drivers being developed for Docker. This talk will give attendees a clear picture of how Neutron can be the unifying networking engine for VMs and containers.

While containers have been around for decades, they are only now becoming the rage for developers and DevOps practitioners alike. Indeed, they have seen a sudden surge in popularity and are rapidly becoming the standard for developing, packaging and deploying applications. What’s behind this?

As more and more companies adopt microservices architecture, encouraging rapid prototyping and frequent change, there may be implications for networking in general as traditional networks haven’t been designed for agility. What are they?

This panel moderated by Eric Hanselman, Chief Analyst, 415 Research will cover the inside scoop on the answers to these questions and more from the leading vendors and nimble startups involved with containerizing networking. We’ll also consider how vendors are able to help organizations operationalize their networks and eliminate the IT silos, given the  “don’t touch the network” attitude. Be prepared to hear about all about the emerging trends in containerized networking.  

In the Liberty cycle we worked on a new addition to Neutron, Quality of Service (QoS). In the network jargon QoS is about limiting, prioritizingor guaranteeing speed of traffic, in this case, on neutron ports.  The work in the Liberty cycle was focused mostly around the implementation of bandwidth limiting and laying out the QoS models to support future work.

In this session we'll cover the API changes and the new concepts that were introduced as part of this effort, like QoS rule, rule types and QoS policy.  We'll also share insights on how to get your Neutron driver to implement the API and make QoS available to your users.

Kuryr is a new project under Neutron's big tent that makes Neutron networking available to Docker containers by means of a Docker plugin.

In this session we will introduce Kuryr and show how it provides networking for containers in plain Docker environments and in mixed Docker, OpenStack environments.

Networking for container is rapidly evolving and overlay network is emerging as a popular choice given its simplicity.  One example is Flannel, a generic overlay network using IP datagrams on top of UDP.  Once deployed on a specific underlying infrastructure such as Neutron, there may be opportunities to optimize for performance and to ease some of the limitation of the overlay network.  In this talk, we look at the particular case of a Kubernetes cluster with Flannel running on Neutron networking and VM or baremetal.  This cluster would be provisioned by Magnum.

We will show how Flannel works with Neutron and Kubernetes' internal routing by tracing a round trip message.  We will quantify the overhead of Flannel and identify potential performance optimization.  We will look at the limitation of Flannel from the container perspective and show how some can be mitigated by Neutron.  We will show a demo for these features in a POC based on Magnum and Neutron.  Finally, we will discuss how the POC can be generalized to apply to other overlay  networks.

Of everything that we can build and deploy in a highly-available fashion in OpenStack, deploying highly available networking has been one of the trickiest, most complex aspects to get right.

In Neutron, when choosing the reference implementation, we have several components to consider when deploying for high availability:

• Neutron-server and its plugins are an API service as any other, with the same HA considerations, albeit with some aspects that deserve extra scrutiny when building for high availability




• Neutron L2 agents on compute nodes must ensure network connectivity for Nova guests




• and Neutron agents on network nodes must gracefully ensure routing and DHCP functionality even in the face of hardware failure

HA has also historically not been a very strong focus of upstream Neutron development, with vendors frequently filling the gaps with automation tools and smart service deployment.

In this presentation we cover:

• Upstream improvements on Neutron HA capabilities over the Juno, Kilo, and Liberty releases




• The approach taken for Neutron high availability in RHEL OSP and SUSE OpenStack Cloud




• Implementation details for Neutron HA in RHEL OSP and SUSE OpenStack Cloud

... All presented in the spirit of friendly competition, and upstream collaboration.

This talk is an update of the talk with the same name presented at the Openstack Design Summit in Hong Kong. Abstract and video for this talk are available respectively at http://openstacksummitnovember2013.sched.org/event/c6478ecf54d639de3b8b9958bfe9d450 and https://www.youtube.com/watch?v=RG6u_UuQOcs

Two years on, things have moved quite a bit.

Advanced services have advanced - and now they live in their own repos, plugin have been decomposed, and Neutron now is a stadium within Openstack's big tent.

This talk will go beyond choosing between a plugin and a ML2 driver, and will discuss the various options available to become part of the Neutron "stadium".

Monoliths versus modular plugins
Despite what one might think, this is not about ML2. Here we will discuss whether developers should develop Neutron integration a single monolitich plugin or as a set of plugins each one providing a different services.

Reasons for developing a ML2 driver
Apparently this is the only real option nowaday, but why? Are there cases where choosing to develop a ML2 mechanism driver is not really a no-brainer.

Extending neutron functionality through new services
The Neutron stadium offers different choices. We will look briefly looks at the services currently available in the "stadium", and provide useful insights into developing a new service, like the callback mechanism and ongoing efforts such as neutron-lib.

Where possible, examples targeting the "Human Defined Networking" Neutron plugin (https://github.com/salv-orlando/hdn) will be provided.

This talk has quite a good amount of technical content; the presenter is however committed to keep it lightweight, and, within the limits of the presenter's ability, entertaining.

OpenStack Magnum is a multi-tenant Containers-as-a-Service that combines OpenStack, Docker Swarm, Kubernetes, Mesos, and Flannel to produce a containers solution that works like other OpenStack services. In this session, we will detail and demonstrate Magnum to show you the features and capabilities, and explain why you should choose Magnum as the foundation for your containers strategy, regardless of what container orchestration software your users prefer.

Monitoring Docker Container and Dockerized Applications
Authors: Meenakshi, Satya, Rahul and Ananth

Container technology has been in existence for a long time in the form of LXC. It combines kernel control groups to isolate a process’s resource and support isolated namespaces. Docker came up with multiple added advantages over LXC,Some of them are listed below.

• Portable deployment across machines. Docker defines a format for bundling an application and all its dependencies into a single object which can be transferred to any docker-enabled machine, and executed there with the guarantee that the environment exposed to the application will be the same


• Application-centric. Docker is optimized for the deployment of applications,as opposed to machines. This is reflected in its API, user interface, designphilosophy and documentation


• Automatic build. Docker includes a tool for developers to automatically assemble a container from their source code, with full control over application dependencies, build tools, packaging etc.


• Versioning. Docker includes git-like capabilities for tracking successive versions of a container.


• Component re-use. Any container can be used as a "base image" to create more specialized components.


• Sharing. Docker has access to a public registry.


• Tool ecosystem. Docker defines an API for automating and customizing the creation and deployment of containers. There are a huge number of toolsintegrating with docker to extend its capabilities.

Challange:

• But don’t we need to monitor our system, containers, Applications running for our production system?


• How can we monitor such a distributed system, containers and distributed applications?

There could be three approaches for monitoring and remedy:

1. Reactive:
This kind of monitoring can be achived by the orchestration engine updates the monitoring system.
Example: 
Puppet: if any changes to configuration happens it revert back to the actual configuration which management config needs

2. Proactive:
This kind of monitoring can be achived by adding precautionary measures for the known issues, where, if the issue occurs it immidiately starts the precaution to eradicate the fault.

3. Adaptive:
This is better suited for monitoring a frequently changing system like docker containers, as it can adapt itself to the micro services that get intorduced into the containers. Now the question is “Is the adaptive montoring a full solution to the abovementioned challenges?"
- Answer is “NO”
- We need solutions at different levels

Different levels need to be monitored

The cluster manager:
The cluster manager manages the life cycle of a cluster of containers, few present day options are kubernetes and docker swarm

What needs monitoring ?

• Checking if the cluster manager is up and running and in ahealthy state


• Are nodes connected as per the correct expected configuration ?

The cluster nodes:
The cluster nodes contains the compute nodes or the VMs over which the containers would be provisioned.

What needs monitoring ?

• CPU utilization


• Memory utilization


• Swap space used


• Disk space used

Docker: 
The docker runs a demon on each docker node we need to ensure the docker daemon is healthy and running

The Container: 
The containers runs the micro services of the application so we need to ensure that the container is up and running and the vital points we need to look are:

• CPU utilization


• Memory utilization


• Disk space used


• Network I/O

This is the most critical part to monitor and the complex part also as the applications are moving towards the distributed applications.

Container Advisor(cAdvisor) provides resource uses and performance characterstics of the running containers . cAdvisor has native support for docker containers.

The Application(micro service):
While monitoring the application, we need something which can be adaptive to the system and quickly adjust to the changes in the new enviornment and be able to take control and sustain itself Which means, one needs to take care of app status, app messages, appinstance failure management, health manager detects and advises, new app instance deployed, routing tables etc.,

Workday as a SaaS leader in human resources management and finances has faced a number of challenges in adopting and deploying open-source technologies such as OpenStack and OpenContrail. One of these challenges was to find a quick way to evaluate and validate Open source technologies in development environment before considering those for production. Another was to integrate these solutions within the framework of existing company application deployment processes. 

In this session, we will talk about how we addressed these challenges by developing an multi-node Openstack CI framework using containers and virtual machines. The topics will cover: 

• Challenges we faced in developing and testing open source Openstack deployment tools


• Drivers for adopting Docker for development environment


• Benefits and challenges of using Docker


• How we built a CI system on Docker for quick validation of community developed Openstack deployment solution


• How (we or) operators can develop a transition path for deploying Openstack in production

Magnum provides container as a service, and Senlin provides clustering as a service to support autoscaling; therefore, to achieve autoscaling for container, it seems straightforward to simply integrate the two services. However, on closer inspection, many issues become apparent.  For example, an administrator would be concerned with scaling the host cluster, while a user would think about scaling the containers.  The different policies from the two levels of scaling may interact in unexpected ways.  Monitoring data for containers and load balancing to containers may require additional support.  Some container managers such as Kubernetes may offer native support to autoscale containers, while other managers such as Swarm may not and require external autoscaling support.  Users may want to autoscale with or without heat template.
                        
To explore these issues and look for best practices, we conduct a POC in integrating Magnum and Senlin.  In this talk we present the results of our POC effort and the new features and enhancements for both projects that we have identified as part of this integration effort.  

Many applications still in use today were constructed before web­scale IT designs were more prevalent and therefore lack the features and flexibility of cloud­developed applications.

Traditionally, these applications would need to be completely re­written or redesigned to incorporate modern auto­scaling and self­healing functionality into the application service itself.

In this talk, we will look at:

• Taking advantage of current capabilities founded on available OpenStack orchestration, modularization, and monitoring and alerting tools to provide scale without rewriting the application


• Relying on Murano orchestration combined with OpenStack layout to enforce best practice self­healing rules in deployment


• Simplifying delivery of auto­scaling and self­healing from the service catalog

By attending this presentation you will understand how to apply key features of Murano, Kubernetes, Docker, and Ceilometer to incorporate auto­scaling into existing application frameworks. You will be able to establish your own specific OpenStack availability zones, host groups, flavors and images to enforce resilient application service best practices and incorporate the use of these features and capabilities in conjunction with currently used deployment mechanisms.

Application developers are rapidly moving to container-based models for dynamic service delivery and efficient cluster management.  In this session, we will discuss a OpenStack production environment that is rapidly evolving to leverage a hybrid cloud platform to deliver containerized micro services in a SaaS Development/Continuous Integration environment.  Kubernetes is being used to simplify and automate the service delivery model across the public/private (OpenStack, AWS, GCE) environments and is being introduced in a way that eliminates extra overhead and engineering effort.  Lithium is actively contributing to key open source upstream projects and working closely with its engineering/development teams to optimize software efficiency with an elastic cloud architecture that delivers on the benefits of cloud automation.

Be prepared to have your mind blown away by witnessing the power of Kolla. Kolla uses Ansible for deployment automation and Docker for containerizing OpenStack services to deliver Operator nirvana.

Learn why Kolla is like Willy Wonka's Chocolate Factory for OpenStack Operators. We will tell the story of the Kolla project and the community best practices used to build it. Do you want simple and reliable upgrades? Experience how Kolla makes that a snap through Docker image-based deployments. Tired of inflexible deployment solutions? Experience how Kolla provides uber flexibility. Want to deploy from source or packages with choice of distribution? See how Kolla provides choice in operating system and OpenStack distribution.  Do you want to run OpenStack as micro-services instead of a slew of individually managed packages? Experience how Kolla transforms the OpenStack Big Tent into a micro-service architecture.  Curious what happens when a disk error corrupts your docker containers volume? Experience how Kolla can quickly repair the containers and get the OpenStack deployment into a non-degraded operational state.

Get an insider view of our diverse community's project and understand why Kolla is your golden ticket to deploying OpenStack clouds.

Load balancing as a service has been one of the critical features asked for by cloud tenants.

For the Liberty release cloud providers such as Rackspace, HP, etc. have partnered with the community and load balancer vendors such as Radware and A10 to enhance the new LBaaS v2 service with Horizon support, L7 redirection, and a new service VM based reference implementation.


 

This talk is on trends for keeping state in clusters of ephemeral containers. Containers are a popular new way to deploy applications. Containers give you benefits like standard container formats, resource isolation, and easy to use deployment tools. However, there are a number of caveats to using containers. One caveat is that many benefits require you to be careful about how you store application state.

Containers are normally managed in such a way that they can be started and stopped easily in order to provide higher availability and better utilize cluster resources. However, this methodoligy has it's tradeoffs. One tradeoff is that each container is generally started in the same state each time. This makes it difficult to save data such as databases or caches in containers.

I will present a number of challenges regarding storing state.

1. Allowing data to survive restarts


2. Allowing data to be moved between hosts


3. Managing storage solutions

I will then present a number of strategies that can be used to meet these challenges.

1. Storing data via mounts to the host machine


2. Storing data in external services (CloudSQL, RDS, S3, Object Storage, Cloud Datastore, etc.)


3. Storing data in cluster native database apps (Cassandra, Riak, etc.) 

I will present these strategies in the context of running Kubernetes on OpenStack. Kubernetes is a tool created by Google for managing clusters of containers and has support for OpenStack.

I believe this talk will help attendees visualize how they can map their current problems and workloads to containers. This should make it easier for them to make the leap to using containers for their deployments.

Containers are a much talked about, much hyped technology, but what exactly are they and how do they work?  What is the different between an Application Container and an Operating System container? This talk will try and take the technically adept beginner through the details of what containers on Linux are, how they're brought up, where the component technologies, like Cgroups and Namespaces fit in and how and why containers differ from hypervisors.

We'll begin by covering ancient history: the BSD chroot() system call and why this lead to the first idea of containing service based applications (and how this lead to the mount namespace), followed by a brief digression into IBM DPAR and Solaris Zones, Moving on to the Parallel developments of beancounters and cgroups and finally describing the unified container interface in Linux today consisting of the current set of Cgroups and Namespaces.  Along the way, we'll describe how container guests can directly interact with the host operating system and vice versa and why this is important for application containers.  How operating system containers effectively run different versions of Linux (like running ubuntu on RHEL) and why they differ from application containers.  Finally, for completeness, we'll look at applications which are themselves integrated with container technology and what they do with it to enhance their own functions.

Container technologies offer the exciting prospect of rapidly scaling applications and services without the large overhead of traditional virtualization environments. However, container technologies bring security vulnerabilities that a skilled intruder running inside a container can exploit to infiltrate other containers and eventually take over a cloud environment.

In this talk, Intel’s security, virtualization and Linux technologists collaborate to show how a trusted container environment can be deployed in an OpenStack environment that will:

• Ensure a root of trust for the platform on which a containerized app is deployed through trusted platform modules


• Encrypt the containerized workload and manage the key exchange process so it can only be decrypted and deployed on the targeted server as a trusted container


• Rapidly launch the trusted container in a fraction of the time it would take to launch a traditional VM


• Protect each container from other potentially rogue containers through isolation technologies already present in Intel® Architecture servers

This capability opens the door to a variety of Enterprise usages for OpenStack, which will be outlined