OpenStack Mitaka Design Summit

Moving from an era of vertical scaling and infrastructure resiliency to a world where reliability in the infrastructure is not assumed and application owners must start thinking about horizontal scaling and fault tolerance at higher layers can be challenging. A deep dive into what are the best practices for deploying distributed, scalable applications on cloud, where are the challenges and what solutions scale well.

Hear real world examples of how Comcast stood up an internal private cloud service leveraging OpenStack and Ceph. In less than 3 years the cloud is supporting over 500 internal customers including large scale, high performance applications such as our X1 platform and Residential Email.

A review of what architectures, tools and services work well for large scale stateless applications as well as how smaller apps tackle their deployments. Answers to where and how our apps securely manage state, store data, and plan for failure.

Manila, at its core, provides basic provisioning and management of file shares to users of an OpenStack cloud. The Sahara project provides a framework to expose Big Data services such as Spark and Hadoop. Together these two projects create a solution that is greater than the sum of its parts.

Natural synergy and popular demand led these three teams to develop a joint solution to expose Manila file shares within the Sahara construct to solve real Big Data challenges.
  
This talk serves as a brief introduction to what these two projects encompass as well as a detailed look at the joint integration work that was created involving:

 o      Sahara Data sources
 o      Manila File Shares
 o      Horizon Integration
 o      Sample Workflows
     
 Some administrative enhancements to Manila will also be covered including:

 o   Manila snapshots
 o   NFS Connector for Hadoop
 o   HDFS driver for Manila
 
Finally, a demo will be presented showing a Sahara data processing job running with binaries, data sets, and results hosted in Manila file shares mounted on a Sahara cluster.

Attendees will leave this session with an understanding of Manila and Sahara integrations and tangible use cases where it can be leveraged as a key component of a Big Data application deployment.

Learn about the newer features of OpenStackClient:

* simplify the experience of communitcating with multiple clouds
* operate as a long-running process to eliminate repeated loads/forks as a subprocess
* explore the plugin interface and capabilities exposed
* what changes with the adoption of the Python SDK?

Let's talk about the application workloads and scenarios you can envision on a truly open cloud. What do you think about owning your health data or fitness data? How about running the infrastructure on an OpenStack cloud?

After learning that her eleven-year-old son was diagnosed with Type I diabetes in 2014, Anne began investigating technology to help with the daunting task of living with the day-to-day management of the condition. True story, she found a way to use OpenStack resources to monitor data from his continuous glucose monitor (CGM) to gather not just five data points a day but nearly 300 data points in a 24-hour period.

Hear the story of finding Nightscout, a collection of open source projects that can run real-time monitoring applications on OpenStack clouds.  While the project's original approach to collect the data required an Android phone attached to the CGM receiver, the project now has a REST API for collecting and displaying data. Justin has a family member with Type I diabetes also, and we can compare and contrast the proprietary approach with the open source approach. We can demonstrate the Nightscout web application and iOS app running on a Rackspace Cloud server with node.js connected to a MongoDB database.

While the CGM use cases are medical devices with medical data, we also want to explore consumer data collection such as fitness trackers. Let's discuss the full spectrum of open source and open data in an open cloud.

In this session, we will talk about stuff that's going to make developing Swift easy by offloading the hard problems of storage. OpenStack Swift makes your application better. We can find dozens of storage systems that can do read write and delete but the fact that you can use swift for these advanced features means that you're able to focus on making your app awesome rather than worrying about the hard problems in storage.

We will cover how application developers can take advantage of some of the more advanced features in Swift, including... TempURL, StaticWeb, FormPost, and Static or Dynamic Large Objects among others.

IoT is one of the biggest topics in IT system today.  In this session, we will discuss how we can achieve an effective IoT system on OpenStack.

Firstly we'll describe IoT use cases, and summarize some generic requirements for IoT backend.  Secondly, we'll present our reference design of IoT backend on OpenStack IaaS.  Finally, we'll discuss the result of fit and gap analysis of OpenStack itself as a platform for IoT backend.

This session includes following items.

What kind of components we need to enable IoT backend
How to design and create network model to gather up all data from distributed sources
How to support flexible data gathering, storing and processing of massive data
How to archieve multi-tenanty required for IoT platform

I'm slim shade yes I'm the slim shade all you other slim shades are just imita ... ok, so the joke only makes it so far before the rhyming breaks horribly.

The OpenStack Infra project is one of the largest and most active OpenStack End Users in existence. As a result, we've learned just about everything about how OpenStack operates for consumers, the ways it can fail and the way different clouds diverge. We've encoded all of that learning in a library called 'shade' which wraps the OpenStack clients in the business logic needed to get your work done. shade is currently at the heart of Infra's nodepool project, which is the program that provides a pool of nodes across multiple public clouds so that everyone's OpenStack test jobs can run, as well as the new OpenStack modules that are in the Ansible 2.0 release.

In this talk, we'll talk briefly about the motivation for and challenges of writing shade - but we'll mostly talk about how to use it to get stuff done. In the process, there may be some ranting about things which could work better but don't.

I do not promise to not rap.

You have got your IaaS cloud up and running but are your developers happy? Has your IaaS forced your developers to also become agile sysadmins? At Symantec, we want out cloud developers to remain developers and this means abstracting away complexities of compute, storage and networking details and instead focus on actual software development.

After we built our Openstack cloud at Symantec, our next logical step was to build a Platform as a Service that to provide even more automation and streamlining to our customer’s experience. In this talk, we will share our experience and lessons learned in building a unified customer facing PaaS solution around our Openstack cloud.

Some of the topics we will cover are:

• The results of our evaluation of different PaaS platforms that are out there like Cloudfoundry, Openshift and Deis


• Potential design architectures to use when building your own PaaS from scratch


• Adding multi-tenancy to your PaaS


• Benefits of running on both containers and VMs

Do you know C#? Want to learn how to write a scalable cloud application
using an OpenStack SDK? Come to this presentation!

We will discuss how you can use OpenStack.NET to
*    Create and destroy compute resources.
*    Scale available resources up and down.
*     Use Object and Block storage for file and database persistence.
*    Make cloud-related architecture decisions such as turning functions
into micro-services and modularizing them

On one hand, OpenStack is quite promising with open APIs, open-source reference implementation with extensible plug-in architecture to support different implementations. On the other hand, it is a nightmare for software developers building enterprise-grade distributed applications on-top-of OpenStack i.e., using only APIs available to users/tenants.  Such applications need to be elastic (scale-out and scale-in as loads fluctuate), highly-available (several 9's of availability), and support high throughput (several Gbps of traffic). Unfortunately, many of the primitives available in the physical infrastructure to be able to build such services are either non-existent or available via ad-hoc extension APIs at the virtual infrastructure layer in OpenStack implementations.


The Good (examples):

• Nova's CRUD API for VMs: The basic VM creation and deletion with several options are pretty solid APIs in the OpenStack and this allows an application orchestration engine to easily scale-out and scale-in the app VMs as needed. The placement options exist albeit bit restricted.



• Neutron's CRUD API for networks, ports, router, subnets: Neutron abstractions of the networks, ports, routers, and subnets makes it easy for an application orchestration engine to connect VMs to right networks for creating multiple app tiers with proper isolation.

The Bad (examples):

• No notification APIs from OpenStack services such as Nova, Neutron, Glance, and Keystone: There is no way for the application orchestrator process to timely detect rebooted/deleted/dead application VMs or ports. The only option is to periodically check.



• Security groups API mess: Depending on whether Nova is implementing the security groups or Neutron, the semantics of the APIs are different!!



• Network performance: The reference implementation's stack is too complex (layers of different bridges connected by veth pairs, iptables with resource intensive nf_conntrak) with multiple bottleneck locations. And different vendors' plugins have myriad of other limitations.

The Ugly (examples):

• Too much restriction on network connectivity without exposing proper APIs to customize: For example, source IP address spoofing is not allowed even in a local network, where as that is an essential primitive for building highly available services. Ad-hoc neutron extension APIs such as allowed-address-pairs and port-security APIs alleviate this but not being part of core APIs means it is not no guarantee that they are available everywhere



• Semantics of APIs left to the interpretation of plugins: for example, in a physical world, all servers connected to a switch can communicate at L2 without any restriction. However, with all of the neutron plugins, there is one or another restriction imposed that is not explicitly indicated (for example, can't have the multiple fixed IPs on a port, can't have the same IP on two ports, etc.)

Terraform is an open source tool that enables users to declaratively design infrastructure and have those designs materialize into working components. It is useful for developers who want to create or manage IT architectures across multiple clouds.

OpenStack support was added to Terraform earlier this year, enabling users to deploy cloud-based infrastructure inside the OpenStack clouds.

This session will cover the following topics:

• What Terraform is


• How to write a Terraform configuration


• Using Terraform with OpenStack


• Terraform and OpenStack internals

No prior knowledge of Terraform is required to attend. By the end of the session, you'll be ready to start deploying cloud infrastructure inside OpenStack using Terraform.

TLS Keys, Disk Encryption Keys, and Service Passwords, are examples of sensitive data that needs to be kept away from prying eyes, yet still needs to be readily available for automated processes. Storing passwords and secrets in config files in your version control system potentially exposes that data to actors who shouldn’t have it. Barbican provides a secure repository to store such data and the controls to ensure only authorized users can get to that data.

As part of Symantec’s enterprise cloud initiative, we are deploying Barbican to handle not only our own OpenStack Key Management needs, but also as a Key Management as a Service option for our product groups. Our journey with Barbican has been fraught with challenges and in this talk, we will share our experience and lessons learned along the way.

Some of the topics to be covered:

• Our uses cases for Barbican


• How we’ve deployed Barbican


• Operationalizing Barbican


• Our practices and lessons learned

Balancing needs of security and scale for an elastic cloud is tricky if not downright impossible. How do you roll out agile, self service Platform as a Service (PaaS) application clouds while in parallel ensuring protection for OpenStack API end points from DDoS attacks, separation of tenant and provider networks, perimeter endpoint security plus satisfy compliance requirements such as encryption in-flight and at rest?

This session will cover security at scale without dependence on existing technologies and tools like 5 tuple and IPTables. Come learn:

• How you can achieve regulatory compliance on per tenant basis




• How separation of tenant and provider networks can be done and simultaneously satisfy both parties security requirements




• How to leverage the use of next generation firewalls for intrusion detection and host quarantine




• How to protect OpenStack API endpoints - for example Nova and Swift - from DDoS attacks that overrun the database

Allowing implementors to “trust but verify” OpenStack clouds makes federation work.  This is done through SAML & Keystone's federation support for multiple OpenStack clouds. But what about audit data? How can you verify that the events emitted from a cloud service provider are true? And what about keys & secrets? How can you verify that the keys you have in your private cloud are being used by a cloud service provider correctly & securely?

This session looks at what federation use cases have been delivered in previous releases, what is currently being worked on, and the use cases left to help ease the experience of cross-cloud operations.   We provide a brief overview of the standard based CADF  federation audit format that has been adopted by the OpenStack community.  We then discuss enhancements that are being added across OpenStack projects beyond Keystone to support federation and audit capabilities.  Finally we discuss future enhancements that are needed to maximize the consumability of OpenStack federated cloud support.

We will share the experience how we use global keystone here at eBay, those are addressed by real questions:

The instances running in production environment have different security level than the ones running in development environment. Projects locates in high secured zones requires 2FA(Two Factor Authentication) to authenticate while others use password credential. We also introduced a more secured authentication method for service access - API Key, which restricts not only what project it would be grant access to but also where the key can be used. The dynamic project based policy makes that happen and easy to use/configure. We will take a deep look at it as well.

We also isolate the controlling services from the production services into the secured control plane. We enhanced the Keystone to a fully armed IAM(Identity & Access Management) and integrate all the control plane services with it.

We will also share the experience on how to reduce the PKIZ token size as for global keystone, the token size would increase per region basis.

• eBay multi-environment security model




• Fill the gap between keystone and a generic IAM




• The answer to more secured service access - API Key




• Dynamic Project Based Policy for API Key authentication & management




• eBay global keystone journey




• Make the token smaller!

Firewall as a Service in OpenStack requires several improvements for real-world deployment. In this talk we will share ideas that improve Performance of firewalls and enhance OpenStack FWaaS by supporting capabilities like Scheduling and Logging.

This session will include a Demo of the work in progress.

Blueprints

• Logging : https://blueprints.launchpad.net/neutron/+spec/fwaas-logging


• Scheduling : https://blueprints.launchpad.net/neutron/+spec/fwaas-policy-scheduler

Performance
The current version of FWaaS configures IPTable rules in a sub-optimal way. The proposed solution aims at segregating the rules dynamically and pushing only the relevant rules on to the IPTables.

Scheduling
One of the  value added feature of firewalls, used by most network admins, is the ability to schedule policies with a specific periodicity and time interval. The proposed solution aims at enhacing the FWaaS Horizon UI and Neutron plugin to enable Tenants to schedule firewall policies.

Logging
The current proposal aims at enhancing the FWaaS and enable logging on the firewall policies. The logs generated can be redirected to a Syslog server and can be analyzed by tools like Splunk.

In this talk, we present Sentinel, the platform providing fine-grained security to applications running on OpenStack. Sentinel is currently being used at web-scale within eBay to secure applications across multiple OpenStack clusters.

Sentinel provides a robust policy-declaration model to represent applications and inter-application dependencies, a highly-scalable policy engine to translate the policies into enforcement rules, a policy agent that applies the rules on endpoints automatically, and monitoring & auditing capabilities. The highly-scalable design of the policy engine enables rapid deployment of rules on hundreds of thousands of VMs deployed on multiple OpenStack clusters.

The talk will be organized as follows:

• Overview of the cloud architecture at eBay


• Architecture of Sentinel


• Policy declaration model


• Policy enforcement methodology, optimizations 


• Integration with OpenStack


• Automatic service-dependency discovery


• Monitoring, auditing and real-time visualization


• Comparison with OpenStack congress and OpenStack Firewall-as-a-Service (FWaaS) 


• Challenges

About eBay Inc.: eBay Inc. enables commerce by delivering flexible and scalable solutions that foster merchant growth. eBay Inc. properties include eBay Market Places, eBay Enterprise and StubHub. eBay Marketplaces delivers one of the world's largest online Marketplaces to customers. With more than 149 million active users globally, eBay is one of the world's largest online Marketplaces with more than 700 million items listed on its site.

Trusted Platform Module / Trusted Execution Technology (TPM/TXT) provides advanced hardware based security, root of trust, geo tag location, and helps deliver compliance reporting. However, integrating TPM/TXT into OpenStack is not for the feint at heart and requires changes to your cloud DevOps.

This session will review our experience, tips and tricks, and provide a live demo on how to accomplish this and reap the benefits of secure cloud. The demo will cover the geo location feature and hardware based trust feature that ensures your workloads are operating within boundary controls and on integrity measured trusted hosts. 

This session is for IT Operations, Sys-Admins, Architects and anyone interested in learning about the IT industry’s advanced hardware security root of trust delivered in OpenStackpolicy better for the future.  This talk is strictly about what is available today, utilities to make operator’s job better and what you can do about it.

Whether you are integrating Docker containers into an existing cloud, or building out a multi-tenant cloud implementation using Docker, it can be a significant challenge to ensure proper security is in place. In this session, we will unravel various threads of security topics that all come together to provide properly configured security and isolation for Docker containers. Many of our findings are based on our experience in building and securing the IBM Container service based on Docker technology on top of an OpenStack IaaS. Topics include:

• Usage and threat model
• Implications of sharing the kernel with the host
• How user namespaces provide isolation from the root user on host
• Docker engine configuration for security and limitations for preventing forkbomb, filebomb, DOS
• Security features and issues for Docker registry
• Docker API and lack of multi-tenancy capabilities

Lets encrypt all the things!

Well, lets not, that's silly - but there's a lot of smart things we can encrypt, some of them require shiny hardware but quite a lot can be done through the clever application of existing software.

In this talk Robert proposes a two tiered encryption model to be applied to an OpenStack deployment.

Foundational - Full Disk Encryption. Encrypting everything on disk is non-trivial when managing large datacentres full of gear. In fact the complexity of this task normally makes it prohibative unless using hardware based solutions. At HP we have developed a new way to approach this problem. It makes Linux Full Disk Encryption pretty painless, scales beautifully and finally does away with retroactive "Log in and type the key" type systems that are just plain horrible. We will peak beneath the covers of this solution and share the code with the community so that we can all deploy full disk encryption at scale in a reliable and safe way.

OpenStack Native - Cinder, Nova and Swift all have native encryption capabilities in the pipeline. During this section of the talk we review their progress and discuss when they can be integrated into running prouction clouds to create a multi-layered encrypted cloud.

Combining these technologies protects everything on disk from accidental loss or compromise while also cryptographically separating tenant data on disk - both have been strong asks for OpenStack for a long time.

In addition, we will introduce Project Marshal.

Project Marshal is an open source implementation of an agent that provides the missing piece of the puzzle for volume encryption.  Using the Barbican client API, it allows running virtual machines to access secrets stored in Barbican to use encrypted volumes with tenant managed keys.

We'll cover: 
- What is project “Marshal”?
- What are its features, claims, and roadmap?
- Where can I get the code?
- How can I help set priorities and contribute to Marshal?

The FWaaS project has been present since the Havana release. There was some serious discussion on what its trajectory should be and the feature priorities at Vancouver. As a community, we have been gathering inputs from operators and users to see what they would like to see happen and prioritizing to present the next steps and direction. We are also looking at the intersect with Security Groups. We will present the usecases, models, plan and welcome feedback.

OpenStack based private cloud environments deliver a variety of benefits to users with respect to flexibility, automation, and cost. The volume of traffic especially intra-vm (east/west) traffic, generated within the OpenStack clouds is enormous, continues to increase, and is not inspected or secured by current perimeter focused security appliances and solutions. Visibility into this network traffic and the ability to apply security controls including deep packet inspection where needed within the private cloud is of high importance to organizations considering next generation cloud architectures including OpenStack. As high profile security breaches continue to make headlines and elevate data center security to a board level concern for organizations implementing proper network security within OpenStack will become vital to the continued success of the OpenStack project.

Companies including both small scale startups and larger established security players have begun to tackle this challenge introducing concepts and products related to the micro-segmentation of networks that rely heavily on network virtualization platforms in some proprietary infrastructure contexts. In the OpenStack world, Neutron security groups and ACL controls provide a form of some of the micro-segmentation functionality available on other virtualization infrastructure platforms. Through its openness, OpenStack and its APIs have paved the way for the integration of third party software defined networking (SDN) controllers such as Midokura MidoNet that provide more complete micro-segmentation capabilities and enable the dynamic insertion distributed virtual advanced network security services such as network IPS, or next generation firewall.

This presentation will introduce the motivation for, challenges, and concepts involved in securing OpenStack private cloud network environments. We will start with a description of the problem space, namely east/west or intra-vm traffic within the data center. We will then discuss how to think about developing solution to this problem including high-level requirements. This will touch on topics including virtual security function orchestration, service insertion, and policy mapping. Finally, we will discuss a partnership and technology integration between Intel Security and Midokura that brings advanced network security service insertion to OpenStack environments. 

Time permitting a demonstration may be provided showing the joint solution deploying an open source SNORT appliance (IPS) and seamlessly inserting it into a MidoNet controlled network to protect workload VMs from being attacked by neighboring VMs on the same network.

Every week we are hearing about more organizations being breached. Whether it is healthcare organizations like Anthem, financial institutions like JP Morgan Chase, content providers like Sony Pictures Entertainment, or government institutions like the US Office of Personnel and Management, it seems like no one is invulnerable. Adjacent to this frustrating trend, is a total upheaval of the enterprise technology stack in the datacenter. Now the datacenter evolved to a private cloud and enterprises are interested in offloading, for cost efficiency purposes, some of those workloads to the public cloud. Hence the emergence of the hybrid cloud.

The hybrid cloud presents unique security challenges that haven't existed before. With workloads moving between public and private clouds, across OpenStack environments and potentially in containers, how is an enterprise IT team supposed to protect their data and their company, from being breached? Is it even possible?

FlawCheck believes data protection is not an insurmountable problem. But as technology changes and threads change, protection strategies and solutions also need to change. In this presentation, we’ll cover the risks associated with hybrid cloud environments, with a particular emphasis on malware, vulnerabilities, remediation management of hybrid cloud environments, and breach avoidance.