Work sessions are for Kolla contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Wednesday October 28, 2015 11:15am - 11:55am JST
Kiri room
Work sessions are for Kolla contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Wednesday October 28, 2015 12:05pm - 12:45pm JST
Kiri room
The newest OpenStack Security project - what features need to be prioritized to accelerate usage and adoption of Syntribos. What metric do we consider as making this a successful project?
Work sessions are for OpenStackAnsible contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Wednesday October 28, 2015 5:30pm - 6:10pm JST
Kiri room
We now have reasonable infrastructure in place to do functional and integration testing across ceilometer, aodh and gnocchi. However the actual level of testing being done is not really complete. In general we confim that the APIs are working but end-to-end full service tests that get into the corners are not really present (a notable exception is the new integration gate job, which is awesome let's have more of them). Some specific concerns: - In ceilometer, when the functional/unit test split was done, coverage provided by the unit tests took a pretty big drop. It's not yet clear if a combined run of the unit and functional tests provides robust coverage. - The aodh functional tests check the api but do not check the evaluator, notifier or event listener.Until recently the gnocchi functional tests did not check that gnocchi-metricd was doing its job. - Ceilometer now has its grenade testing in tree but very little is tested, really just sanity checks. We can do whatever we like here so it would be useful to more complex investigations. One thing to consider is what's being called partial-upgrade-testing (in line with rolling upgrades above): Upgrading some of the pieces of the system to the new version and leaving others behind and confirming that behavior is okay. - For those projects which are not on the release cycle, do we need some kind of upgrade testing for them? - We need more complete tests for those systems which use the messagaing bus and are primarily driven by notifications. This has proven a bugbear in testing because latency can throw things out of wack. - We also need better tests of transformers and other members of the measure sampling pipeline as corner cases are constantly revealed or situations where a change elsewhere in the system violated an untested assumption in the pipeline. The new testing tool "hypothesis" might be useful here. - We need tempest plugins for Ceilometer, Gnocchi and Aodh. (right? or the new integration gate job can cover the same level of testing?)-
we stopped providing volume=1 samples so we need to fill that gap it'd be good to start generating statistics from events such as: - lifespan of resource (duration of a resource) - event count - what else?
there are many possibilities to derive data from events: - track time difference between events (compute.instance.create.start and compute.instance.create.end) to derive meters - other stuff?
instance metadata refinement each poll for instance related meters per host, polls the nova api for metadata details and subsequently polls hypervisor for actual metric values. this means for each host, we hit the nova api. the idea here is to separate the metadata polling from metric polling so that we poll nova-api only once. this does mean that the metadata associated with metric potentially can be unsync'd.
healthcheck events we are blocking volume=1 samples so now the polled volume=1 samples are useless. they do offer ability to function as resource healthcheck which is still useful as events. this is discuss what we need to do build events from polling. - do we just switch the output to build events rather than samples? - how do we define/name healthcheck events
creating a new polling definition we currently leverage the pipeline definition to define polling. we probably shouldn't. if we define new interface, we can truly separate polling from processing. - we can also consider how to handle more granular polling.. ie polling frequency per project. - Stevedore as configuration is a bit problematic too: http://lists.openstack.org/pipermail/openstack-dev/2015-September/074203.html - polling logic to handle same multiple interval polling - ie. one pollster set for 5, another set for 10, we should really only poll once. - Do we want to continue with batching the polling tasks as a single task per interval. Isn't it more logical to have a single task per poll at whatever interval it wants? - this works but we need to redefine what a 'task' is. currently a 'task' is a request to retreive a single meter. using this definition we will have a redundant calls as we often are able to get multiple data from a single api req. - small requests are more scalable (horizontally) and things which do just one thing are easier to debug
As a system Ceilometer violates the DRY principle in a pretty horrible way: It keeps, within itself, all kinds of knowledge about the projects that it monitors and measures. This is a technical and social error which makes it difficult for the system to evolve quickly and address its own shortcomings because it is constantly looking to add and maintain support for those other tools. Ceilometer is already stuctured to make this way of doing things not a requirement. We should considering pushing functionality back to the monitored for the sake of both sides of the arrangement. Areas to consider: - We now have declarative meters in yaml that could be broken up into files kept by the projects. - Pollster plugins and notification listeners are stevedore plugins. These do not need to live in the ceilometer namespace, thus just need to use the right entry points. These kinds of changes mean that the projects will be able to fix issues more quickly and have greater control over data that is published. It will also make it more clear for end users that they have the power to add meters and plugins under their own steam without needing to wait on the highly frictioned OpenStack process.
Pros: - Projects can manage and maintain their own definitions - less code in ceilometer to worry about - less review and maintainance time for ceilo devs Cons: - Coordinating effort with other projects and getting them on board
currently, if we have multiple event alarm evalutors, there might be some race conditions in the following scenario: 1. ceilometer-notification-agent gerated 2 events of the same type in sequence, but with different traits, the 1st one would trigger the alarm, and the 2nd one won't, it then send the 2 events to aodh alarm evaluators 2. if there're multiple aodh alarm evaluators, it's possbile that the 2nd event would be processed before the 1st one, in that case, the alarm status in the DB might not be consistent, depends on the sequence of how the events are being evaluated
Work sessions are for Kolla contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Thursday October 29, 2015 2:40pm - 3:20pm JST
Kiri room
Work sessions are for Kolla contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Thursday October 29, 2015 3:30pm - 4:10pm JST
Kiri room
Work sessions are for RefStack contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Thursday October 29, 2015 4:30pm - 5:10pm JST
Kiri room
Work sessions are for RefStack contributors to discuss implementation details and making quick progress over specific issues, in a small work group environment.
Thursday October 29, 2015 5:20pm - 6:00pm JST
Kiri room
