Loading…
Thursday, October 29 • 9:00am - 9:40am
Ceilometer: Work session

Sign up or log in to save this to your schedule and see who's attending!

Functional and Integration Testing

We now have reasonable infrastructure in place to do functional and integration testing across ceilometer, aodh and gnocchi. However the actual level of testing being done is not really complete. In general we confim that the APIs are working but end-to-end full service tests that get into the corners are not really present (a notable exception is the new integration gate job, which is awesome let's have more of them).
Some specific concerns:
- In ceilometer, when the functional/unit test split was done, coverage provided by the unit tests took a pretty big drop. It's not yet clear if a combined run of the unit and functional tests provides robust coverage.
- The aodh functional tests check the api but do not check the evaluator, notifier or event listener.Until recently the gnocchi functional tests did not check that gnocchi-metricd was doing its job.
- Ceilometer now has its grenade testing in tree but very little is tested, really just sanity checks. We can do whatever we like here so it would be useful to more complex investigations. One thing to consider is what's being called partial-upgrade-testing (in line with rolling upgrades above): Upgrading some of the pieces of the system to the new version and leaving others behind and confirming that behavior is okay.
- For those projects which are not on the release cycle, do we need some kind of upgrade testing for them?
- We need more complete tests for those systems which use the messagaing bus and are primarily driven by notifications. This has proven a bugbear in testing because latency can throw things out of wack.
- We also need better tests of transformers and other members of the measure sampling pipeline as corner cases are constantly revealed or situations where a change elsewhere in the system violated an untested assumption in the pipeline. The new testing tool "hypothesis" might be useful here.
- We need tempest plugins for Ceilometer, Gnocchi and Aodh. (right? or the new integration gate job can cover the same level of testing?)-

https://etherpad.openstack.org/p/mitaka-telemetry-testing

Thursday October 29, 2015 9:00am - 9:40am
Kiri room
  • format json

Attendees (0)