I've just enabled nightly builders from the release automation system on the Mozilla 1.8 tree (see bug 417147 for details).
I've blogged on this previously, but just to reiterate some of the reasons:
- unify the (very fragmented) nightly and final release processes (tools, procedure, etc).
- move away from Tinderbox client to Buildbot
- use the same set of machines for both nightly and release
The first point is a really big one for me, using totally different tools for nightly and release means that we don't get much testing of our release-only procedure and tools, so we often hit unexpected bugs on release day, and it also leaves nightly users without the benefits we provide for releases like automated update verification, updates for all locales, thorough error checking and monitoring of build machines, automated staging runs before pushing changes live, for a start.
The current setup still uses Tinderbox, it's just being invoked by Buildbot, so developers should notice no change besides new hostnames. We're trying this out on 1.8 branch first before we tackle 1.9, so far it has been quite smooth but please let us know if you notice anything out of the ordinary. We have not switched over perf tests yet, but we expect the results to not change (although we may want to merge some graphs for developer convenience, etc). This will happen before the old machines are turned off.
We're planning on turning off the older 1.8 builders sometime after February 25th, so please do let us know if you see any problems. I've left a note with the names of the new builders at the top of the Mozilla1.8 Tinderbox tree.
This is only one tiny step towards improving life both for the build&release group and also developers and nightly testers, but it's quite significant from an infrastructure point of view, and has been brewing for a long time. I'm not sure what the next steps are going to be, but I've written up some thoughts on where I think we should go and why.