rhelmer's blog

Tinderbox

I have been meaning to respond to a bit of Aki's post which linked to me a while back.

I totally agree on quite a bit, although I'd argue that unless someone really steps up, takes a leadership role, and sets a clear future direction, then sticking with Tinderbox indefinitely is going to continue to give you diminishing returns. Tinderbox 1 has been in maintenance mode for a very long time, although cls, bear and reed do a great job of keeping it secure and limping along. Tinderbox 2 was maintained by bear for a while when he was at OSAF but he suggested Buildbot as a better alternative, and Tinderbox 3 looks like a great proof of concept but has been inactive for a very long time.

I feel that it's better to contribute to an already active community that has a lot of momentum behind it, instead of trying to build support behind home-grown products like Tinderbox and Bonsai, given the amount of work it is to build and maintain an active community and the current state of these projects. There were no active competing projects when these tools were released, and they really set the bar at a time when "continuous integration" had yet to be coined. Overall they've been hugely successful and delivered a lot of value to Mozilla and others, but without a driving force behind new development, they are not keeping up with demand. I could give you a bunch of little examples, but I think that the fact that the "blame" column (which is a critical feature) has been empty since the switch to hg says it all.

rhelmer covered the current tinderbox/buildbot split, and is among the voices I've heard/read calling for a move away from the waterfall view, which I don't completely understand. I do understand that the waterfall is far from ideal as a solitary view. But it does represent the activity of builds and build machines over a brief amount of time quite well. Even better when you have a guilty column ;-)

So, why not have both? Or multiple? Not to clutter, but to present different ways of accessing the data. Each with their own strengths.

I don't think that the waterfall is bad, it is actually quite brilliant for certain use cases; however the waterfall is at one end of the spectrum, with something like Dolske's isthetreegreen.com on the other side, and things like tinderboxpushlog somewhere in the middle. So in essence I agree, but I think the waterfall is actually not that useful in most cases. It's a pretty low-level, diagnostic type of interface.

Why do people visit Tinderbox? Here is what I think:

Should I pull the tree ("Will It Build?")
Can I check in ("Is the tree open?")
Who broke the build (and how)?
Has there been a regression in performance or other metrics?

Out of these, only the latter two are served by the waterfall, and that's only a starting point for this kind of investigation (which the waterfall does an OK job at).

I think that the first two are a much larger subset of users, and a huge and complex display is actively hurting them. Regression hunters need a much larger arsenal of tools, and the waterfall may not be the best place for them to start, and certainly isn't the last place to visit (they'll need build logs, graphs, etc.).

There's a ton of innovation going on around build and release right now, for example I really like how Hudson approaches the problems here, and also has direct support for release processes. Like Buildbot, it doesn't do everything Tinderbox does, and it has it's own tradeoffs. It's not a drop-in replacement for Tinderbox.

A drop-in replacement for Tinderbox is an interesting notion, but I think it's worth taking a step back and figuring out if you're really getting the value you could be. I think this says it better than I can:

The telephone destroyed the telegraph.

Here's why people liked the telegraph: It was universal, inexpensive, asynchronous and it left a paper trail.

The telephone offered not one of these four attributes. It was far from universal, and if someone didn't have a phone, you couldn't call them. It was expensive, even before someone called you. It was synchronous--if you weren't home, no call got made. And of course, there was no paper trail.

If the telephone guys had set out to make something that did what the telegraph does, but better, they probably would have failed. Instead, they solved a different problem, in such an overwhelmingly useful way that they eliminated the feature set of the competition.

The list of examples is long (YouTube vs. television, web vs. newspapers, Nike vs. sneakers). Your turn.

rhelmer's blog

Other articles

making updates easier

releases on tap

openSUSE build service

OS as platform

rel-o-mation slideware!

moving 1.8 nightlies to release machines March 5 2008

moving nightly Mozilla1.8 Firefox to release automation system

summarizing build-on-checkin feedback

tinderbox to buildbot: moz18 branch

Other articles

links

social