Recently, Martin Fowler updated his Continuous Integration paper, "For most projects, however, the XP guideline of a ten minute build is perfectly within reason. Most of our modern projects achieve this. It's worth putting in concentrated effort to make it happen, because every minute you reduce off the build time is a minute saved for each developer every time they commit. Since CI demands frequent commits, this adds up to a lot of time."
"The commit build is the one that has to be done quickly, as a result it will take a number of shortcuts that will reduce the ability to detect bugs. The trick is to balance the needs of bug finding and speed so that a good commit build is stable enough for other people to work on."
"A simple example of this is a two stage build. The first stage would do the compilation and run tests that are more localized unit tests with the database completely stubbed out. Such tests can run very fast, keeping within the ten minute guideline. However any bugs that involve larger scale interactions, particularly those involving the real database, won't be found. The second stage build runs a different suite of tests that do hit the real database and involve more end-to-end behavior. This suite might take a couple of hours to run."
Breakage patterns for a build is listed in, "Avoiding Continuous Integration Build Breakage Patterns" it includes: "Five O'clock Check-In", "Spoiled Fruit" and "It Is a Small Change".
The overall effect of having a slower build is that tested user visible functionality becomes more difficult over time. Each new high level test increases the build time and understanding what has broken takes longer. Often time is not taken to ensure that your faster atomic/unit/integration tests cover what was missing. So the importance of maintaining a green build when you have worse feedback becomes more important but the cost of maintaining it becomes more expensive.
Some of the discussion that I've had is on this thread on the boost mailing list.
No comments:
Post a Comment