06.13.07

Builds suck

Posted in Software Development at 6:42 pm by BestFriendChris

Builds suck. Normally I’d rag on Ant, but I don’t really blame the tool (even if it does suck…). Ant doesn’t kill kittens, developers do. I think the fundamental problem is that nobody (ESPECIALLY developers) take a build seriously. The sad reality is that we treat a build system the same way we used to treat regular code. We write it once, maybe throw in a few echos, and run the build. We check to see that the files we expect it to create are there, and then we forget about it. That is, of course, until we notice that it stops working (usually LONG after it really stops working). This is ridiculous. There’s a reason we have jUnit.

I was talking with my partner in crime, Nilanjan about this. The project we are on has an insane build system. The last couple of days I’ve had to immerse myself into it. Ick. The sad thing is that it is not even close to being the exception to the rule. So, we started talking about what we think the best way to avoid this kind of intertwining dependencies and properties nested an insane number of imports away… Here’s the list we came up with (keep in mind that a fair number of things here we haven’t tried, they just seem to make sense. If you try any/all, please let me know. I’d love to know how they work out):

Your build is code. Treat it as such.

• Your build has pretty set requirements that show up over the course of your project. For example:
  • given a java file, create a class file
  • given this war file and this running app server, deploy the war to the app server
• We need some sort of unit testing framework that allows you to verify that your build still works as expected
  • should allow you to fake out all calls to the os and the filesystem so it will be fast and nondestructive (when testing a task to deploy to your production server, you should mock out the sftp call in some way)
  • should mock out all other imported build files. I think it would get too complex to test when your build file depends on another build file that depends on another build file… This probably is debatable, however.
• Refactor mercilessly. Please.

Nested builds are a tree

• Nested builds are when you have multiple sub-projects, each with their own build file.
• You should only be able to call build files down the tree, or siblings (more on this in a bit)
• A fair amount of the complexity I personally have seen deals with any build file being allowed to call any other.
  • It hurts to figure out who exactly depends on a specific task.
• Following this rule, at least gives you a smaller set to look towards when debugging your build.

All projects should have the EXACT same public interface.

• By public, I mean the tasks that other build files can call
  • not just what can be called from the command line
• If one project has a “jar” task, all should. And the jar task should do the same thing, in a common build file (see below)
  • That way, when project A requires a jar from project B, it knows that all it needs to do is call the “jar” task and it will be in a known place.
• This doesn’t mean that all build files are limited to the same interface, just that if you need to call a task in another build file, all build files need to provide that task in a consistent way.
• This is probably the weakest suggestion of all of them. Just something I came up with. Blame me, not Nilanjan.

Refactor out commonality.

• When factoring out common tasks, the common build file can not depend on any other files.
  • Trust me. One off is best.
  • This means that if your pull up the compile-unittest task that depends on the compile task, both need to be pulled up.
• Only use one common build file.
  • Again, even though it might seem to make sense, debugging becomes hell, especially when you factor out the common tasks between two common files. Arg.
• If a task to be pulled up is not used in EVERY build file, turn the task into a function (or macrodef in ant) and call it.
  • Functions are much easier to test than tasks with dependencies (any of which could possibly be overridden)
  • They should not depend on the state of any sort of global variables or properties. Only on arguments passed in.
• No global variables/properties
  • This will drive you mad if you depend on an imported build file to define your state.
  • Insanely hard to debug.

Due to the fact that I’m not suicidal and don’t like to cry (it really hurts your credibility as a consultant… I’ve… heard…), my initial stab at writing a unit testing framework for a build system will be using rake. I’ll post something when I have anything even started…

Comments?

Permalink