06.13.07
Posted in Software Development at 6:42 pm by BestFriendChris
Builds suck. Normally I’d rag on Ant, but I don’t really blame the tool (even if it does suck…). Ant doesn’t kill kittens, developers do. I think the fundamental problem is that nobody (ESPECIALLY developers) take a build seriously. The sad reality is that we treat a build system the same way we used to treat regular code. We write it once, maybe throw in a few echos, and run the build. We check to see that the files we expect it to create are there, and then we forget about it. That is, of course, until we notice that it stops working (usually LONG after it really stops working). This is ridiculous. There’s a reason we have jUnit.
I was talking with my partner in crime, Nilanjan about this. The project we are on has an insane build system. The last couple of days I’ve had to immerse myself into it. Ick. The sad thing is that it is not even close to being the exception to the rule. So, we started talking about what we think the best way to avoid this kind of intertwining dependencies and properties nested an insane number of imports away… Here’s the list we came up with (keep in mind that a fair number of things here we haven’t tried, they just seem to make sense. If you try any/all, please let me know. I’d love to know how they work out):
Your build is code. Treat it as such.
- Your build has pretty set requirements that show up over the course of your project. For example:
- given a java file, create a class file
- given this war file and this running app server, deploy the war to the app server
- We need some sort of unit testing framework that allows you to verify that your build still works as expected
- should allow you to fake out all calls to the os and the filesystem so it will be fast and nondestructive (when testing a task to deploy to your production server, you should mock out the sftp call in some way)
- should mock out all other imported build files. I think it would get too complex to test when your build file depends on another build file that depends on another build file… This probably is debatable, however.
- Refactor mercilessly. Please.
Nested builds are a tree
- Nested builds are when you have multiple sub-projects, each with their own build file.
- You should only be able to call build files down the tree, or siblings (more on this in a bit)
- A fair amount of the complexity I personally have seen deals with any build file being allowed to call any other.
- It hurts to figure out who exactly depends on a specific task.
- Following this rule, at least gives you a smaller set to look towards when debugging your build.
All projects should have the EXACT same public interface.
- By public, I mean the tasks that other build files can call
- not just what can be called from the command line
- If one project has a “jar” task, all should. And the jar task should do the same thing, in a common build file (see below)
- That way, when project A requires a jar from project B, it knows that all it needs to do is call the “jar” task and it will be in a known place.
- This doesn’t mean that all build files are limited to the same interface, just that if you need to call a task in another build file, all build files need to provide that task in a consistent way.
- This is probably the weakest suggestion of all of them. Just something I came up with. Blame me, not Nilanjan.
Refactor out commonality.
- When factoring out common tasks, the common build file can not depend on any other files.
- Trust me. One off is best.
- This means that if your pull up the compile-unittest task that depends on the compile task, both need to be pulled up.
- Only use one common build file.
- Again, even though it might seem to make sense, debugging becomes hell, especially when you factor out the common tasks between two common files. Arg.
- If a task to be pulled up is not used in EVERY build file, turn the task into a function (or macrodef in ant) and call it.
- Functions are much easier to test than tasks with dependencies (any of which could possibly be overridden)
- They should not depend on the state of any sort of global variables or properties. Only on arguments passed in.
- No global variables/properties
- This will drive you mad if you depend on an imported build file to define your state.
- Insanely hard to debug.
Due to the fact that I’m not suicidal and don’t like to cry (it really hurts your credibility as a consultant… I’ve… heard…), my initial stab at writing a unit testing framework for a build system will be using rake. I’ll post something when I have anything even started…
Comments?
Permalink
05.11.07
Posted in Erlang at 12:04 pm by BestFriendChris
Ok. I know I already posted today, but this was just cool. A performance diagram comparing Apache to Yaws (An Erlang webserver). Apache dies at around 4,000 parallel sessions, while yaws still works at 80,000. Amazing.
Permalink
Posted in ErUnit at 8:15 am by BestFriendChris
I recently was pointed to this article by pragdave about test-first development in Erlang with EUnit. I’m going to use his examples to show you how it would be done in ErUnit instead. First, some history…
The reason I wrote ErUnit instead of just using EUnit is for a couple of reasons
- The best way to learn a language is to reimplement xUnit. There’s so much meta stuff going on that it really requires you to dig in the language. And when I say reimplement, I don’t mean, “Follow the xUnit spec exactly.” What I mean is leverage the things that make the language good to create a good fit.
- I didn’t see it on the web. Mostly because I assumed EUnit was for the E programming language. So, I searched for ErUnit and found nothing…
- Even after I found out about EUnit, it didn’t gel with me. Now, that might be due to there not being any good tutorials that I could find, but I just wasn’t seeing it. I don’t like using macros specifically for assertions as it makes the failures hard to debug.
- I really don’t like having your tests live in the same file as your production code… It makes your code much harder to understand if you have to double the size of the file just to test it…
- I’m not a big fan of the standard that I saw of using pattern matching for the tests. I personally think there should be a difference between a failed assertion and a failed pattern match within your code, and I don’t know how to make that distinction.
A huge benefit of going with EUnit is that it is MUCH more mature than ErUnit. I’m still tweaking it as need arises. One thing that I’m not sure if it really matters yet is all tests are run in their own thread, concurrently. I did that because it seemed super easy and fun to do, but I don’t know if you really get any benefit with that yet…
Well, on to creating a word wrap utility, test first, with ErUnit.
ErUnit in >60 Seconds
When creating a ErUnit test, you first name the file starting with “test_”. In our example below, when testing “text.erl”, we create the test file “test_text.erl”. ErUnit, with erunit_suite, has the convention of either:
- If your code and tests are in the same folder, add that folder to the path
- If your code is in the “ebin” folder and the tests are in the “ebin_tests” folder, adding both folders to the path
This allows you to break up the tests from the code it’s testing pretty easily. Within the test class, you have to add the erunit_test behaviour. Behaviours in Erlang (well, technically the OTP library in Erlang) are used to define what callback functions are required. In this case, the erunit_test callback requires a function call “tests” with no arguments to be exported.
I personally also add a no argument “run” function that just delegates to erunit:run() with the tests passed in. I also like to import all of the erunit functions I’m going to be using (test/2, assertEquals/2, run/1) and the functions I’m going to be testing. This makes the code later on easier to read.
The file so far is:
-module(test_text).
-behaviour(erunit_test).
-export([tests/0, run/0]).
-import(erunit, [test/2, assertEquals/2, run/1]).
-import(text, [wrap/1]).
run() ->
run(tests()).
As mentioned earlier, you must declare a no argument function called “tests”. All this returns is an array of tests. So, for the first example we want to test that the “wrap” function works with no words.
tests() ->
[
test("Should wrap no words", fun() ->
assertEquals([""], wrap([]))
end)
].
When you use the erunit:test/2 function, you pass in the description of what you’re testing and a fun/0 of the actual test. assert(Description, true-false-test), assertEquals(First, Second), assertEquals(Description, First, Second), and fail(Description) are all currently implemented. Behind the scenes, the “test” function spawns a new thread that will run the fun/0 and send the answer back to the current test runner thread.
The code for the “text” module is the same as in pragdave’s example, so I won’t repeat it here. In fact, I’m just going to show you the entire test file:
-module(test_text).
-behaviour(erunit_test).
-export([tests/0, run/0]).
-import(erunit, [test/2, assertEquals/2, run/1]).
-import(text, [wrap/1]).
run() ->
run(tests()).
tests() ->
[
test("Should wrap no words", fun() ->
assertEquals([""], wrap([]))
end),
test("Should wrap one word", fun() ->
assertEquals(["cat"], wrap(["cat"]))
end),
test("Should wrap two words", fun() ->
assertEquals(["cat dog"], wrap(["cat", "dog"]))
end),
test("Should wrap when more than 10 chars long", fun() ->
Expected = ["cat dog", "elk"],
assertEquals(Expected, wrap(["cat", "dog", "elk"]))
end),
test("Should allow word larger than 10 on its own line", fun() ->
Expected = ["cat dog", "hummingbird", "ibix"],
Actual = wrap(["cat", "dog", "hummingbird", "ibix"]),
assertEquals(Expected, Actual)
end)
].
The output when running from erl is:
1> test_text:run().
.....
ok
2>
When a test fails with this extra test added:
test("This will fail", fun() ->
erunit:fail("Called erunit:fail/1")
end),
…you get…
1> test_text:run().
....F.
-"This will fail" failed:
Called erunit:fail/1
fail
2>
To show an error, this is the test:
test("This will have an error", fun() ->
erunit:assert("This method does not work with a integer", wrap(1))
end),
…which produces…
1> test_text:run().
....E.
-"This will have an error" had error:
error:function_clause
{erunit,'-test_process/3-fun-0-',3}
fail
2>
…Not the best error message, but I’m working on it.
Permalink
04.30.07
Posted in Software Development, Thoughtworks at 2:55 pm by BestFriendChris
This last weekend I attended CITCON in Dallas. Awesome. I had some amazing conversations, watched Bret Pettichord blow me away with customizing RSpec, actually met Elisabeth Hendrickson (and received a sweet green wristband that says, “test obsessed”), joined in a great conversation about what we are looking for in a testing language, and was surprised to discover there are people out there that actually like ant (to be fair, though, he does work on AnthillPro). The only downside was there were 2 topics at the same time that I wanted to go to, so I attended a talk on code metrics and missed out on the CI server cage match (There were like 6 different CI Servers represented at the conference, and they all got in a room to compare features and benefits of each).
If you’ve never been to a conference in an OpenSpace format, you’re really missing out. The short (-ish) version is that there are no preassigned speakers for the conference. On Friday, we all sat in a circle and people could write ideas down on a 3×5 card stating what they want to talk about. It might be some tool you want to show off, some concept you want to tell people, or, just as likely, something you know nothing about and what to learn. The only requirement is if you suggest the topic, you have to facilitate it and help get the conversation going.
After we had about 30 topics, we set them up on a board showing the rooms and time slots. Then, we go to the next room to drink and chat. As the night moved on, people we encouraged to wander back over to the schedule and rearrange topics, vote on what they want to see, and combine topics together. If you see two topics at the same time you really want to see, you just swap one of the topics with a different one so you can see them both. While you would think this would cause all sorts of problems, it turns out that, in general, people aren’t jerks, so you don’t get a constantly shifting schedule as much as a gradually improving one. A key element of OpenSpaces is the rule of 2 feet. If you don’t feel you’re getting the most out of a session, it’s up to you to walk to a better session. If you suggest a topic and nobody shows up (Actually happened for one of them), feel free to move to a different one. This puts the onus on you to get something from the conference. There are plenty of smart people attending, so if you don’t learn anything, you must have been actively avoiding conversations.
One of the interesting things is that the conference is free and takes place from Friday night to all day Saturday. The effect of this is the people that attend are doing so from a very real desire to learn. They are spending their free time to improve their craft and learn from their peers. It really raises the bar of the attendees. It’s amazing how much a difference actually caring about what you do can make.
Well, I guess that’s about it. There are 2 more CITCONs this year: CITCON Asia/Pacific 2007 in Sydney, July 27 & 28, 2007 and CITCON Europe 2007 in Brussels, October 19 & 20, 2007. I’m pretty sure I’m going to go to the one in Brussels, (London 06 was just as awesome as this last one). If you’re in the area, I can safely recommend attending.
Final thoughts: When I first heard about CITCON, I focused on the “Continuous Integration” part of the name (which I didn’t have much passion around). I think it’s important, however, to understand that Testing, Agile, TDD and other xp-ish stuff are just as represented (if not more so). I’m glad I decided to go.
Permalink
04.23.07
Posted in ErUnit at 10:06 pm by BestFriendChris
So, I just wrote about my new favorite language Erlang. When learning a new language, it usually helps me to work on a simple problem I’ve solved before to understand how things work. So, I grabbed one of the coding exercises we use in recruiting at t-dub (no, I’m not going to post about them). Now, as every good xp-er, I started writing my test. Or, I should say, started looking for an xUnit framework for Erlang. Nada.
So, I wrote one myself. Test-first, I might add. (Trust me, it doesn’t get much more meta than that).
Download me. (or svn co http://codeby.bestfriendchris.com/svn/ErUnit/tags/erunit-0.1.0/)
To install, put it in the Erlang code path (on my OSX machine, it is /opt/local/lib/erlang/lib). Also, if you want to use the erlang script, add the bin folder to your PATH.
To write tests, you just need to name your test file test_whatever_you_want and add the following to the file:
-behaviour(erunit_test).
-export([tests/0]).
I’d show you some more example test code, but my code highlighter doesn’t support Erlang, and I don’t really have the time to hack a new language file together for it right now. So, I’ll link directly to the test case I used in writing ErUnit: test_erunit.erl
To run tests from the command line, type:
# run all tests in the current folder
erunit
# run tests in the “tests” folder
erunit tests
To run the test from erl, type:
% run all tests in the current folder
erunit_suite:run()
% run tests in the “tests” folder
erunit_suite:run(”tests”)
A few more notes:
- If you use seperate folders ebin and src for your .beam and .erl files respectively, ErUnit has the convention of using ebin_test and test. Again, look at the way ErUnit itself is packaged to get the idea.
- erunit_suite will run all tests in every subfolder of the one it is run in.This allows you to have some sort of hierarchy to your code and tests.
- erunit_suite is pretty smart about loading test files. It will make sure that all of the folders the test.beam files are in are included in the code path and auto-magically handle the case where you are using seperate ebin and ebin_test folders. Subfolders within either ebin or ebin_test are not yet supported.
- You might notice that a bunch of the features of most other xUnit implementations are missing. This is, at least in some cases, on purpose (in others, it’s because we’re talking 0.1 version). I’m only adding features as I need them. For instance, I haven’t added Setup or Teardown methods, mostly because I’m not yet convinced that they will be required in a functional language. And even if they are needed, it’s pretty trivial to wrap your test functions in a setup/teardown pair of funs.
- No Documentation? Yeah. I’ve actually been sitting on ErUnit for at least a week because I wanted to write some good docs for it. Since that hasn’t happened yet, I figured I’d at least put this out there and maybe get around to it eventually.
Comments/Suggestions/Complaints?
Permalink
Posted in Erlang at 9:01 pm by BestFriendChris
So, the other day I was shopping around for a new language. Specifically, I was looking for a functional language that wouldn’t cause my brain to bleed (and not Scheme. Love it dearly, but I needed something new). Enter Erlang. I found out about this little guy from a new beta book from the prag guys called Programming Erlang, Software for a Concurrent World (totally recommend it).
Bullets:
- Functional
No state == Hooray!
- Supports Hot Swapping of code
I know I wouldn’t even think of swapping any other way than hot
- Designed to support distributed, fault-tolerant, soft-real-time, non-stop applications
I actually know what that means! To be fair, I swiped that bullet off the
wikipedia page
- Super Sweet Thread Communication Mechanism(tm)
Threads are actually an extremely common part of Erlang development. In most languages, threads hurt kittens. In Erlang, they only hurt nasty dogs who probably deserved it.
I’d recommend checking out the prag book linked above and see for your self. In a bit I’m going to post about ErUnit, the xUnit testing framework I wrote for Erlang. In the mean time, here’s my initial stab at a TextMate Erlang bundle.
Permalink
04.19.07
Posted in Software Development, Buddha at 3:40 pm by BestFriendChris
ex·pert (ĕk’spûrt’)
n.
1. A person with a high degree of skill in or knowledge of a certain subject.
I’ve been thinking a lot about experts lately. This came about, as most things do, from several areas in my life.
First of all, I recently read a book called Freakonomics
. While I did enjoy the book (heartily recommend it in fact), I was really surprised by the disdain the authors had for “experts”. The bulk of the issues they had seemed to be focused on two things. Not only are some of these experts making unverifiable (or sometimes flat out wrong) statements about how things are, but everyday people are taking these statements as reality. For example: A parent hears on CNN (from an expert, of course) that a teenager killed his classmates because of a game. So, of course they do the sensible thing and not allow their kids to play violent games. While whether this is a good decision is debatable, but my point here is they are basing a decision of how to raise their child on the statements of a person that CNN has labeled an “Expert”. Statements that seem to make “common sense” are rarely even questioned.
The other area this line of thought came from was actually from the other end of the spectrum. At my job, we are actually hired by clients to be experts. On my first client, since I had the most experience with fit and subversion, I was labeled the “fit expert” and the “svn expert”. One thing to keep in mind is the extent of my fit experience was a two hour class a month before.
Now, I’ve never considered myself as an expert in anything. I was good, but expert? Anyway… One of the things I learned from this experience is that when you are the expert, people came to you when they have problems. Now, starting out I really only knew a little more than everybody else. As the project moved on with everybody asking me questions that I dutifully looked up, I really did become an expert on fit. An interesting thing started to happen, though. By being steeped in the domain, I started to be able to make educated guesses about what the problem they were experiencing without know for sure. Most of the time I was either right, or right enough that it got them to a solution.
I think that this is the point where problems can start to happen.
ex·pert·i·tis (ĕk’spûrt’-ī’tĭs)
n.
1. Inflammation of an individual’s perceived value.
When you, as some sort of expert, get to the level where you understand your domain fundamentally, it becomes really easy to believe that you are the domain. What I mean by this is you feel that you are effectively infallible when it comes to your expertise. Effectively infallible because, while you know that you very well could be wrong, you know you are able to recover from any mistakes you make. It’s a powerful thing having the kind of confidence like that.
Unfortunately, this kind of confidence doesn’t translate well. Laypeople (by this I mean non-experts) rely on experts to help them get thru or understand situations they are not able to handle themselves. What they want is a 100% correct, actionable answer, right now. The reality is that usually they will only 2 of the 31. Either it is correct right now, but so general as to be not useful, or it is correct and actionable… after 5 years of research. The most common case, however, is an actionable answer right now that is more or less correct… -ish. Some experts will never give an answer like that. They, of course, don’t end up on CNN. I believe that most of those who do aren’t bad people, however. They aren’t just trying to get famous. They just don’t see anything wrong with giving an answer that they personally think has a 70% chance of being correct. The problem is laypeople don’t have the experience to be able to recover from those mistakes, so they either go to another expert, or swear off experts entirely (not easy to do nowadays…).
Another example of this situation is when you take a car in to the shop. The mechanic is an expert in automobiles. So, when you hear a clicking sound when you are driving, he can make some informed opinions about what the issue is. He might take those opinions and decide on a course of action. Say, replace a belt. He doesn’t think about warning you that he’s only 70% sure because most of the time, in that situation, he is right. So, if you find out later that you still have the problem, you get mad at him and take it to a competitor.
ex·per·tec·to·my (ĕk’spûr-t’ĕk’tə-mē)
n.
1. Surgical removal of… expert… something…
I think the important thing to take away from this is that Experts are people too (some of them at least). Rely on them when you need to, but keep in mind that they could be wrong.
Expert /= Infallible.
1. This is what I believe differentiates experts from other people. Experts usually get 2/3, non-experts get 1/3. Nobody get 3/3 consistently.
Permalink
04.11.07
Posted in General, Software Development at 3:53 pm by BestFriendChris
So, I just recently (about 3 hours ago) discovered the wonderful site Twitter.com. The gist of it is it allows you an easy (insanely so) way of letting people know what you’re doing. It’s pretty addictive. There are three things I especially like about this service.
- You have only 140 characters to type what you’re doing. This way not only do I not feel obligated to go on and on about what I’m doing, I’m actively forbidden to. Nice.
- You can send a text message from your phone to update what you’re doing. Perfect for when you’re out and about.
- You can set it up to remind you if you haven’t updated it in more than 24 hours (by text, natch). Should keep the updates from getting stale.
I searched around and found this post about how to get a list of the last 10 tweets on your website. Since it didn’t quite work for me, and I’m kinda anal when it comes to code, I fixed it up a bit. Essentially I fixed the bug in it that would cause it to fail if you have less than 10 tweets in your account and I removed the duplication of the count of tweets the javascript returns. Also, I added in a few extra “span”s for my css to make it all pretty.
You can find the code after the jump…
Read the rest of this entry »
Permalink
03.04.06
Posted in Software Development at 2:53 am by BestFriendChris
I just read an excellent article on the O’Reilly Ruby Blog, which talks about Martin Fowler’s book, Refactoring. While the article does speak on the issue of not having refactoring tools available for Ruby at this time, it really is about a deeper and more important message relating to refactoring in general. In short, while refactoring tools do help us fix the issues we create, many refactorings simply cannot be taken care of in an automated way. No matter what language you use, it’s more important to have an idea of why a class needs to be refactored and what to do about it than to have a keyboard command to automatically change a method name every place it’s used. I really recommend you read the article, though.
Permalink
11.30.05
Posted in Software Development at 12:00 pm by BestFriendChris
So, while wandering around the net last night, I stumbled upon Rent-a-Coder.com. While I had heard of this service before, this is the first time I had ever actually looked at the site. Initially, I thought, “Sweet! Now I can pick up some money doing odd coding jobs.” While I still may do that, I did find a more useful byproduct of the site for entry level developers, such as myself.
I was browsing around, looking at the jobs to bid on, when I found a simple request that I totally felt I could do. The task asked for somebody to write a VB(possibly .Net) app that would read in two files, compare the two, perform some logic, and then write the output to a third file. Seems simple enough, although VB will never do. Rather than bid on it, I decided to just implement it. If I finished before anybody won the bid, I’d bid on it (I didn’t), and if not, at least I’ll learn a thing or two about Ruby on the way (I did).
Ultimately, I think the best part is it gave me a unique problem to solve. I feel, in general, I have two issue when practicing coding. First of all, I tend to stick to problems that I’ve solved way too many times before (This means you, CyberPet). While it is useful to reimplement old designs when learning a new language, this can only take you so far. Second of all, when I’m not using toy examples, I have a tendancy to use problems that are too large to finish (my hardrive is cluttered with half-finished projects). I get frustrated with a problem, and set it aside. I usually do come back to these projects, with varying degrees of success, but it is still frustrating to not feel the sense of accomplishment at solving a problem. By looking throughout the cheap bids at Rent-a-Coder, I find fresh problems to tackle and stay sharp in Ruby!
Out.
Permalink
