Interviewing software developers

I've been doing some interviews recently for Jave EE developers. Its for a new team and we had four positions with no particular experience level in mind. Part of the interview was a programming test in which the candidate was asked to write the legendary 'FizzBuzz' program. However, I decided that to spice it up a little the candidate would also have to write some unit tests. I also suggested that they may wish to use a TDD approach, although this wasn't mandatory.

For those unfamiliar with FizzBuzz, the problem is described as such;

Write a program that prints the numbers from 1 to 100.
 - for numbers that are divisible by 3 print "Fizz" instead
 - for numbers that are divisible by 5 print "Buzz" instead
 - for numbers that are divisible by 3 and 5 print "FizzBuzz" instead
You may use a TDD approach if you want.

Results? A lot, lot worse than I thought. Some candidates had over 5 years Java development experience on their CVs and really struggled with this. Out of 7 candidates, only one failed to complete the task (7 years java experience) but the others took on average 45 minutes, and none of them finished with great solutions (their tests were flaky).

The most common mistake made was to start out right away with something like this;

public static void main(String[] args) {
  for (int i=1; i<=100; i++) {
    if (....)
     System.out.println("FizzBuzz");
   else if (....)
     you get the picture
  }
}

This solves the problem, but has a major downside. It is very hard to test. The program has no inputs and only outputs to System out. I would expect a competent programmer to come up with a solution in under 10 minutes. All but one took over 20 minutes. Once they started writing the unit tests they realised that they couldn't test their application, so they started refactoring it. Things generally went further downhill from here. There were some wierd and wonderful results, with programs looping 1 to 100 and filling an array with strings, returning a map of numbers to strings, returning lists, all kinds of stuff!

The one guy who aced the interview was a recent graduate and the only one to write the unit tests before the solution. He finished the whole thing in about 15 minutes - something which supposed senior developers with 7 years experience failed to do in an hour.

Here's what I'd consider a reference solution.

public class FizzBuzz {

  public static final String FIZZ = "Fizz";
  public static final String BUZZ = "Buzz";
  public static final String FIZZBUZZ = "FizzBuzz";


  public static void main(String[] args) {
    FizzBuzz fb = new FizzBuzz();
    fb.print();
  }
  
  public String getString(int i) {
    if (i % 15 == 0) return FIZZBUZZ;
    if (i % 5 == 0) return BUZZ;
    if (i % 3 == 0) return FIZZ;
    return String.valueOf(i);
  }


  public void print() {
    for (int i=1; i<=100; i++) {
      System.out.println(getString(i));
    }
  }
}

..and the tests;

public class FizzBuzzTest {

  private FizzBuzz underTest = new FizzBuzz();

  @Test
  public void testFizzBuzz() {
    for (int i=15; i<=100; i = i+15) {
      Assert.assertEquals(i + " is FizzBuzz", FizzBuzz.FIZZBUZZ, underTest.getString(i));
    }
  }


  @Test
  public void testFizz() {
    for (int i=3; i<=100; i = i+3) {
      if (i % 5 == 0) continue;
      Assert.assertEquals(i + " is Fizz", FizzBuzz.FIZZ, underTest.getString(i));
    }
  }


  @Test
  public void testBuzz() {
    for (int i=5; i<=100; i = i+5) {
      if (i % 3 == 0) continue;
      Assert.assertEquals(i + " is Buzz", FizzBuzz.BUZZ, underTest.getString(i));
    }
  }


}

I didn't really expect a unit test around the printing, just the logic for determining if a number should be Fizz, Buzz or FizzBuzz.

What did this tell me about the candidates?

I wasn't so much interested in whether they could do it or not; all the candidates claimed a fair amount of dev experience or to have studied computer science at University so I expected they could. The key was that they worked on a laptop which was plugged into a flat panel TV so I could watch them work. Watching how someone arrives at a solution tells you a wealth of information that you wouldn't know just by looking at the finished product. What mistakes do they make, how do they correct them, how do they refactor, how do they use the tools available in their dev environment? Sure, it ups the pressure with the interviewer watching your every key stroke, but the task isn't a complicated one and doesn't require a huge amount of thought.

Where did people go wrong?

• The most common mistake was to dive straight in and write something hard to test; a main method that looped and printed to system out. Most realised their mistake when they wrote tests (some required a little prodding in the right direction) and refactored.
• Quite a few candidates struggled with the default case of returning the number; counldn't understand why the compiler didn't like them returning an int when the method signature specified a String return type. Only one candidate figured out String.valueOf(i) straight away, the others who got it had to think about it for several minutes.
• Several candidates didn't follow the spec, even though it was clearly written; eg their method returned "3: Fizz" for the number 3.
• Two candidates solutions printed "Fizz" or "Buzz" for 15 because they checked divisibility of 3 or 5 first (rather than 15, or 3 and 5) and fell out of they if construct.
• One candidate (12 years experience in software dev, 8 in java) didn't write unit tests because he didn't know how to! 12 years and never wrote a unit test!
• Some candidates seriously over complicated the problem, using arrays, lists or maps to store all outcomes. This wouldn't have been too bad except they all utterly failed to test their overly complicated solutions.