Basics - XML

Here's the next entry in my Java basics series. Earlier we looked at how to read a text file. Now we'll look at some simple XML processing. XML is pretty ubiquitous; at some point you're going to run in to it. This post will show how to read XML and it parse into a Document. We'll extract data from the Document and then create a new one.

There are many XML libraries around that help with XML processing, such as jdom or dom4j, but the standard JDK comes with everything you need. Lets get started!

Getting a Document

The first step in processing XML is to create a Document from the raw XML. The Document is the central class for XML processing. We use a DocumentBuilder to parse XML into a Document. The builder has several overloaded parse methods allowing you to create a Document from a File, InputStream, InputSource or a URI. Here we are using a file.

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
File file = new File("employees.xml");
Document doc = builder.parse(file);

In this example we're going to work with a basic XML file, employees;

<employees>
  <employee id="1">
    <name>Fred</name>
    <age>20</age>
    <department>Sales</department>
  </employee>
  <employee id="2">
    <name>Bob</name>
    <age>30</age>
    <department>Sales</department>
  </employee>
  <employee id="3">
    <name>Jim</name>
    <age>23</age>
    <department>Marketing</department>
  </employee>
</employees>

Working with the Document

Now we have our Document, we can extract data from it using XPath. XPath is a powerful query language for selecting nodes from an XML Document. Lets find the name of the employee with an ID of 1.

XPathFactory xpFactory = XPathFactory.newInstance();
XPath xpath = xpFactory.newXPath();
String qry = "/employees/employee[@id = '1']/name";
String name = (String)xpath.evaluate(qry, doc, XPathConstants.STRING);
System.out.println(name);

When this code is run, it will output Fred, which is the name of the employee with an id attribute of '1'. Now lets get all the employee nodes in the sales department.

xpath.reset();
qry = "/employees/employee[department = 'Sales']";
NodeList employees = (NodeList)xpath.evaluate(qry, doc, XPathConstants.NODESET);
System.out.println("Employees in Sales department;");
for (int i=0; i<employees.getLength(); i++) {
  Element employee = (Element)employees.item(i);
  name = employee.getElementsByTagName("name").item(0).getTextContent();
  String age = employee.getElementsByTagName("age").item(0).getTextContent();
  String id = employee.getAttribute("id");
  System.out.format("%2$s - %1$s age %3$s\n", name, id, age);
}

In the above example we get a NodeList and loop over it to create a report of the employees in the Sales department;

Employees in Sales department;
1 - Fred age 20
2 - Bob age 30

Creating a new Document

Now lets look at creating a new Document from scratch. We use a DocumentBuilder to create a new empty Document, then we create some elements and build up the DOM tree.

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.newDocument();

Element order = doc.createElement("order");
order.setAttribute("number", "1");
doc.appendChild(order);

Element total = doc.createElement("total");
total.setTextContent("10.00");
order.appendChild(total);

Element status = doc.createElement("status");
status.setTextContent("dispatched");
order.appendChild(status);

Now we have a small order Document! In order to see what a Document looks like we can output the Document to the console using a Transformer.

TransformerFactory tFactory = TransformerFactory.newInstance();
Transformer transformer = tFactory.newTransformer();
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
DOMSource source = new DOMSource(order);
StreamResult result = new StreamResult(System.out);
transformer.transform(source, result);

which gives us the output;

<?xml version="1.0" encoding="UTF-8"?>
<order number="1">
<total>10.00</total>
<status>dispatched</status>
</order>

Basics - reading a file

I quite often see new developers struggling with the basics in Java. I guess everyone has to begin somewhere, so here's the first in a series of posts demonstrating some of the basics the you might need when you're starting out.

This post deals with reading a text file line by line. First of all, lets create a text file called names.txt;

Bob
Fred
Jim

Now were going to write a class to read the file. This revolves around the BufferedReader class.

package querky.blog.basics;

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;

public class ReadAFile {

  public static void main(String[] args) {
 
    String fileName = "names.txt";
    BufferedReader reader = null;
 
    try {
      System.out.println("Reading " + fileName);
      reader = new BufferedReader(new FileReader(fileName));
      String line = null;
      while ((line = reader.readLine()) != null) {
        System.out.println(line);
      }
    } catch (IOException ioe) {
      ioe.printStackTrace();
    } finally {
      if (reader != null) {
        System.out.println("Closing " + fileName);
        try {
          reader.close();
        } catch (IOException ioe) {
          ioe.printStackTrace();
        }
      }
    }
  }
}

You can see that we create a BufferedReader and then call it's readLine() method in a loop until it returns null. This gives us each line in the file in turn. Notice that we also close the reader in a finally block. This is important as we want to release the resources used.

When run, the program should output this;

Reading names.txt
Bob
Fred
Jim
Closing names.txt

Common problems

The only problem you're likely to encounter is a FileNotFoundException - that is, your program can't find the input file it is supposed to be reading. The error will look something like this;

Reading names.txt
java.io.FileNotFoundException: names.txt (The system cannot find the file specified)
   at java.io.FileInputStream.open(Native Method)
   at java.io.FileInputStream.<init>(FileInputStream.java:106)
   at java.io.FileInputStream.<init>(FileInputStream.java:66)
   at java.io.FileReader.<init>(FileReader.java:41)
   at querky.blog.basics.ReadAFile.main(ReadAFile.java:16)

First, check you have a names.txt file in the right location. What's the right location? As the file name is specified as simply;

String fileName = "names.txt";

Then our program will look for the file in the current working directory, that is to say the directory where the program was launched from. If you don't know what that is, then you can easily find out with a debugging statement;

System.out.println(new File(fileName).getCanonicalPath());

Here we're using the getCanonicalPath() method to show exactly where on the file system Java is looking. If you're getting a FileNotFoundException odds are that the file isn't there or Java doesn't have permission to read it.

Interviewing software developers

I've been doing some interviews recently for Jave EE developers. Its for a new team and we had four positions with no particular experience level in mind. Part of the interview was a programming test in which the candidate was asked to write the legendary 'FizzBuzz' program. However, I decided that to spice it up a little the candidate would also have to write some unit tests. I also suggested that they may wish to use a TDD approach, although this wasn't mandatory.

For those unfamiliar with FizzBuzz, the problem is described as such;

Write a program that prints the numbers from 1 to 100.
 - for numbers that are divisible by 3 print "Fizz" instead
 - for numbers that are divisible by 5 print "Buzz" instead
 - for numbers that are divisible by 3 and 5 print "FizzBuzz" instead
You may use a TDD approach if you want.

Results? A lot, lot worse than I thought. Some candidates had over 5 years Java development experience on their CVs and really struggled with this. Out of 7 candidates, only one failed to complete the task (7 years java experience) but the others took on average 45 minutes, and none of them finished with great solutions (their tests were flaky).

The most common mistake made was to start out right away with something like this;

public static void main(String[] args) {
  for (int i=1; i<=100; i++) {
    if (....)
     System.out.println("FizzBuzz");
   else if (....)
     you get the picture
  }
}

This solves the problem, but has a major downside. It is very hard to test. The program has no inputs and only outputs to System out. I would expect a competent programmer to come up with a solution in under 10 minutes. All but one took over 20 minutes. Once they started writing the unit tests they realised that they couldn't test their application, so they started refactoring it. Things generally went further downhill from here. There were some wierd and wonderful results, with programs looping 1 to 100 and filling an array with strings, returning a map of numbers to strings, returning lists, all kinds of stuff!

The one guy who aced the interview was a recent graduate and the only one to write the unit tests before the solution. He finished the whole thing in about 15 minutes - something which supposed senior developers with 7 years experience failed to do in an hour.

Here's what I'd consider a reference solution.

public class FizzBuzz {

  public static final String FIZZ = "Fizz";
  public static final String BUZZ = "Buzz";
  public static final String FIZZBUZZ = "FizzBuzz";


  public static void main(String[] args) {
    FizzBuzz fb = new FizzBuzz();
    fb.print();
  }
  
  public String getString(int i) {
    if (i % 15 == 0) return FIZZBUZZ;
    if (i % 5 == 0) return BUZZ;
    if (i % 3 == 0) return FIZZ;
    return String.valueOf(i);
  }


  public void print() {
    for (int i=1; i<=100; i++) {
      System.out.println(getString(i));
    }
  }
}

..and the tests;

public class FizzBuzzTest {

  private FizzBuzz underTest = new FizzBuzz();

  @Test
  public void testFizzBuzz() {
    for (int i=15; i<=100; i = i+15) {
      Assert.assertEquals(i + " is FizzBuzz", FizzBuzz.FIZZBUZZ, underTest.getString(i));
    }
  }


  @Test
  public void testFizz() {
    for (int i=3; i<=100; i = i+3) {
      if (i % 5 == 0) continue;
      Assert.assertEquals(i + " is Fizz", FizzBuzz.FIZZ, underTest.getString(i));
    }
  }


  @Test
  public void testBuzz() {
    for (int i=5; i<=100; i = i+5) {
      if (i % 3 == 0) continue;
      Assert.assertEquals(i + " is Buzz", FizzBuzz.BUZZ, underTest.getString(i));
    }
  }


}

I didn't really expect a unit test around the printing, just the logic for determining if a number should be Fizz, Buzz or FizzBuzz.

What did this tell me about the candidates?

I wasn't so much interested in whether they could do it or not; all the candidates claimed a fair amount of dev experience or to have studied computer science at University so I expected they could. The key was that they worked on a laptop which was plugged into a flat panel TV so I could watch them work. Watching how someone arrives at a solution tells you a wealth of information that you wouldn't know just by looking at the finished product. What mistakes do they make, how do they correct them, how do they refactor, how do they use the tools available in their dev environment? Sure, it ups the pressure with the interviewer watching your every key stroke, but the task isn't a complicated one and doesn't require a huge amount of thought.

Where did people go wrong?

• The most common mistake was to dive straight in and write something hard to test; a main method that looped and printed to system out. Most realised their mistake when they wrote tests (some required a little prodding in the right direction) and refactored.
• Quite a few candidates struggled with the default case of returning the number; counldn't understand why the compiler didn't like them returning an int when the method signature specified a String return type. Only one candidate figured out String.valueOf(i) straight away, the others who got it had to think about it for several minutes.
• Several candidates didn't follow the spec, even though it was clearly written; eg their method returned "3: Fizz" for the number 3.
• Two candidates solutions printed "Fizz" or "Buzz" for 15 because they checked divisibility of 3 or 5 first (rather than 15, or 3 and 5) and fell out of they if construct.
• One candidate (12 years experience in software dev, 8 in java) didn't write unit tests because he didn't know how to! 12 years and never wrote a unit test!
• Some candidates seriously over complicated the problem, using arrays, lists or maps to store all outcomes. This wouldn't have been too bad except they all utterly failed to test their overly complicated solutions.