Over the last two weeks, I have been working with Christopher Foo and Eldon Visitacion to design a console application using the WattDepot API. Throughout the duration of the project, in addition to the Subversion, Ant, Eclipse, and JUnit tools discussed in past entries, we began using a class Jenkins server as a continuous integration tool. Configuring Jenkins to automatically build and test the project after each repository change helped us to pinpoint which Subversion commits introduced faulty code into the system and which developers were responsible, making us more careful about verifying our code in the first place. We used Google Project Hosting to manage our repository and a small wiki.
Our work and school schedules prevented us from meeting in person very often, so we designated issues and set intermediate deadlines mostly through email. Each group member was responsible for posting and closing their own issues and notifying the member assigned to a certain part of the project of any bugs they found while examining or testing each others' code, and all group members were responsible for testing the final .jar executable.
The Hale Aloha CLI
The console application's purpose was to retrieve data from the WattDepot server for the University of Hawaii at Manoa's Hale Aloha freshman residential complex, consisting of four towers where energy consumption and power data were being logged for the Kukui Cup. It needed to display the current levels of power being used, the amount of energy consumed on a given day, the energy consumed since a given day, and the rankings in energy consumption of the four towers from a given start date to a given end date.The Command Line Interface
Our team's implementation does not use Java's Reflect class.
Current Power
>current-power [tower | lounge]
Daily Energy
>daily-energy [tower | lounge] [date]
Energy Since
>energy-since [tower | lounge] [date]
Rank Towers
>rank-towers [start] [end]
JUnit Tests
JUnit tests were incorporated into our system fairly early on. The first draft tests for EnergySince were running by November 12 (continuous integration build 15). The first tests for Control, CommandProcessor, and Quit were posted by November 15 (build 25). CurrentPower and DailyEnergy tests first appear on November 16 (build 27), a test for Help was posted on November 17 (build 38), and the first version of TestRankTowers was also committed on November 17 (build 39). JUnit tests had mostly reached their final states by Build 54 on November 24, and the system stayed at a consistent level of at least eighty percent instruction coverage afterwards.
Since parts of our command implementations were dependent on the time reported by the server, there were some pieces of my code - mostly those that converted time information to strings or to some other format based on the current date - which could not be tested. Despite this, I was able to test most of the methods used in my classes, especially the utility methods responsible for creating the XMLGregorianTimestamp objects used to retrieve energy consumption data. As I had chosen to make all of my setter and utility methods private, I had to learn to use Java's Reflect class to invoke them in tests. Oracle's tutorial and Lasse Koskela's exception testing examples were very useful here.
Another odd problem I had was using Reflection to invoke a method with a parameter that was a non-primitive-type array. If the parameter is just declared as foo[].class, Java interprets it as a varargs parameter instead of an array. In this case, I was trying to test the setter method for an array of XMLGregorianCalendar timestamps.
The correct way to do this, described at Your Mitra here, is to declare the array as a new class which contains the array:
Method testSetEnd = testClass.getDeclaredMethod("setEndTimestamps", new
Class[]{XMLGregorianCalendar[].class});
Then, in the method invocation, the array being passed as a parameter needs to be cast to a new class as well:
XMLGregorianCalendar sampleTimestamp =
Tstamp.makeTimestamp("2011-11-11T11:11:11.111-11:00");
XMLGregorianCalendar[] holdTimestamps = new XMLGregorianCalendar[]{sampleTimestamp,
sampleTimestamp};
// Cast the array to an object, otherwise it will be treated as varargs.
testSetEnd.invoke(testObject, new Object[]{holdTimestamps});
The large number of date-processing utility methods in my classes made it difficult to test all possible branches of the code, especially those which executed based on times retrieved from the WattDepot server. We also did not implement tests for the main method in the Control package, which we tested manually. JUnit’s policy of factoring in what percentage of test case code was executed greatly decreased the branch coverage score: the test which branched based on the time returned from the server executed only a few of their branches in the average unit testing. In the end, we achieved roughly 85% instruction coverage and 80% branch coverage.
Our software system is thoroughly documented and contains input-checking code that ensures that the right number of command-line arguments are given, dates do not overlap with each other or exceed the current date, and that source names match with an existing source's name, and most or all of these input checks are intentionally triggered in each class's Junit tests by passing bad input to the each Command's execute method. In addition, all of the Commands which take date parameters are capable of collecting data for the current day up to the most recent data reading from the requested source. Though it has its flaws, I consider it to be fairly high-quality in its error checking, operation, and test case coverage.
Server Issues
Though it had been difficult and at times annoying to get used to continuous integration practices, it was harder to coordinate system changes when the Jenkins server briefly crashed, or when it began automatically failing builds when the WattDepot server crashed and our Junit tests failed automatically after being unable to establish server connections. As the cliche goes, "You don't know what you got till it's gone." For all the annoyance of having a Jenkins build fail over some minor formatting error, it was still preferable to having that error remain in the code because the committer forgot to verify it beforehand.
In the end, Jenkins forced me to adopt better development practices by encouraging me to obsessively run Checkstyle, PMD, FindBugs, and JUnit tests - both in order to commit workable code, and to avoid having my usually careless errors broadcast to my team members. This makes continuous integration an effective tool for pinpointing which committers are responsible for errors and encouraging better practices (though maybe not quite as effective as a foam rocket to the head).
Debugging A Time Problem
The other server-related issue I experienced was in retrieving energy consumption data from the server at midnight. The WattDepotClient.getEnergyConsumed method retrieves data given a source name, two XMLGregorianCalendar objects representing the start and end times for data collection, and an integer representing the sampling interval in minutes. The original versions of my code had the interval set to 15 minutes (or 96 data points per 24-hour period). I noticed that JUnit tests would always fail mysteriously for current-day data requests made within about twenty minutes of midnight, throwing BadXMLExceptions which stated that my time range requested data beyond the available range of data on the server. Since my ending time was always based on the timestamp of each source's most recent sensor data on the server, my start time was hard-coded to midnight of the starting day, and I was in the same time zone as the server, I knew I was not experiencing synchronization issues between local time and the server. Further testing on November 24 revealed that the problem disappeared at about 12:17 AM, which was suspiciously close to my sampling interval. When I reported my problem to the class mailing list, Professor Johnson confirmed that the bug was the result of my sampling interval exceeding the range of available data. The solution was to set the sampling interval to 0 minutes, which allowed it to make data requests even when the starting time and the ending time were exactly the same.
Conclusion
Learning about the proper use of the tools of automated quality assurance and continuous integration has helped me reduce the number of syntactical errors I make in the first place, while making sure that most of my worst development mistakes are eliminated by automated quality assurance before entering the repository. Having team members with whom communication was easy made it faster and less confrontational to diagnose issues and suggest solutions, and enabled our group to proceed at a steady rate. Incorporating unit testing into our builds early on helped us get test coverage up to a reasonably high level by the end of the project, even if it was not possible to test some parts of the code. Ultimately, the combination of good time management and industry-standard quality-assurance tools helped us work around server crashes, data-request bugs, and test coverage problems and see the project to its completion. A good team can compensate for occasionally dysfunctional technology, but even good technology usually cannot save a project from a truly dysfunctional team.
