Training Registration System

At the University where I work, there’s always a ton of Professional Development activity going on around campus, most of which is centered on training sessions for which people can register. That has always been an arduous, labor-intensive process, with real live humans handling every aspect of managing those registrations — reporting on class sizes, processing registrations, processing cancellations, maintaining a waiting list, communicating with attendees, etc.

To ease that chore, I’ve started work on a piece of software that manages all of those common tasks, driven by the requirements of our training organizations. So far, it allows a user to register for a class, cancel a reservation, subscribe to a webcal calendar that shows the training for which she has registered, download an ICS file to put the event on an existing calendar, receive email confirmations of registrations, to sign up for the waiting list for classes that are full, and to receive email notification when they get moved from the waiting list to the class roster. It’s also the most rigorously test-driven development I’ve done to date, so the code should be of good quality.

I’m managing it as an open-source project, so if you’re comfortable with Ruby on Rails and are interested in jumping in, or would like to download a copy to fool around with, you can visit the project on Google Code. I’d be delighted to have other contributers if it turns out to scratch anybody else’s itch as well. It is still very much geared toward folks who know a little bit about Rails and are willing to customize it to their needs. If that’s not you, you might want to steer clear for now.

Texas State iPhone App Released

At long last, the official Texas State University iPhone App is released!

The team brainstormed and prototyped the original version as a learning exercise at the beginning of the year, but once Marketing got wind of it, it quickly became a high-priority project. I’m really proud of my crew, who have all stepped up and contributed design ideas and code to the final product, which turned out really well.

We have some ideas for improving it going forward, and have plans to create an Android version as well, but are, for now, just delighted to finally have it out there in the world!

More Good Thoughts on Games and Work

I wrote a piece about a year ago called Workplace Motivation and Game Mechanics which summed up some of my ideas for bringing some of the compelling character of video games to the office. Since then, some much smarter people have been doing some thinking and experimenting with similar ideas.

Jane McGonigal gave a terrific talk at TED called “Gaming Can Make a Better World”. It’s well worth the 20 minutes it takes to watch:

Additionally, I’ve started reading Total Engagement: Using Games and Virtual Worlds to Change the Way People Work and Businesses Compete. It covers the authors’ experiences in this area, starting with poolside conversations and ending up with forming a company around these ideas. It’s a good read so far.

Based on these, it looks like the invasion of games into the workplace has the potential to create some important productivity gains and is already underway. It should be fun to see what the next decade brings.

On Software Testing

Over the past several years at the University, we’ve had to hire for a variety of programming and programming-related positions. One of the interviewing practices we’ve adopted is to give candidates a programming assignment which they can complete at home, at their own pace using any resources they can muster. Because these conditions approximate the conditions under which they would normally be working, we find this to be a really good indicator or what sort of work they’ll do if hired. (Besides, it’s one of the practices from The Joel Test! We currently score about 8.5/12.)

One of the jobs we’ve been hiring for is what I refer to as the “Quality Assurance Czar” — someone who will spearhead and organize our efforts to make sure the code we’re fielding is as awesome as we want it to be. The programming test for this includes both writing a small program to compare two hands in poker and, more importantly, writing a suite of tests to verify that the program works correctly. (This is not the exact programming challenge we use. It was actually one we considered, but it’s a thornier problem than one would guess at first glance, and there are a number of published algorithms out there already. We decided to make up our own problem so that applicants couldn’t just google for an answer.)

A few months back, one of my good friends — let’s call him Larry — interviewed for this position. He did a fine job in the interview, and while his code was very good, it didn’t include the comprehensive testing we were looking for. He wrote a small PHP application that generated a couple of random hands and then compared them, relying on the person running the program to verify that the winning hand the program chose was correct. His solution was 100% accurate, but didn’t include any automated tests, as he had a hard time imagining what sort of testing would have been useful in that case. Here’s a note he sent after the position had been filled:

I was real proud of my [Poker] app, the one that was supposed to include a testing approach.  Now I’ve tried Selenium, and I still don’t see how to test my app, because it has no user inputs.

What did you guys ever have in mind?

Okay, I can see these tests:

  • entering the URL brings up the page.
  • A refresh causes a change to the page (a new checkers arrangement).

But that don’t seem adequate.

He is, of course, completely correct. While this would test a few trivial aspects of the application’s functioning, it would leave the important and juicy bit of code, analyzing the poker hands, completely untested. (This is called “incomplete code coverage” in the argot.)

There are several things one can do to make this application more testable. First off, one needs to create a mechanism to feed the program a specific input. Once you can do that, you can give it a couple of predetermined hands and verify that it chooses the correct one. Doing this with a variety of different hands will demonstrate that the program is working correctly in those cases.

There are, however, about 2,400,000,000,000,000,000 combinations of two 5 card hands. For obvious reasons, we can’t test all of those. How do we choose which ones to check?

Let’s start with easy ones. Verify that a flush beats a pair. Verify that a full house beats a flush. Verify that four of a kind beats three of a kind. Verify that a pair of 8s with hearts and spades beats a pair of 8s with clubs and diamonds.

Once you have the basics covered, we want to look for “edge cases”. These are where our data bumps up against boundaries of some kind. They’re very important because “off by one” errors are very common in programming, and these tests help to expose those errors. In poker, for example, we would want to check that an “A2345” hand would count as a straight, and that a “10JQKA” would also count as a straight, but that “QKA23” would not.

By this point, we’ve accumulated perhaps a few dozen tests — a miniscule percentage of the possible total, but strategically enough chosen that we have confidence that things are working as they should be. If we later discover that we’ve missed something and that our program is inaccurate in some cases, we simply add another test that demonstrates the failure, and then work on our algorithm until all the tests are once again succeeding.

Next, it’s important to try invalid data and make sure that our program handles the error condition as we would expect. Asking the program to evaluate a hand with 5 aces of spades is meaningless, and should generate an appropriate message for the user. (Something like “You’re a dirty cheating dog” might be appropriate.) Having the 17 of acorns as one of the cards in your hand is similarly out of bounds, and should also trigger an appropriate error.

In Larry’s program, all of the logic was stored in a single PHP file. Though it’s a bit more work to set up, I would recommend using a Model-View-Controller design pattern, and splitting the “evaluate the poker hand” portion of the program from the “show the poker hand on screen” portion and the “handle user input” section. By doing so, we can test these portions of the code independently. If we know that the code that evaluates the hand does so correctly, but it’s showing the wrong answer on the screen, we have immediately narrowed the code that might contain the bug to just the bit that’s responsible for presentation.

Finally, once we’re using MVC, it becomes practical to use one of the many excellent unit testing frameworks that encapsulate a lot of hard-won experience doing effective testing. Since Larry was working in PHP, PHPUnit and SimpleTest appear to be decent options. Using an established testing framework has a number of advantages: the code for common testing tasks (like logging into a site) will already be written, it becomes easier to integrate our tests with other systems, test output can be reported in succinct, attractive forms, and other programmers will have an easier time understanding how our tests work.

So, in summary, we’ve updated our program to accept input, fed it a number of test cases and verified that it’s returning the results we expect, adopted MVC and a standard test framework. We now have both a high degree of confidence that our current code is working as expected and a “safety net”: if we decide to refactor the program or change the algorithm we’re using to compare hands, we’ll know immediately if anything we’ve done has broken it.

For a more comprehensive introduction to testing and using it to drive development, see this article. And if you’ve followed this, want to learn more, and are interested in a job, our QA Czar position is open once again!

High Ed Web 2008 Talk (or “A Cure for Insomnia”)

Last October, a couple of my coworkers and I presented at [High Ed Web 2008]. The conference organizers have, at long last, posted the transcript and audio recording of our session. The quality of the transcript is fairly rough, and they didn’t include the visual aids (which, by the way, I put a good deal of time into converting to a format they’d asked for), but the audio quality is good. If you have an interest in our experiences with content management systems at Texas State University, give it a listen!

Planning Poker

One of the things about writing software for a living is that customers are rarely happy when you tell them “It will be done when it’s done.” Even though development is fraught with uncertainty, it’s still incumbent upon us who practice this craft to be able to give people some idea of what kind of effort will be involved to implement a particular feature.

One way to do this is to have a project manager or senior programmer sit down, think really hard about it, and come up with numbers. This has the advantage of being  expedient, but the disadvantage of being almost certainly wrong. The reasons for this are many: it’s hard to estimate a job accurately when you’re not the person doing the work. All of us as individuals possess blind spots. And programmers are notoriously optimistic about how hard it will be to do something. (Not as optimistic as marketing people are when faced with a programming task, but still…)

One of the approaches I’ve seen that seems to promise the best results over time is Evidence-Based Scheduling. By having programmers estimate the tickets they’ll be working on, then record how much time they actually spend on it, comparing one to the other, factoring in how much productive time they have during the day once meetings, email, and the ilk are all done, and doing a lot of math, one can come up with good quality estimations — especially as one accumulates data on which programmers tend to exaggerate and which think they can finish nearly anything before lunch today. This approach has the advantage of producing really good estimates, but requires detailed tracking both of estimates and actual work time to produce good results.

Somewhere in the middle is Planning Poker, an approach that allows teams to collaboratively create estimates with a minimum of fuss. The basic idea is that you get all your developers together, give each of them a deck of cards with a time estimates written on each card, and have everyone choose how long they think a given task will take by throwing the corresponding card on the table. If there’s not agreement, everyone discusses it, explains their reasoning, and then you do it again until there’s a consensus. (You can read more details here if you’re interested.)

When we’re planning what to do for the next phase of a project, we have to consider two things: how important an issue is, and how much work it will be to do the fix. The latter has, up to now, been done off the cuff by whomever of the developers happened to be in the room. But in the past week, we’ve added a field to our ticketing system for estimates and started doing Planning Poker sessions to capture estimates for the tasks ahead. These seem to be going well, and I’ve noticed some interesting things about them:

  • These sessions are actually pretty fun. Nobody has come away hating the process, and several of the devs involved have said they’ve really enjoyed them.
  • Because the rules of the game call for the people with the highest and lowest guesses to explain their reasoning, we hear from people who are sometimes a bit more reluctant to speak. Having their opinions respectfully listened to can be a very affirming, especially for new folks who may not have a history with the team yet.
  • We’ve actually been doing on-the-fly technical design as we discuss the issues, and have come up with some really good ideas and insights that we’ve then recorded in the ticket since we already had it open there. This was a pleasant surprise to me; I’d only been hoping to get estimates, but have also benefited from some terrific collaborative technical brainstorming.
  • It provides natural cross-training, since even programmers who aren’t involved in a particular project get to listen, ask questions, and vote on its tickets.

So after two sessions, I’m counting Planning Poker as a success, and have scheduled regular sessions to finish estimating our backlog of tickets and to handle new ones as they show up.  Many thanks to Mike Cohn for the great writeup that got me excited about it!

Workplace Motivation and Game Mechanics

I manage a crew of about 5 programmers at work. During our performance reviews this year, I asked each of them how I could do my job as a manager better. One of them, in a bit of much-appreciated candor, said that motivation was one of his biggest challenges, and that he would be grateful for anything I could do to help.

This guy — let’s call him Ivan — is a capable programmer, and does good work, but (like all of us) sometimes has a tough time overcoming inertia and focusing on the task at hand. Interestingly, Ivan is also a World of Warcraft player. (World of Warcraft is an online Role Playing Game where hundreds of thousands of people play together in a shared, computer-based fantasy world.) One of the interesting characteristics of  online RPG’s like World of Warcraft is that they often require one to do a lot of repetetive,  mundane tasks to gain in-game experience and thereby increase your character’s prowess. This is referred to as “level grinding”.

When one is “grinding,” one isn’t doing it for the satisfaction of the task being performed, but for an extrinsic reward of some kind that keeps players at it. In the case of online RPG’s, the reward is an improved character (with a higher level or improved skills), better equipment, the chance to participate in more challenging gameplay, or bragging rights among one’s game-playing peers. Ultima Online, the RPG on which I used to work, would allow the player to link to a web page that showed off her character with her honorifics and arrayed in all her finery.

This sense of accomplishment is one of the things that makes computer games so compelling for many people. In the early days of arcades, the chance to put one’s initials on the High Score list on Tempest was often as big a draw as the gameplay itself. We liked the sense of competition and achievement that list of scores brought with it. More recently, Microsoft did something very clever when they introduced the XBox 360: every game for the system has “Accomplishments” that earn “Gamer Points”. People who play for fun will inevitably rack up a certain number of these, but in order to maximize one’s Gamer Score, one has to again do things that aren’t inherently enjoyable: search out every pigeon in a vast virtual city (and shoot them all), get through a level without firing a gun once, or breed 200 worm-shaped piñatas. One can then put a “Gamer Badge” on one’s website or on Facebook to show the world what you have accomplished as an XBox player. This system has been so successful that Sony has recently copied it for the Playstation 3, introducing a “Trophy” system to the platform with their most recent software update.

So as I looked at all these different mechanisms that games use to take gameplay that might otherwise be uninteresting and make it appealing, I wondered if something similar could be applied to work. After all, try as we might to make the office interesting and enjoyable place to be, the fact remains: if we have to pay somebody to do something, it’s probably not inherently much fun. (There are a lot more musicians who will perform gratis than there are accountants who will do your taxes for no charge.) As I mulled this over, several possible systems came to mind:

  1. Keeping Score: the simplest approach is that of the arcade games of yore: you get points when you do something good, and try to get the highest score within a given period, say a week or a month. (“Something good” for work purposes might be completing one’s timesheet, successfully fixing a bug, dealing with a support call, or presenting at a conference. Each would have a different point value based on its value to the business and degree of difficulty.) This system has the advantage that it provides pretty quick feedback, and that new folks and people who have worked at a job for a while are on a fairly level playing field. The biggest failing of this approach is that a single score doesn’t tell anything about what things one is doing to rack up the points, so one has less motivation to expand one’s skills than to simply concentrate on areas where one already has facility.
  2. Classes & Levels: As with the Keeping Score approach, one is awarded “Experience Points” for doing something good. These points are tallied and, when one has accumulated enough, redeemed for “levels” in various classes. For example, after completing a performance review and a month of development work, I might be a “Programmer Level 3; Bureaucrat Level 2”. Levels generally become geometrically more difficult to attain as they get higher, so a new employee would “Level Up” more often than someone who had been working at a given job for a long while. This provides a more nuanced view of what sort of work someone is doing and maps more directly to a traditional RPG mechanic.
  3. Achievements: With this approach, one has a list of specific tasks to accomplish. For programmers, this might include things like “fix your first bug”, “check in 10 changes to our source control system”, or “get a patch accepted by one of the open-source projects we use.” One accumulates achievements over time, and can see a list of which ones have been accomplished, and which ones are still out there waiting for further efforts. (The requirements for a given achievement can also be kept secret until it’s unlocked.) The advantage with this approach is that one is encouraged to stretch one’s wings and try new sorts of tasks. Unfortunately, it lacks any kind of aggregate view of one’s progress.

At this point, I’m thinking that a hybrid of #1 and #3 might be the best for an office. Each person would have an overall score — perhaps cumulative, or perhaps reset monthly — which would be increased accordingly whenever one did something constructive. In addition, one could unlock achievements by doing specific things. Those achivements would then go on one’s record, and would also provide an extra boost to one’s score.

Whatever system we used would need to have a few specific features:

  1. As much of it as possible would have to be automatic. Nobody wants to spend all day tallying scores, so it would be important to tie much of this to our existing systems and have them add points without human involvement. Some things, like points for helping a coworker, will inevitably require human tallying, but these should ideally be the exception.
  2. Part of the fun of accomplishment is being able to show off what you’ve done. We would need to be able to publish the information on scores and accomplishments various ways: website badges, added to our Twitter Status Board, screen savers, etc.

I’d really like to hear from people on what they think of this idea. Is it something you would enjoy having around your office? Do you think it would help with motivation? Do you think any particular mechanic would be a more effective motivator than another? I’m still brainstorming on all of this at the moment, so would welcome any diverse input folks might have on the subject. Thanks for anything you can contribute!

UPDATE: As I’ve been giving this more thought, another mechanic came to mind that I think would be worthwhile. In the game Team Fortress 2, each player is given a score for the match. The points, however, are earned in various categories, and those categories tracked individually. This results in a lot more affirmation, as even if one had a relatively poor game, it might still be your best performance as a sniper, or the most points you’ve earned by healing other players. Being able to lump work points into various categories could easily result in similar kinds of encouragement when they are tallied: “You earned 625 points this month, which is lower than your personal monthly best of 1027. However, you earned 400 points for helping coworkers, eclipsing your previous best of 150 points!”

Twitter Status Board

Several months ago, I noticed that one of the nearby offices was using a whiteboard mounted outside their office door to keep people informed of their comings and goings. “That’s good communication!” I thought to myself, “but my team is made up of geeks. Surely we can do something nerdier!”

Thus was born an experiment with Twitter that is yielding some good results. It began with our six-person development team: programmers who were interested in participating set up a twitter account and began posting what they were up to through the day. We subscribed to each other’s updates and easily keep a finger on the pulse of the team’s activity. (This was especially helpful when working off-site or stuck in meetings.) In addition to the standard Twitter functionality, we also wrote a little Rails app that uses Twitter’s APIs to build a replacement for the old-style In-Out board. People can pull up the status screen from a web browser or look at a monitor we stuck in the window of one of our cubicles to see at a glance who was around and what they were up to:

Status Board In Situ

Status Board In Situ

After a month of voluntary participation, we had a meeting to discuss whether it was worth keeping up, and decided that the minimal effort required to improve our communication was well-spent. We’ve been using the system regularly for several months now. (One interesting side-effect that I’ve noticed is that it helps me to keep on task a little better: when I post that I’m working on ticket #726, I feel a little guilty if I’m not actually working on it!)

Of course, people walking by quickly noticed the screen and asked whether it could be expanded to include other people as well. At my boss’ requenst, we put together another status screen for use at the department’s front desk, where the people there can quickly ascertain whether a call can be transferred or a visitor directed back to someone’s office. It looks like this (with strategic blurring to protect the unwitting):

Front Desk Status

Front Desk Status

The color-coding is done by preceding a status update with “IN:”, “OUT:”, or “OFF:” (for those times you’re working but not at your desk), a feature of our status board app. While some of the folks around the department were initially a bit skeptical, they have quickly become enthusiastic once they use it for a day or two. The front-desk folks decided that they’d rather have timestamps than the “x hours ago” notation we’d originally used, so we tweaked the app to better meet their needs.

We’re still refining our software (I need to make the caching smarter), there are at least two other Status Board applications under development in the department (using Javascript and FLEX), and I now quickly hear about it if something goes wrong with the system. It should be interesting to see where this thing goes over the next few months! (If there’s interest, I can clean up the Rails code and open-source it once I finish the caching improvements.)

I’m a Mythical Beast

And it’s not just my wife who says so.

When I’m going to take a day off from work, I email our team to let them know I’ll be out. Here’s a recent missive:

Hey y’all,

I’m planning to take Friday off as a sanity day and to strip down, don the
headdress of the fabled jackelope, and beat a drum in a sweat lodge in the
woods for a while while shuffling around the smoky fire in ritual native
american dance.

Or maybe just enjoy a book and some tacos. Either way.

Laura, one of our awesome graphic designers, responded with this:

May the spirits guide you on your journey

Sean as Jackelope

Sean as Jackelope

Truly, I work with some amazing people.

High Ed Web 2008

I’m currently at the High Ed Web 2008 conference in Springfield, MO. The conference is geared to web professionals in Higher Education, and is an interesting combination of marketers, technical folk, and the occasional vendor. There’s lots of good information to be had (as well as an immense amount of equally good food — who suspected that “travel is broadening” should be taken literally?), and all of us who are attending are getting a lot out of it.

Yesterday afternoon, James, Jeff and I did our planned presentation: University-Wide CMS Implementation: Failure, then Success. We had a lot of fun with it, taking a pretty irreverent approach to the history of our CMS project, and using some cool 3D timeline software to present our information in an interesting way.

Conference attendees have been using Twitter to communicate about the conference pretty extensively, so I told people at the start of our presentation that I wanted them to help me make all the people in other sessions feel as bad as possible for missing ours. Toward that end, I encouraged them to post the most outrageous lies about our presentation they could think of. Here’s the Twitter chatter that was posted during our presentation:

  • jesseclark: I have recieved more temp tattos in the past two days than the last 10 years! #heweb08
  • tonydunn: #heweb08 live mexican wrestling in SAC4 session! AWESOME!!
  • stomer: Admiring the wrestling masks at SAC4 #heweb08
  • carlenek: #heweb08 SAC4, CMS Failure then success. They just gave each of us $5!!
  • stomer: In unison “politics” #heweb08
  • stomer: “Uploading images has more steps than AA” #heweb08
  • tonydunn: #heweb08 massive EPIC FAIL… what a story! if you ain’t in SAC4 ur missing it
  • tonydunn: @pberry blackops meetings… sounds like us 🙂 #heweb08
  • stomer: Another plug for adding content best practices along with CMS training #heweb08
  • tonydunn: ‘brand the service, not the product’ – excellent advice! #heweb08

I’m having a great time getting to enjoy some of my work friends in a more social setting than usual, as well as meeting people from other Universities and Colleges around the country, getting to see Springfield, and taking photos with team members in luchador masks in front of local landmarks. We have another day and a half of conference to go, after which I’m looking forward to the chance to meet up with my Uncle Rick and some old friends who also live in the area.

Sidenote: before leaving for the trip, I told the kids that I was going to Springfield for a conference. “Cool!” said Liam. “You’ll get to meet the Simpsons!” I launched into an explanation of the fact that one of the running gags on the show is that they don’t ever let on where Springfield actually is, and how in the movie Ned Flanders points to the four adjacent states, Ohio, Nevada, Maine and Kentucky, which are of couse widely spread. Thus I wouldn’t actually get to meet the Simpsons, because the Springfield they live in is a fictional construct, not the one in Missouri.

Ten minutes later it occurred to me that I would also not get to meet them because they’re cartoons.

Yep, I’m a dork.