An Agile Pace
You’ve seen those job listings: the ones that say, “Must be able to work additional hours, beyond 8 per day, as necessary, to complete tasks required or meet deadlines.” If it’s an ostensibly Agile job such as Scrum Master, though — is this a bad sign? Does Agile’s notion of “sustainable pace” mean it's inappropriate to work more than 40 hours a week?
Anyone who has been in software long enough recognizes an ongoing need for overtime as a “process smell,” a sign of an organization needing rescue. Once there is a mandate for exceptional, regular, weekly overtime — officially instituting a “death march” project — it is past time to pull in an Agile process consultant.
A company may see frequent overtime as an expression of devotion and dedication, but Agile consultants know it is a destructive practice. It is more humane and productive to use prioritization, Work-In-Progress (WIP) limits, planning, velocity tracking, and iterations to manage throughput. Agile and Extreme Programming (XP) technical practices can make teams many times more productive even while creating a higher level of quality. When practiced correctly, Agile makes overtime unnecessary.
Constant overtime is harmful. According to author and famed software developer Robert Martin, “After the first day or two, every hour of overtime requires another hour of straight time to clean up after.” This is what Evan Robinson's IGDA paper told us six years ago (and Yourdon’s Death March well before that), but the naive appeal of overtime continues unabated.
Should an Agile team ever work overtime? In the steady state, as a matter of course or mandate, no. The team should choose the work week duration that is sustainable, meaning that they can tend to themselves and their families and careers in addition to doing the work their employer or client has for them. Agile requires whole people; and whole lives are not compatible with spending all waking hours in the office.
The official XP explanation of sustainable pace does not allow for much wiggle room. The practice calls for a “sustainable, measurable, predictable” pace, with warnings that “Incomplete or buggy software represents an unknown amount of future effort, so you can't measure it.” Tired people make mistakes, cut corners, and have trouble maintaining focus, resulting in code which makes multiple round-trips to QA before and potentially after release.
In a May 2008 InfoQ article, George Dinwiddie tried to explain the relation between sustained pace and number of hours:
Increasing hours is, in my experience, the least fruitful way to get more done in the software development field. It's a choice that fits H.L. Mencken's description of "simple, obvious, and wrong." As Alistair Cockburn has so delightfully described, people are non-linear systems. Expecting a linear relationship between input (more hours) and output (quicker accomplishment) is misguided.
Overtime is for bumps, not for normal work flow. Predicating project plans on constant overtime is a bad idea, but there are circumstances in which team members may work additional hours without running afoul of Agile principles. Let's examine five such situations.
There is a lot of groundwork to be done at the start of a transition. The coach and some team members may have to spend extra time formulating plans and preparing. The first iteration may have unusual challenges in machine setup or tool choice. A few team members may have to put in a little more effort in the interest of clearing obstacles for the rest of the team. This should not last beyond the first iteration, though.
If bootstrapping issues persist, then they are no longer bootstrapping issues but regular issues. Handle them in working hours and let the team’s velocity honestly reflect its struggles. Overtime is for handling bumps, not for long-term trends.
Sometimes the best-laid plans can go awry for one reason or another. It may be necessary to have a special push because of the timing of maintenance windows or problem severity. If a team is to sometimes work overtime, they need to be fresh enough that a sudden episode of overtime can be tolerated. As Robinson’s paper shows, an already-exhausted team cannot function well when called upon to expend additional effort.
Teamwork does matter here. One of my colleagues, Micah Martin of 8th Light, recently tweeted about pulling a late night in order to rescue an iteration. Online responses ranged from “You’re crazy” to “It’s a shame” to “It happens sometimes.” Micah said of the event, “We were short on the biggest story, a 13 pointer. In the stand-up on the last day of the iteration the team decided that we had to get the story done. I told everyone I planned to stay at work until the story was completed. If the rest of the team had gone home, I would have been demoralized all night, and there's no chance I would have finished the story. The fact that the rest of the team stayed with me made all the difference. Instead of a lonely and painful night, it was almost like a party. We played music and joked about how crazy late we were working. We didn't think it'd take as long as it did, but by 6 a.m. we had completed all the stories, and we felt damn proud of it. It was fun.” The team was fresh enough to accept this “bump” with good humor and energy. Had they been exhausted from weeks of mandatory overtime, the story would have been remembered quite differently.
When overtime is no longer exceptional, it can mask problems. What does it mean if you delivered 10 points out of 12 last time, and even after adjusting down to 10 you cannot complete this iteration? What changed to put your velocity on a downhill slide? Is there anything other than “Try harder” that can address the problem? Is it mere estimation error? Are you losing time to quality issues (bug fixes) resulting from a past rush to completion? Is the design of the application suffering from poor implementation? Using overtime to “overcome the velocity problem” can stop you from seeing the real problem.
Rescue overtime has another problem. Respected Agile coach and consultant J.B. Rainsberger once phrased it this way: “If I work late tonight, I won’t be as effective tomorrow. The question is whether tomorrow is the right day to be impaired.”
Agile teams work to make stories as small as possible, load them into an iteration so that they do not need a work week extension to complete them, and then complete the stories in priority order. When the end of the iteration is looming, the incomplete stories should be the least important of the set. This arrangement allows you to complete as much important work as possible and it takes out some of the sting of having a small story incomplete at the end.
In Agile teams, we don’t say “No” to our teammates. You can ask for a couple of minutes to detach from whatever you are doing, but saying “No” to a teammate is not in the best interest of the team.
“Aid” overtime is a lesser case of rescue. Most people need a hand now and then, and normally it is considered part of the daily effort. Sometimes a coworker may be uniquely troubled by a problem, and it is respectful and discreet to help him handle it without a crowd about. The Agile In A Flash card #33 (earlier version here) recommends “Humanity: respect for the struggles of others.”
It is exceedingly rare to have an inept teammate who is in constant need of someone to write his code for him. Usually a little aid and patience turns a troubled teammate into a solid performer. If a coworker cannot or will not learn to contribute for an extended period, it might be acceptable to let him go, but refusing to help one’s teammates is a sign that the concept of teamwork has not really set in.
Aid is something you can do in the short term, but it is not appropriate for the steady state. Anything that remains a problem for the team needs to be reflected in the team’s velocity and to be addressed by the team. Again, overtime is for bumps, not for regular problems.
Systems administrators and release coordinators know that sometimes we have to work unusual hours in order to fit a change within a “maintenance window.” While working unusual hours does not have to equate to working extra hours, a developer might put in some unplanned hours when there are unexpected deployment issues.
In some situations, an architectural change touches many files, perhaps hundreds or thousands of them. If you are certain that the change can be made (and fully tested) in one evening or one Saturday, and the change is sufficiently important to devote that time to it, then some members of the team may stay late to make the change while the other team members are out.
This used to be more common before the rise in popularity of distributed version control systems. With these new systems it is comparatively easy to make a sweeping change, re-base it on the shared code line, and then use a few hours in the evening to fully integrate and test it. In less enlightened version control systems, it can be a nightmare that requires locking out team members from the code base while the change is made.
Sweeping changes are risky things. I’ve seen undetected errors sneak through poorly-tested areas of the code. The urge to “just fix it all” can be an expression of frustration instead of strict necessity and may not end as desirably as one might wish. I speak from experience.
Luckily, such sweeping changes don’t come along every week and the need to stay late is disappearing in modern tool sets. Such changes should be made by at least a pair of programmers, and maybe a few pairs, considering the risk of widespread system breakage. There should always be a plan for reverting the entire change in case it does not go as planned.
Sometimes programmers enter a state we like to call flow, where we are unusually productive and lucid and able to see past our editors to the code in an unusual way.
Flow requires a kind of hyper-focus on the task at hand. When the work is flowing and we’re learning rapidly, we become afraid to answer the phone and can become irritable if interrupted. As a result of our fascination with a problem we can simply forget to go home (ask my poor wife).
People don’t have this problem every day or every week, but it happens. Watch for signs of trouble if one person’s “flow” results in bugs that make the rest of the team clean up after her for days. It may have been self-deception. We can think ourselves more clever than we are.
Extra work is sometimes necessary as standalone episodes, but there are risks. You have to ask if your overtime is hiding problems instead of exposing them so that the team can solve them, and you have to avoid letting overtime become the new standard. It is not an answer to any long-running problems, but even Agile teams sometimes have to handle a bump in the road.
Co-founder of Extreme Programming (XP) Ron Jeffries said, "Extreme Programming teams are in it for the long term. They work hard, and at a pace that can be sustained indefinitely. This means that they work overtime when it is effective, and that they normally work in such a way as to maximize productivity week in and week out. It’s pretty well understood these days that death-march projects are neither productive nor produce quality software. XP teams are in it to win, not to die.”