Programming work is not very much like digging ditches (the work for which project-management techniques were originally designed); it is a lot more like digging holes in very stony ground. Sometimes things go beautifully, sometimes a programmer spends whole days beating his head against the wall on a single point, until finally the light bulb lights up. These are two very different modes of mental operation, commonly known in the industry as Flow and Stuckness.
Flow is the mental state when a programmer understands what he is doing and directly interacting with the process of program writing or debugging. In Flow, hours can go by like minutes, and amazing productivities can be achieved. Error rates drop drastically, and work becomes a kind of highly productive play. Maintaining this state as much as possible is very important to successful programming work.
The difficulty with Flow is that it takes about 15 minutes to enter the state, and an interruption that must be dealt with (phone call, visit, bio-break) will terminate the state. Therefore, if there is one interruption every 15 minutes on average, the amount of Flow-assisted work (which in practice can mean the amount of work, period) drops to zero! It is no accident that programmers still report, as they have for the last 50 years, that the most productive programming times are after hours, because it becomes possible to enter Flow and remain in it hour after hour until the belly or the bladder demand attention.
Stuckness is the reverse of Flow; the state where the programmer simply can't see what's wrong. The point may be and often is completely obvious in hindsight, but discovering what that point is, is literally the same kind of discovery process that scientists use (or mechanics, for that matter): make a hypothesis, test it, attempt a fix, see if it solves the problem, test to make sure a new bug hasn't been introduced. There is also variety of Stuckness that arises during design rather than debugging, a kind of analysis paralysis where it is clear that the current way of doing things is wrong, but no clear vision of the correct way has yet emerged.
Stuckness cannot be overcome by will or planning, only by insight, and the arrival of insight is not predictable. Striving for insight can often be what prevents it from being achieved. The motto "Sleep on it" reflects the perception that blocking the conscious mind can be just the right thing to allow the unconscious mind to operate, often giving the key to the problem as if from nowhere. Distraction is an equally successful approach. During times of stuckness, therefore, interruptions are actually desirable, as they allow this useful distraction.
All these things make completion time estimates for programming projects something akin to a black art. One cannot predict from one day to the next when interruptions will occur, of course, but their effects on the project can vary from very good (intense Stuckness) to very bad (intense Flow disruption).
A relatively successful model for programming management, however, is based on the notion of trouble-ticket tracking. Since projects are not for the most part self-generated but rather arise out of requests from editorial, sales, or customers directly, each of them can be treated as a trouble ticket at whatever scale, and then the question becomes not "Is this project past its highly arbitrary deadline?" but rather "How many projects being starved of the time and attention they need?" Mechanised and human support for this process is extremely helpful and is not usually provided to the extent necessary.
References:
Weinberg, Gerald. The Psychology of Computer Programming.
Brooks, Frederick. The Mythical Man-Month.
DeMarco, Tom, and Tim Lister. Peopleware.
10 comments:
Helpful and well-put. Thanks for this post!
Well said (as always)!
I've been trying to communicate this to my bosses, with little success. Since my work is almost entirely interrupt-driven, with the interrupts being quite unpredictable, getting anything done that requires problem-solving is almost impossible.
We are banned from doing any work whatsover related to our jobs outside of the hours during which we are clocked in, yet there is little time to think then. While we may have entire hours during which no customers interrupt us, we have to maintain a state of attention to them that prevents us from entering flow mode. I've had a long memo forming in my head about simple fixes that will make our store computer systems far easier to use, but I never have a chance to actually sit down, organize the thoughts, and write it.
Harrison Bergeron lives. *sigh*
Thank you for so eloquently putting this down on [virtual] paper. This post is an excellent description of exactly what goes on in programming with respect to interruptions, and extends far past the business/workplace environments; my personal experience is identical when programming for work or for pleasure, except with one major difference. The level of frustration at interruption during a flow period at work is much less than that during play. While at home, interruptions from family members can be both very demanding and very annoying. Just another look at the matter from what may be a more practical perspective for many of us super-geeks.
Nice essay; it really captures to mindset of the lone programmer.
I find that explaining the problem to someone else is the best way to get past stuckness. Often it doesn't even matter whether they can help - just launching into an explanation is often enough to help me think of a solution. (So, if nobody else is available, explain the problem to your cat.)
Of course, unless you're pairing, asking for help is someone else's interruption.
Brian, see the description of Teddybot, an IRC bot designed to help with this problem. He mostly just says uh-huh and go on, but occasionally spews one of Brian Eno's Oblique Strategies (hi, Joe).
I note that most of the comments here and elsewhere (kai Technorati) focus on the badness of interruptions in flow, rather than the goodness of interruptions in stuckness. Hmm.
So how do you deal with external depenendencies on your project if you don't want any deadlines?
Not very different from link fixing a motorcycle is it?
Well said indeed. What struck me recently is that slow edit/debug cycles much more severly affect performance than is explained by the cycle-time itself. This is because, in my experience, having to wait for the system for too long can do one of two things: (a) make me think about what I am just doing, or (b) make me slouch off to the Internet or a magazine article or an interchange with someone else (which, incidentally, breaks their flow too). Given that the reddit et al. are always beckoning, choosing (a) is getting rare. So waiting often breaks my flow.
I am thus led to believe that, quite apart from all its other benefits, test-driven development is extremely good for keeping you focused. This is because, in this style, you typically build fairly isolated tests that have quick edit/debug cycles, rather than having to launch an entire application, click your way through it for the nth time, etc.
Noting Pradeepta's posting, should not Robert M. Pirsig's "Zen and the Art of Motorcycle Maintenance" be in your biography. I thought he coined the word "stuckness". Correction cheerfully accepted. Would really like to know.
Zen and the Art of Motorcycle Maintenance is a wonderful book. Consider it added!
As for who invented the word stuckness, it's an obvious compound made of two productive parts stuck and -ness, so anyone could have invented or reinvented it at any time.
Post a Comment