Billy Pilgrim and the software estimation debate

Warnings:

• tl;dr
• sarcasm
• calories from fat: (don’t ask)

I’d like to begin with an insightful quote. Until I find one, here’s a quote from Vonnegut:

“I am a Tralfamadorian, seeing all time as you might see a stretch of the Rocky Mountains. All time is all time. It does not change. It does not lend itself to warnings or explanations. It simply is.”
— Billy Pilgrim in Slaughterhouse Five, Chapter 4 (Kurt Vonnegut)

The online debate about software estimation may well be the greatest waste of pixels since the invention of porn. Like porn, the debate comprises a never-ending repetition of the same scene over and over again, the very definition of “boring.” And the actors conclude each day in the same state they were in when they awoke. Every day the same, every day exhausting, every day pointless.

The argument, like Kenny of Southpark, refuses to die. It wakes up each morning as if nothing has happened. In a sense, I guess, nothing has.

Anyway, here’s the thing: Despite people’s best efforts and their protestations to the contrary, no one can predict the future accurately and precisely.

If we were Tralfamadorians, “unstuck from time” like Billy Pilgrim, the idea of predicting the future would be meaningless to us, and even if we could grasp the concept, it would have no value for us because we would already know what was predestined to occur. But we aren’t Tralfamadorians, and sometimes we need to get a sense of what is likely to happen when we choose one course of action or another.

I wonder if the question isn’t “Should we estimate?” or (still less) “What is the single best way to estimate regardless of context (and, by the way, you’re stupid if you disagree with me)?” but rather, “What are we trying to accomplish by estimating?”

With that in mind, allow me to make a few mild assertions that surely won’t raise anyone’s hackles.

1. It is not possible to know the future, for any interesting meaning of the word, “know.” (To those who believe they can “know the cost” in advance through estimation, please note the remarks below related to Joe Taxpayer.)
2. From time to time we have to choose among different courses of action, unless we intend to live out our lives as passive actors in the making of our destiny. (Already noted: We’re not Tralfamadorians.)
3. Nearly always, we have to make choices based on incomplete information. (We only have complete information after the fact.)
4. Nearly always, we have to make choices within a given amount of time, or it will be too late to choose. (As the old saying goes, “If you don’t make a choice, the choice makes you.”)
5. Every prediction we make about the future is conditioned by our own cognitive biases. There is no perfectly data-driven estimate, arrived at with perfect objectivity. (People are predictably irrational.)
6. Any prediction about the future is a guess, although some guesses are better-informed than others. This includes estimates based on robust, well-documented statistical methods (possibly because of #5).

I suggest, then, the purpose of estimation is not to arrive at accurate and precise predictions of the future, but rather to inform our choices as best we can. In fact, I’ve mentioned before that it’s a bit arrogant of us even to think we ought to be able to predict the future accurately and precisely. It seems to me that particular flavor of arrogance causes a great many people a great deal of stress. (I don’t know why people choose stress. Go ask a psychologist.)

I further suggest that it’s often a good idea to minimize the overhead in our work — the muda, or non-value-add activity, if you will. As customers do not purchase estimates as such (although, as some friends are quick to point out, they often do ask for estimates before authorizing any work), the activity of estimation (however it’s done) is “overhead.” Therefore, we will tend to prefer estimation methods that incur the least overhead possible in a given context. (A popular alternative is to redefine estimation as a value-add activity. That way, we can stop thinking about it. Most convenient, indeed.)

Some of the circular debates have involved participants with very different backgrounds, and very different assumptions about what the word “estimation” means. There have been years of repetitive back-and-forth between people who have enjoyed long and lucrative careers working with US Department of Defense subcontractors, and people who have enjoyed long careers writing business application software. (No, I didn’t forget the word “lucrative.” Thanks for noticing.) They seem unable to understand one another at all. Often it seems they are unwilling even to try. (Question: Does this sort of debate rise to the level of porn after all? Perhaps it is only masturbation.)

On more than one occasion, the various combatants have been bewildered when I told them both sides were right. They could only conceive of a zero-sum outcome to their debate. Some people aren’t satisfied with being right; they have to make the other person seem wrong. At times it seems as if a debater is willing to go down in flames himself, if that’s the only way to make the other party seem wrong. (Yes, making them seem wrong is sufficient. There is no need for them to be wrong. It all boils down to who can continue shouting the longest. And it isn’t about estimation, really; it’s about who’s the toughest dog in the pit.)

Yet, both “sides” are right. I don’t mean each is partly right. I mean each is completely right, within their respective frames of reference. The problem is that neither can see the other’s frame of reference. In some cases, a person thinks his or her own context and experience constitute 100% of all possible contexts and experiences, and can’t grasp the idea that another frame of reference exists. (The degree of simplification this mindset can produce is certainly enticing. Perhaps that is the reason it is so popular.)

Despite all that, the debate does offer occasional light entertainment value. A classic pattern is that debaters insist they respect each other and are only interested in exploring alternative perspectives on software estimation. But in reality, they are trying to lure the other party to bite a hook; to trap them with a carelessly-chosen word, and beat them about the head and neck with a dictionary. Within three or four exchanges on your favorite mailing list, discussion board, or Twitter thread, their mutual disdain comes to the fore. The pattern usually ends with the offending parties insisting they intended no offense and they hope for a fruitful discussion tomorrow. (It’s just like porn actors wrapping up another repetitive day at the office, washing off the remains of the day, learning nothing and growing not at all, ready to ensure tomorrow will be intellectually identical to yesterday.)

Anyway, where were we? Oh yeah: Context.

“Context” is, of course, another one of those far-reaching words, like the word “estimate.” One aspect of “context” is the scope of the decision we need to make. There are contexts that call for fairly rigorous and formal estimation methods. There are contexts in which lightweight, empirically-based forecasting is sufficient. There are contexts in which estimation does not help us make reasoned choices at all, and only amounts to waste. (People who sell estimation software might not embrace that idea, for $ome reason.)

Someone, somewhere, must make the decision whether to go forward with a development initiative to create a next-generation fighter aircraft. Such a decision is of large scope. It has implications for national security, which in turn relates to national sovereignty. The work is paid for by Joe Taxpayer, who is every government employee’s boss (although government employees don’t seem to realize this), and every government contractor’s boss (although government contractors don’t seem to realize this). So, you’d better come up with a pretty good estimate of time and cost. Then Joe Taxpayer will figure you’ll go over your estimate by a factor of ten, because that’s what you always do, Mr. Accurate and Mrs. Precise. (Or did you think poor old Joe didn’t notice that, because you’ve kept him distracted with all your boasting about how skilled you are at estimation?)

On the other hand, somewhere in the world today there is a small software development team that is planning its work for the coming week. The decisions they must make are of small scope. The amount of time before “the choice makes you,” and the cost of making a poor choice, are very different than in the fighter aircraft example. (Note: People do raise examples like aerospace programs, although the debate is really only about software development. Sometimes one must stretch a bit to find a counterargument. Perfectly understandable. All’s fair.)

What aspects of “context” might influence our choice of estimation method (if any)? One is whether the work is of a familiar sort or not. Another is the relative skill of the individuals who must make the prediction. Another is whether we are planning the execution of work that has already been laid out for us, or we are exploring possible solutions to a problem. I have pontificated…er, that is, I have expressed an opinion about that, as well. (As you grow older, you’ll find you’ve accumulated a bunch of opinions no one cares to hear. You’ll also find that doesn’t prevent you from expressing them. No one is forcing you to read this, you know.)

What if we wanted to discuss software estimation rather than the software estimation debate? Well, okay, but only for a moment. To generalize a bit, regardless of one’s approach or methods, estimates are based in part on at least two of the following:

1. Empirical (“by observation”) information about past performance or trends, including lessons learned from previous attempts, assessment of the efforts of others, historical statistical data, and so forth;
2. The application of an algorithm (possibly part of a formal estimation method, or a heuristic, or a gut intuition) to generate predictions; and
3. Intention, commitment, hope, ambition, need, subjective goal-setting, wishful thinking, the tendency to throw good money after bad, the selective dismissal of inconvenient data, the belief that you can bend the fabric of spacetime by calling it names, the belief that you become invisible when you close your eyes, the belief that you are far too rational to fall victim to such fallacies, and other Science Stuff.

People tend to get tangled up in the details. If your estimation method seems ad hoc to me, I might call it “guesswork.” If your estimation method seems highly formal and standardized, I might call it “heavyweight.” If I like statistical methods and your estimate is based on heuristics, I might not trust your estimate, and vice versa. There are many variations. Often, people who have come to rely on a single method tend to disparage all other approaches categorically. When that’s the case, it isn’t hard to forecast the outcome of a debate.

Many people who engage in debates about estimation want to make a distinction between an estimate and a commitment or promise. This is a reaction to the historical reality in software development work that management treats estimates as promises, when in fact they are only estimates. Oddly enough, I’ve expressed an opinion about that, too. Go figure.

An oddity in the software development field is that we are often told our job is to become better at estimating. But isn’t our job to deliver valuable software? Whatever the purpose of my life might be, I usually wear shoes when I pursue it, but that doesn’t mean the purpose of my life is to tie shoes. In the course of delivering valuable software I sometimes estimate something, but that doesn’t mean the purpose of my work is to make estimates.

The emphasis on “accurate” estimates is doubly puzzling in light of the fact that most estimates are requested to justify a decision that has already been made. (At least, that’s the case in the software field. I don’t work in other fields, so I can’t speak for them.)

Early in my career, I worked for a manager who liked to yell at people when reality failed to conform with their estimates. On one occasion when I was the target of the yelling, I asked him: If I had the gift to predict the future both accurately and precisely, why would I submit to working for a person who lacked that gift? He replied thoughtfully, “Shut up and get back to work, punk!” (or words to that effect). Ah, memories! <sigh>

I haven’t followed his career since then. I don’t know whether all that yelling got him anywhere other than an intensive care unit for heart patients. I don’t know if he ever paused to estimate his chances of a heart attack. Or maybe it was good for his health; maybe he was like Mr. Ken Dove, the character in that one Monty Python sketch who was “interested in shouting.”

In any case, my question stands. Surely, if I were “unstuck in time” like Billy Pilgrim, I wouldn’t need to work for a person like that manager. I wouldn’t need to work at all. I would know future stock prices and real estate values and could invest accordingly. I would know when and where natural disasters, wars, and disease would strike, and I could avoid those places at those times. I would even know when, where, and how this dull, predictable life would end.

But I’m not unstuck in time. So I work. And when I need to make an informed decision about alternative actions, I estimate, one way or another…or I don’t, depending on context, burdened as I am with the inconvenient awareness that multiple contexts exist.