The "planning fallacy" refers to a psychological theory that people systematically underestimate how long it will take them to complete a given task. My work life is one long example of the planning fallacy. I set deadlines for myself, scramble to meet them, miss the earlier deadlines, rinse, lather, and repeat until the work somehow gets done.
Here's a description of the planning fallacy from Daniel Kahneman and Amos Tversky in a June 1977 report they did for the Defense Advanced Research Projects Agency (DARPA), called "Intuitive Prediction: Biases and Corrective Procedures." They write:
The context of planning provides many examples in which the distribution of outcomes in past experience is ignored. Scientists and writers, for example, are notoriously prone to underestimate the time required to complete a project, even when they have considerable experience of past failures to live up to planned schedules. A similar bias has been documented in engineers' estimates of the completion time for repairs of power stations (Kidd, 1970). Although this 'planning fallacy' is sometimes attributable to motivational factors such as wishful thinking, it frequently occurs even when underestimation of duration or cost is actually penalized.
Roger Buehler, a researcher in this area, put it this way in a short explainer piece in 2019 (Character and Context blog of the Society for Personality and Social Psychology, "The Planning Fallacy: An Inside View," May 30, 2019).
The planning fallacy refers to an optimistic prediction bias in which people underestimate the time it will take them to complete a task, despite knowing that similar tasks have typically taken them much longer in the past. An intriguing aspect of the planning fallacy is that people simultaneously hold optimistic expectations concerning a specific future task along with more realistic beliefs concerning how long it has taken them to get things done in the past. When it comes to plans and predictions, people can know the past well and yet be doomed to repeat it. ...For example, university students typically acknowledge that they have typically finished past assignments very close to their deadlines, yet they insist that they will finish the next project well ahead of the new deadline. Then, predictably, they go on to finish the next project (as usual) right at the deadline.
The planning fallacy is remarkably robust. It appears for small tasks like daily household chores (such as cleaning), as well as for large scale infrastructure projects such as building subways. It generalizes across individual differences in personality and culture, and it applies both to group and individual projects. For example, conscientious people often get things done well before procrastinators, but both groups typically underestimate how long it will take them to get things done.
Why does the planning fallacy happen? Kahneman and Tversky explained in 1977:
The planning fallacy is a consequence of the tendency to neglect distributional data, and to adopt what may be termed an 'internal approach' to prediction, where one focuses on the constituents of the specific problem rather than on the distribution of outcomes in similar cases. The internal approach to the evaluation of plans is likely to produce underestimation. A building can only be completed on time, for example, if there are no delays in the delivery of materials, no strikes, no unusual weather conditions, etc. Although each of these disturbances is unlikely, the probability that at least one of them will occur may be substantial. This combinatorial consideration, however, is not adequately represented in people's intuitions (Bar-Hillel, 1973). Attempts to combat this error by adding a slippage factor are rarely adequate, since the adjusted value tends to remain too close to the initial value that acts as an anchor (Tversky and Kahneman, 1974). The adoption of an 'external approach', which treats the specific problem as one of many, could help overcome this bias. In this approach, one does not attempt to define the specific manner in which a plan might fail. Rather, one relates the problem at hand to the distribution of completion time for similar projects. We suggest that more reasonable estimates are likely to be obtained by asking the external question "how long do such projects usually last?", and not merely the internal question "what are the specific factors and difficulties that operate in the particular problem?"At some level, this explanation strikes me as exactly correct. I am overly optimistic when thinking about how long it will take me to do a task because I assume that everything will go smoothly and that I won't be interrupted or distracted by other immediately pressing tasks. When I edit a paper, I think about the actual editing going smoothly, not about what happens when I hit a snag that takes a day or two to resolve or what happens when the rest of my life sneaks up on me and demands attention.
I find myself asking: If wasn't subject to the planning fallacy, would I get things done on time? For me, daily motivation seems to be some combination of optimism and self-imposed stress. Both of these are embodied in the planning fallacy: that is, the planning fallacy gives me an optimistic view of how much progress I should be making, but then also stresses me when that progress doesn't happen. If I instead started each day with: "Things are going OK, more-or-less on schedule and it's 50:50, at best, whether I will get that paper edited by tomorrow," I'm not confident that I would be happier or more productive.
The end result of this dynamic is that I've been the Managing Editor of an economics journal for 34 years, putting out quarterly issues more-or-less on time, but also feeling perpetually behind schedule. This seems a potentially unhealthy combination.
So I try to strike a balance. At some level, I know I'm fooling myself most mornings. In some strictly rational part of my brain, I know the day's work isn't likely to go as well as I hoped, and I also know that I'm not as behind as I feel. For me, the tradeoff is that when a task is completed like editing a paper, or when an issue of the journal is published, I get a jolt of surprise and happiness. In that same strict strictly rational corner of my brain, it's really not a surprise. After all, I've been editing papers and putting out issues for a long time. But rather than seeing my life as a straightforward process of linear movement, plodding step-by-step to an expected outcome. I apparently prefer to see it as a mini-drama: I could do it! But I'm not doing it as quickly as hoped! I'm behind! But it's getting done! It's a race against the schedule! I've done it! Then I start all over again.
For 2021, I hope that you too can use the emotional energies unleashed by the planning fallacy to your advantage, both as an encouragement and a goad for daily effort, and to give you a sense of accomplishment at the result.
