Explaining late projects
"I love deadlines. I love the whooshing noise they make as they go by." — Douglas Adams
Several analyses have shown that roughly half of all projects will run late or never finish. Yet, many organizations continue to be blindingly optimistic when they plan projects — a cognitive bias referred to as the planning fallacy. This causes waste from an economic perspective, but more importantly, an immense amount of stress for pretty much everyone involved.
There are a lot of opinions on why projects go awry and how to fix them, but I wanted to take a shot at distilling down the essence of the problem into a visualization:
Even projects that enjoy a fairly solid shared vision, thoughtful requirements, diligent estimation work, valiant communication practices, etc. still suffer from the same core problem — each task’s estimated length is still just that: an estimate. That means that the actual time to complete is probabilistic in nature, with an expected value and a distribution of likely outcomes. Yet, most project plans ignore this element of variability, or treat the task length as the upper bound on outcomes vs. what it is — a best guess at the mean. As a result, if any one task’s actual completion time is higher than its expected value, the entire project timeline immediately shifts. And given that humans are inherently poor at estimation, the likelihood is that many of the tasks will end up this way, resulting in a project that’s drastically behind schedule.
Of course, this basic statistical reality of scheduling tasks one after the other is compounded by all the other familiar reasons for projects going haywire. It’s both funny and sad how quickly most projects fail to even remotely resemble their beautiful Gantt chart origination. Two ways I’ve seen teams proactively deal with this issue: 1) buffer time, and 2) agile processes. In the former case, the resulting visualization looks like:
The benefit of this approach is that you’re saying: “Yes, the deadline is a long time from now, but at least we won’t be late.” In my experience, the reality is that one of two things will happen: 1) the project doesn’t get approved because the timeline is too long; or 2) the project gets approved, but because of Parkinson’s law (“Work expands so as to fill the time available for its completion”), each task ends up filling its buffer allotment. As Don Reinertsen would say, “Your buffer has now converted uncertain earliness to certain lateness.”
Proponents of agile business and development processes will point this out, and suggest that we should embrace the fact that we don’t and in fact can’t know exactly when a project will launch, and to instead focus on optimizing the process around short term iteration and reassessment. I am broadly a proponent of this approach but I’ve learned that “agile” is often used as a replacement for “strategy”, and to many stakeholders in an organization that makes it feel like:
Even in very agile minded teams and organizations, this “trust the process” pill is hard to swallow, which often means that either: 1) the project manager gets beaten back into putting together a Gantt chart / offering up a hard timeline; or 2) the project moves forward but tensions run high and trust breaks down, the effects of which diminish the gains provided by the agile process.
Is it all doom and gloom? I don’t think so — in my experience, there are a mixture of principles that when employed together can balance agile processes with long term planning. And by doing so, not only can a team better achieve their goals in projects that have a relatively predictable set of steps/phases (as visualized above), but also the more frequent situation where projects must adapt to new information and ever-changing variables.
I am planning on writing more about this balancing act in subsequent posts, with the open-eyed perspective that it’s not a “just put the solution in place” type thing. The types of changes that make this possible involve getting buy-in, changing ingrained habits, and figuring out the right approach for your particular situation and people. And even with all of that, it’s still a continual learning process.