Does the world need another blog post that attempts to explain the differences between agile and waterfall methods in project management? Maybe not, but I was asked to put one together anyway. So now I share it with you.
The waterfall method of project management relies upon a linear series of activities to reach an end product. The series is always the same: plan for the product by assembling a list of requirements that it must fulfill; build the product based on those requirements; test the product once built to ensure that it does what it’s expected to do; and deploy the product to the marketplace or deliver it to the customer for whom the work was done. Each of these steps are thought of as “gates” that must be passed through before the project can progress to the next step in the sequence. This method is simple, easy to understand, and has been in use for decades. While appealing in its simplicity, this method often fails when applied to very complex projects or those with a duration exceeding six months.
Having such a long history of use, the waterfall method has a well-documented track record. We know precisely where its shortcomings lie. It suffers from a number of foundational assumptions that are often false.
For instance, it assumes that all requirements can be known in advance of product development. What actually tends to happen is that the customer’s requirements change once they see the final product. Paradoxically, the very act of creating a new product changes what’s desired. A related problem is that project planners, in an attempt to prevent shortcomings in the final product will “overscope” the project, leaving the engineers or developers with responsibility for including much more than the product’s essential elements.
Perhaps the most significant problem with this method is its inability to respond quickly to changes in product requirements. Once the planning gate has been cleared (and possibly even the testing gate), any new requirements introduced by the customer results in re-starting the process, losing weeks or months in the process. Change orders of this type can drain project funding. And the reverse can also happen: because the waterfall timeline is linear, a delay in clearing one gate, delays all subsequent gates. There is no flexibility.
Lastly, this method assumes that the innovation and research necessary to create a new product can be scheduled in a predictable way. Again, decades of empirical evidence show this assumption to be false.
The agile method of project management tries to overcome the limitations of the waterfall approach. Its adherents tout its ability to make projects faster and make products that are better and cheaper. Where the waterfall method is heavily process- and documentation-driven, agile emphasizes “value and principles.” Where strict application of the waterfall method results in reams of documentation, the agile method emphasizes the production of working products. Having a working product which does not meet all of the requirements is preferable to building a full-featured product that fails to make it to market in time to be valuable. To that end, it encourages the production of working intermediates and sub-components instead of waiting until waterfall’s “integration” gate is reached.
It actively rejects the aspects of the waterfall that make it inflexible and unresponsive to changes in project requirements. Agile embraces change and stays responsive to it by focusing on frequent and free-flowing communication between the engineers or developers and the customer. In the ideal application of the method, status update meetings take place daily, obstacles are discussed, and project priorities are re-evaluated and re-assigned. Supporters of the agile method believe that collaboration with the customer is time better spent compared to waterfall’s up-front investment in contract negotiations.
Agile seeks to avoid the problematic assumptions of the waterfall method: it believes that customers cannot know all of their requirements in advance; it does not assume that mid-project changes will be small and manageable; it doesn’t believe that integration can be left to the end of the project, it encourages continuous integration; and it does not assume that significant innovation can be scheduled according to a known timeline. And it seems to work. Like waterfall, the method has been around long enough to compile data on its effectiveness. That data shows that agile methods result in higher developer productivity and product quality.
But agile is not a panacea. It has known problems too—different from those of the waterfall, but problems nonetheless.
If a project commits to agile methods, it has to be all-in. For example, the hefty communication commitment is an important component of success. If business owners aren’t fully invested in daily meetings and providing constant feedback, the process won’t work. And the communication should occur face-to-face. But what if project teams are geographically distributed? This method requirement becomes a problem that could lead to failure.
One of agile’s great strengths—its commitment to rapid responses to change—can be one of its great weaknesses: some agile experts have noted teams that experience “change burnout.” Not every organization can handle the required pace.
Finally, the greatest threat to a project run on agile principles is the slippery nature of how it is defined and understood. Because it’s not as straightforward as the linear waterfall, some may interpret it simply as a laissez-fair waterfall method, one that is lax about documentation, planning, and milestones. Agile proponents emphasize that attending a couple of agile seminars is not enough to be a successful agile project manager. It requires rigorous training and commitment.