Programmers are all too well aware that fixing one problem is all too likely to create even more, leading to a reluctance to make changes that aren’t actually necessary. This found its way into a little ditty recently:
Ninety-nine little bugs in the code; 99 little bugs: you take one down and patch it around… 117 little bugs in the code!
Developers have tried for years to figure out ways to avoid it. One common approach is to keep the same developer working in a particular area of the code, because he knows it so well. You want something changed, you just ask Chris, who can tell you, “oh, that needs a new clause in the third if statement after the second comment block in this really long method.” And then Chris leaves the project and Jan takes a look at the code and shudders, as it is not remotely readable without Chris’s knowledge, developed over years of working with it. And when we cannot rely on code familiarity, we fall back to trying to be very very careful, and making the smallest change we can think of to fix the first problem and hope its effects don’t ripple very far. Of course, that makes the changed code that much more complex, and that much more liable to breakage, the next time we want to change it. We need a better way.
Agile, you have have heard, is that better way. Agile is the rage, nowadays. Software organization after software organization is adopting it, often by management decree – and many of them aren’t seeing much benefit. These organizations are faithfully adopting Agile terminology and rituals, handing out Agile-defined titles, and producing Agile-prescribed deliverables, but not getting a lot of improvement. That leads them reasonably to conclude that Agile is just another worthless fad. But what, exactly, is Agile, and why doesn’t it appear to be working? To understand, we need to take a step back and look at the software lifecycle and software development in general.
In theory, changing code should be easy. We have no concrete to chisel away, no steel to cut and weld, no circuits to try to reposition on a circuit board. In practice, as noted above, it’s usually not. One change can easily ripple through many components, with results that aren’t always obvious. For years, the software industry has recognized that reality and tried to structure projects in ways to minimize the need for change.
The traditional way to try to minimize the need for change is to do very thorough analysis and planning, which is the way projects in other industries have long worked. In this approach, called the waterfall model or Big Design Up Front (BDUF), we spend a lot of time analyzing our requirements, design our solutions carefully, and only then start coding and testing. The problem is that software projects are not like all those other types. BDUF makes a lot of sense in planning many types of projects, such a building houses or roads, where each result is very similar to old ones, so most functionality and problems can be anticipated. With software, each project is generally unique; if it were the same as a previous project, it could simply be copied digitally. That means that the users and developers learn a lot about what is possible and desirable only after they have had a chance to try it. Serious flaws may not be discovered until well into the construction. There was a project, many years ago, to implement direct deposit for one employer’s in-house payroll system. After they had spent many months developing it, the team discovered that one of their main assumptions – that all employees would want their checks deposited into their checking accounts rather than savings accounts – was incorrect. Astonishingly, this assumption had been so thoroughly baked into the system, that it took substantial effort to change. The developers actually claimed that the entire system would need to be rewritten!
But this puts us in a quandary. If we cannot completely and accurately design the system before building it, and it is costly and error-prone to make changes to systems once they are built, how do we improve things? Agile development steps back and challenges one of our basic assumptions, and asks, what if we make software that is easy to change? Think of the consequences of such a different reality. Instead of code being lumbering and slow to change if we want to avoid creating problems, code changes would be quick and system building would be flexible.
This has massive implications for our requirements and planning activities. We can start with the simplest and most certain requirements, implement those, and get feedback from our business customers. They can see how their ideas translate to reality, bit by bit, and decide what they like and don’t like, and change direction before much of the system is even built. No need for extensive analysis, trying to get everything right from the beginning, which probably isn’t even possible. Rather, we try something, see how that works, and make changes as the customer directs us. That would be an agile development process.
Of course, for this to work, we need to be able to change code quickly. We cannot have code that is hard to understand, hard to read, hard to change, and with consequences we cannot predict – or least we need to minimize those issues. Fortunately, we actually do know how to do this using Test-Driven Development (TDD). That involves very fast unit tests and aggressive refactoring to minimize coupling and duplication, and generally keep the code readable. The unit tests allow us to tell very quickly if a change is going to break the system.
We go further by insisting on sharing knowledge of code, whether by scheduling tasks that move work on code modules by multiple people, pair programming, or some other means. That increases the motivation towards code readability, and means that we actually can take advantage of code familiarity with multiple developers at once. And that way, when one developer is unavailable, we have others who can step in without a massive learning curve.
This, then, is what I would consider to be the essentials of Agile:
- Iterative development, with the coders working very closely and regularly with the business customer,
- Clean code practices, aggressively followed and enforced, and
- Shared code ownership, so that there are always multiple developers who understand any particular code unit.
I will talk more on these topics in future articles.