At the time the term “agile software development” was coined, it filled a need. Different people had come up with different responses to the endemic problems in IT. Most of these innovations were ad hoc, localized, and not widely shared. A few had been published in book form and were better known than others. But there was no single flag around which people interested in improving the state of the art of software development could rally.
After the Snowbird meeting, we had such a flag. “Agile” was loosely enough defined, and yet clearly enough focused, to serve as a basis for people to innovate in the areas of process, practices, and human factors to achieve better results than were seen with traditional methods, as indicated by the many industry studies and surveys carried out in the 1990s and early 2000s. The loose definition of “agile” was, in the early years of the movement, its main strength. The very looseness of the definition enabled innovation. But every strength is also a weakness, when it is taken beyond the point of diminishing returns.
Now, more than ten years later, the word “agile” has suffered from the inherent downside of its loose definition. The word has been co-opted by every pundit, author, consultant, trainer, and coach in the IT field. For them, “agile” means “whatever we sell.” For people in IT organizations, the perception is that they had better embrace “agile” or they will be marginalized and their careers will stall. For them, “agile” means whatever it has to mean to qualify them to sit with the cool kids in the school lunchroom. They are more concerned with achieving a high score on the latest “agile” assessment than they are with software quality, consistent delivery, and sustainable performance. Form trumps substance.
Today, people fall into two main camps when it comes to “agile.” In the first camp are those who seem to think “agile” is a magic potion that cures all ills. They ask for “agile” this and “agile” that without having first carried out any sort of root-cause analysis to identify their problems, or any sort of strategic planning to identify their goals. When you ask them how they think “agile” will help in their situation, they have no answer. “Agile,” whatever it means, will just magically fix everything. They tend to overlook or misunderstand two of the most fundamental tenets of “agile” thinking: People over process, and continuous improvement. They are looking for a canned “agile” model they can implement once and repeat forever by rote. Yet, by definition, such a thing cannot exist.
In the second camp are those who go out of their way to look for “agile failures,” which they use as a convenient excuse to maintain the status quo. These are the people who constantly demand studies that prove “agile” works, and consistently dismiss any studies that don’t say what they want to hear. At the same time, they cannot provide any studies that prove their own preferred methods work. They don’t understand how “agile” might help for the same reason they don’t understand how traditional methods cause problems: They don’t really understand how or why any approach works or doesn’t work in a given context. They’ve memorized one way of getting their work done, and anything different is simply a threat to them. They talk about “waterfall” as if it were a proven effective way to build software, and “agile” as if it were some sort of heresy.
In both cases, the danger is that people tend to overlook the substance of things. The faithful assume that if they can just manage to label whatever they do in “agile” terms, then somehow things will turn out well. The nay-sayers are quick to discard anything that appears to be associated with the “agile” movement, including methods and practices that were well-proven many years before Snowbird, and that are relevant to software development and delivery whether one uses an “agile” approach or not.
There’s an unhealthy focus on the word “agile,” to the detriment of critical thinking, goal-setting, and problem analysis. “Agile” is not an end in itself, but only a means to an end. People have lost sight of the end, and look only at the means.
It’s time now to parse out the substance of the lessons learned in the “agile” movement, add those lessons to our overall IT toolkit, and move on.
Good to know we agree on something Dave 🙂
Jordan
I fully agree, well worded 🙂
Well put.
Hi Dave,
I just read your blog entry and couldn’t agree more, having talked about this very thing back in November, with the same title. Did you go to the LJC Open Conference or see my slides?
http://speakerdeck.com/u/sleepyfox/p/agile-considered-harmful-sleepyfox-2011
Or is this just a case of convergent evolution and an idea whose time has come?
Nope, not familiar with that conference or your slides. Great minds think alike, and so do we. 😉
I think the problem with the people considering agile harmful are the ones working in organizations requiring some guarantees – something quite common in business setups.
My experience is that especially in larger organizations agile can be misused quite easily, masking management and team self-organization failures.
I’d be a lot happier if one statement would be added to the agile manifesto, something like “don’t do things of which you aren’t 100% sure they are the right thing to do”. This would allow the introduction of a formal validation and control process in an agile setup – and I don’t mean tests.
Other than that, to me the statements of the agile manifesto are overlapping to a large extent with generic good management practices (if you replace “working software” with “working products/services”), which is why I’m not very impressed by the agile manifesto – the fact that few companies apply these management principles doesn’t change their validity, and doesn’t limit them to software development.
What I do value about Agile is the focus on various techniques considered specific to agile. However, these techniques are not a monopoly or invention of agile. Agile merely recognizes and epmhasizes their importance.
So, IMO it’s really not that much that Agile is harmful, it’s just that it’s not that much of a novelty, on one hand, and it’s a lot easier to abuse than more formal and controlled processes. In a way, the way agile was promoted is very un-agile: there’s a significant lack of unit tests for the process itself in what many companies designate as their agile process – even if the last item in the list of principles behind the agile manifesto emphasizes precisely this aspect.
Thanks for your comment.
Three responses:
First, I didn’t mean to say that agile was harmful, I meant to say that “agile” was harmful. It’s the word that causes confusion and misunderstanding, not the values and principles. People aren’t “abusing” agile, they are doing the best they can in their present circumstances based on their present level of understanding and skill. It sounds trite, but it really is a journey and not a destination.
Second, a statement such as the one you suggest would completely “break” the Agile Manifesto. One of the core ideas is that we should not be afraid to try things and find out if they work. If you want to wait until you’re 100% sure of the right thing to do before you dare to do anything at all, then the Agile Manifesto is not the document you’re looking for.
Third, agile methods are not less rigorous than most traditional ones; they are moreso. They place significant demands on all participants in all roles to stay engaged and to apply the highest standards of professional practice in their respective disciplines. Formal processes that function in the same way as a cookbook are far less demanding, far less rigorous.
Sometimes it’s the right thing to do some experiments. My comment was about something else. My worst experience with some sort of “agile” process practised by a customer was that a simple project spent about four times the budget and five times the time it should have spent – and the customer was happy. Only, all developers on the project team were extremely frustrated. The reason was that since change was part of the process, it got embraced so dearly that we literally started maintaining two versions of code in some places, and just swap them when the customer changed his mind again. Even if the project could have easily been developed incrementally, it was iterative all the way, mainly because the customer did not care to think about his needs early on. This simply led to waste which programmers alone could not avoid – we had … um, constrained access to all stakeholders. I see no mechanism in agile to avoid such waste.
Thanks for the conversation.
So, on the project you mentioned, the team did a poor job of managing change. The natural impulse was to revert to traditional practices – a desire for the customer to “think about his needs early on.” All normal and understandable.
Connecting the story with my three responses to your first comment, it sounds as if:
The team did the best they could under the prevailing circumstances based on their then-current level of understanding and skill. It’s a journey, not a destination, and this team was not yet very far along the road. They encountered the same sorts of speed bumps and pot holes as other teams that begin the journey.
The “solution” is not to ask stakeholders to define all their requirements up front and minimize the amount of change. The “solution” is to learn how to handle continuous change gracefully. What you described sounds like thrashing. Teams can learn how to deal with continuous change without thrashing, but there is a learning curve, just as there is for any other new skill. I know it can be frustrating at first.
Waste that programmers cannot avoid…because they haven’t yet learned how. “Agile” methods demand a high level of skill on the part of all team members; more so than traditional methods. I don’t mean programming skills exclusively. The reason you don’t see a mechanism to avoid waste is not that it doesn’t exist. “Agile” methods aren’t going to hand-hold a team through every difficulty they encounter. The team is the mechanism to avoid waste. They just have to learn how.
“Agile” is just a set of values and principles. It advises us to embrace change. It doesn’t tell us how. Specific process models or methodologies commonly associated with “agile” values and principles do tell us how. For example, Scrum is based on the timeboxed iterative process model. According to Scrum, once the Product Owner has agreed to the solution increment that is to be delivered in the Sprint, he/she cannot change the definition of that solution increment until the end of the Sprint. This helps control thrashing in situations when the Product Owner has a lot of fresh insights. Kanban as a software development method is based on the Lean notion of continuous flow. It doesn’t require timeboxed iterations. The mechanism to control thrashing is to limit work-in-process and not to change the definition of work items once they have been started. In both those examples, radically different functionality has to be defined as a new work item, rather than just extending a User Story indefinitely. So, there are mechanisms “in agile” to control this form of waste. The team you described simply had not learned them yet.
Don’t give up.
🙂 As you can see, I don’t give up, at the risk of pissing you off. IMO the team did manage the whole process the best way they could, however, the team could not impose a process. The essential stakeholders were intentionally kept away from the team by the customer, so we had no way of comming into direct contact with them – when we do, the result is usually a very satisfied customer, with each project delivered within budget and time, and also feature-complete – when doing outsourcing, these are the terms the customer uses most frequently, even when he talks about agile methods. And we do quickly integrate changes, and never complain about them – of course, while also thoroughly discussing them with all stakeholders, so they should be aware of the consequences.
The problem, IMO, was that that particular customer, who was not a software development organization, could not find a prescribed way in various agile methodologies on how to control waste production – his simplistic, initial understanding of agile was that thrashing was perfectly normal. That’s my whole complaint about agile (only I didn’t get to formulate it properly on my own): there’s no hint whatsoever as to how to avoid thrashing. Which is why many organizations, especially more forward-looking but isolated departments in larger companies (from what I could see until now) consider thrashing normal, when they decide to do something “agile”. The problem (IMO) is specifically that in such organizations thrashing doesn’t have such an impact as in smaller ones (resources are plenty), so there’s no strong indication (other than common sense) that thrashing is actually bad, and no incentive to eliminate it.
I underwent some PRINCE2 training some time ago. Esentially, you can use PRINCE2 to manage SCRUM. PRINCE2 is in a way orthogonal to SCRUM. If any organization would actually do this, there wouldn’t be a problem with agile methods, because the whole controlling aspect would be taken over by PRINCE2, and managed strictly from a business POV. Unfortunately, this most often doesn’t happen, and many organizations, more precisely many lazy people in many organizations, just use agile methods as an excuse for giving up any controlling activity, letting projects drift. My only complaint with agile methods is that the formulation of the agile manifesto is at least partially to blame for this, since it doesn’t include any hint at controlling/acting upon feedback. The last of the 12 principles is IMO too weak to enforce the idea that agile doesn’t mean out of control.
Why would I be pissed off? We’re not having a debate of any kind. I understand everything in your story, and I know the experiences of your team are quite normal. I’ve only been sharing my thoughts about what happened, based on experience.
I’m sure the team did the best they could under the circumstances, based on their then-current knowledge and skills. I thought I already said that. I wonder if the difference in our perspectives is that you believe the team not only did what they could under the circumstances, but that they actually did everything possible; that they did everything anyone could have done, while what I’m hearing is a story like many other stories of teams that didn’t have sufficient knowledge and experience with “agile” methods to plow through the organizational and process-related issues they were having.
As for essential stakeholders being kept away from the team intentionally, that’s quite common in organizations that are new to this way of working. They probably thought they were doing the right thing. It’s a matter of education, training, coaching, etc. – not just for the team, but for the stakeholders, the “customer,” and anyone else involved.
You say the customer “could not find a prescribed way in various agile methodologies on how to control waste production.” Well, that’s unfortunate, but not a failing of “agile” methods, as they do contain ways to minimize waste. I provided a couple of examples previously, and they aren’t the only examples. You also write, speaking for yourself this time, “there’s no hint whatsoever as to how to avoid thrashing.” That just means you, too, would benefit from some education, training, coaching, etc. All this is perfectly normal and understandable.
It’s possible the only part where you and I have a different understanding is that you don’t know what you don’t know, as the saying goes. I don’t mean that in a negative way. It just seems to be an obvious explanation for your comments regarding the apparent lack of any mechanism in “agile” methods to control change. It’s all normal, for teams that are at a certain point on the path. I keep saying this, and maybe it just doesn’t make any sense to you, but it’s a journey, not a destination. That implies (to me, anyway) that we have to keep on walking. If we’re having difficulty managing change, if we’re having organizational challenges such as non-availability of key stakeholders, or whatever other issues we are facing, we just have to break them down and solve them as best we can. In the situation you described, the team simply did not know what else to try. That doesn’t mean the approach they were attempting to use objectively lacked any mechanisms to help them. It only means…it only means…they didn’t know what else to try.
You write, “My only complaint with agile methods is that the formulation of the agile manifesto is at least partially to blame for this, since it doesn’t include any hint at controlling/acting upon feedback.” Okay, now I have to ask, and again I don’t mean this in a negative way, whether you’re clear about the meaning of the English word, “manifesto.” It’s a type of document that states general values and principles. That’s all it’s supposed to be. The Agile Manifesto serves that purpose. Specific methodologies or process frameworks such as Scrum, Extreme Programming, Crystal Clear, Kanban, and others are the places to look for more explicit advice about managing change. Criticizing a manifesto for being a manifesto doesn’t seem very constructive.
There’s no question of getting pissed off…I hope that’s true on your end, as well. I’m not arguing with you. I understand what you’ve been saying, and I’m certainly not disputing any of the experiences you and your team have had. I’m just offering observations and information based on experience. If you find any of those observations useful, then it’s all good.
🙂 Well, at least thanks for your patience.
Anoymous Coward permalink
I think the problem with the people considering agile harmful are the ones working in organizations requiring some guarantees – something quite common in business setups.
LOL, as if there are some “organizations” that don’t need projects to be successful.
Sorry, I can’t agree with your article. Every time I’ve worked with the product of an agile development effort in the field, it’s been a cobbled together, non-orthogonal disaster.
Here’s a “user story” fo you… I’M A field engineer AND I WANT gee, a matching delete function to go with this new upload function after I just sent this 1.2GB ISO file up to the system.
Oops. Gee, how did that make it past the scrum?
Maybe because all the fields in the test cases in Rally are blank except for the “PASS” status line.
IMHO Agile is the development paradigm of the ADD generation. “Let’s get ~something~ out there quick and let the customer figure it out. Worst case, we can always ssh in & fix it in the database. Then we can not record the defect because we’re busy fighting 100 other different fires of the same ilk, each lovinging put out in their own hand-crafted way. No need to document defects…I already edited the code on the bus home from work. We’re good!”
Please, don’t tell me that “that team didn’t implement Agile correctly”. In and of itself, IMHO agile encourages bad practices, the main one being, “it will never be perfect so don’t bother trying”. You might as well try driving the speed limit on the highway.
Agile ~might~ be good fo developing rapid prototypes, not actual products that are “guaranteed” to work.
For a reply to this comment, please see http://wp.me/p1CUP0-3N.
[…] comment was posted on an earlier item entitled "Agile" considered harmful. I thought it was interesting enough to deserve a little more visibility than it would have as a […]
Dave, I think you will like this: http://effectivesoftwaredesign.com/2014/01/20/attention-agile-programmers-project-management-is-not-software-engineering/