I attended a couple of Agile talks at NDC2011. Particularly there was Michael Feathers' "The Mistake at the Heart of Agile" and Scott Belware's "Beyond Agile" The first of those was well covered by Gojko Adzik over here, in short saying that the software development that is managed via scrum needs to grow closer to the other functions of the business, not isolate itself from it.

I felt a bit frustrated by the start of Scott's talk. He mentioned how "agile" now basically means scrum, and scrum basically means a dumbed-down, fixed template turned out by a small industry as a standardised product. He raised how "scrum isn't the ultimate methodology" and "doesn’t always work". I could have used an example right about there to understand where he was coming from and going too; scrum worked well for me compared to other other ways that I've seen software done. I got a lot more out of the second part of his talk, where he talked about lean and eliminating waste and bringing value to the whole business, not just the software development part of it.

In Michael Feathers talk I raised an interesting artefact of scrum as Ken Schwaber and Jeff Sutherland  originally articulated it: The rigid fixing of work to-do during a sprint, and Abnormal Sprint Termination if this cannot be maintained. This "is a rare act, not to be taken lightly" - it's an act of high ceremony; you pull back and nuke the sprint from orbit and start again amidst the rubble.  It's an intentionally drastic and high-ceremony act to make you ask "are you really sure you want to do that?".

To me it seems like an oddly specific prescription, and must reflect some force that they had experienced and needed to counteract. It's fairly clear what: an atmosphere of chaos towards the software team where there is so much change in conflicting directions that the delivery of actual value suffers - at worst, the software team can't finish anything before they are dragged off onto some other "top-priority" interrupt which is itself overridden before it's done! So they tried to get some kind of wall between the software team and the rest of the business, with the product owner as the gatekeeper. The "customer" does the equivalent in XP. Thankfully, not every workplaces this chaotic, but I suspect strongly that Ken Schwaber and/or Jeff Sutherland were feeling this pain when they set down their rules.

Yet, as was raised in Michael Feather's talk, this chaos is not universal, and even where it is present, it can be alleviated - confidence can be raised by a software team that begins to deliver value consistently, and education can reduce pointless side-tracks. The drivers seem to be different now, or the ways of dealing with them better. The wall can eventually be lowered or dropped entirely. Rules are contextual; they are there for a reason and that reason may have already left the building.

Rules are great for keeping on track or tweaking your process; but sometimes you ned real change and that can't be distilled in rules, it requires insight and creativity. No methodology on earth can code for that. Look at any methodology from high enough, and it will be saying "be clever", "learn" and "be nice to each other" in various ways. Anders Norås suggested using Oblique strategies for creativity but he also said that this was a joke and it would do as much good as using tarot cards. So tarot cards are good then? ;)

Is scrum over?

I received an email a few weeks ago from a recruiter, and as you do I skimmed though it before deleting it. One of the skills listed as a pre-requisite was "formal agile". I've never heard the term "formal agile" before. What could this mean? The high-functioning scrum teams that I have worked on all went about their business in jeans and t-shirts, not suits and ties, so was that then "informal agile"? (only joking, I doubt it). I suspect that they were trying to draw a distinction between the common "yeah yeah, we're agile, we sometimes have meetings standing up" and the "real, true, actual, been doing it since before it was cool, agile" however you define it. Buzword inflation is setting in, since the old buzzword of "agile" is seen as debased currency now.

Still, "formal agile"? There's got to be a better buzzword. There is something in agile, and lean, that is against excessive, empty formality.

I think of those high-functioning jeans-wearing agile teams of a few years ago. They're probably still delivering great value by similar means. A method doesn't stop working just because there's an epidemic of other people getting it wrong. If lots of people start insisting that 3 + 3 = 5, does that mean that addition isn't as powerful as it used to be, or just that lots of people could get more out of their addition? I think I understand what Scott Belware was saying; that poor agile is largely the way it is in practice. That doesn't make "scrum doesn't work any more" a universal truth and certainly is not the way it should be.

But agile isn't as deterministic as arithmetic, in fact it has the opposite rules for good outcomes. Different organisations will use the agile toolkit of "inspect and adapt" in different ways, and end up with different practices. Doing it right is a process not a fixed point, and that's maybe a way to tell if you're doing it wrong, if you have your eye only on some fixed point.

The good teams have a measured appetite for risk. They try a few new things all the time, with the understanding that not all of them will work. They are empirical. Too many risks results in failures, but not taking any risks results in stagnation. The occasional failure is to be welcomed as a sign that you're pushing hard enough.

Teams that have been doing agile for years may start again from a point that looks more like kanban and lean, for all I know. Scott Belware said that "scrum is methodology, lean is a toolkit" I don't know, scrum's "inspect and adapt" seems like a toolkit not a methodology to me. As it should be. Sure, some people are getting it wrong. That hasn't changed what agile is.

I was initially thinking that lean was an agile method a subset of agile, like TDD or scrum, but Scott is of the view that agile is a subset of lean methods. That's interesting, but I'm not well-versed enough to agree or disgree yet.

I can see that there are businesses, particularly startups or small businesses, where the structures that scrum has by default might get in the way, and you may be better off not starting from there, since for one thing the wall between the software team and the rest is really going to get in the way if the whole business is an early start-up, just 5 people or so, hustling for a break. There are also large organisations where the level of independence and adaptiveness in scrum is too much to work well with the corporate or consultancy culture, and the goal of adding maximal business value is not aligned with the incentive structures.

Bob Martin's agile talk was titled "The land that scrum forgot", saying that "Scrum forgot to incorporate the technical discipline". I haven't watched this talk yet, I will get it on video. Bob Martin's advice on developer technical practices is always excellent. However, while you have to be aware that scrum needs addtional technical practices added, this is a design decision, not an oversight. Yes, scrum is only a partial theory of software, not a complete one. Scrum is Agile project management. It deliberately says nothing about development practices besides letting the team have a high degree of autonomy, and helping them inspect and adapt. In practice though, you tend to head straight for the known agile development best practices such as CI, pairing and unit testing to compliment scrum.

To be clear, if you don't have good technical practices, you aren't doing scrum well - you aren't doing software development of any flavour well. But it's a good idea to let these evolve independently.

Where scrum relates to the rest of the business, how it adds value through the whole arc, is proving to be a trickier interface.