There have been a number of threads I have followed in a few different forums recently where people have discussed requirements, what it means for requirements to be ‘good’, and what it might mean for requirements to be unambiguous. What usually follows is a long-winded back and forth, with no resolution.

At the heart of this deja vu is the fact that one person’s requirements are not necessarily another’s. I resolve this by drawing a clear distinction between specifications and requirements. The distinction may be obvious to some, but in practice it seems to be something we struggle with.

In “Software for dependable systems: Sufficient evidence?”, Daniel Jackson and his co-authors take care to highlight this issue:

“Software systems that are developed specially for a particular client are typically built to meet preagreed requirements, but these requirements are often a long and undifferentiated list of detailed functions.”

“The requirements of a software system should describe the intended effect of the system on its environment and not, more narrowly, the behavior of the system at its interface with the environment, which is the subject of the specification.”

They also take care to point out that the environment of the system includes the software product, plus the humans that use them, and other environmental factors external to the software.

The specifications are not about things that anybody *needs*. Specifications represent the end result of negotiations, conversations, politics and expediency that some group of people thinks represents an understanding that is good enough for now. The specification is at best, our best guess of what’s going to make the world a better place for the numerous people who have a stake in the thing that we’re building. Specifications are a waypoint on the path to something else.

Specifications are never equivalent to requirements in the case of things that will be used by humans.

Specifications apply to the pointy end of a screwdriver that needs to fit into the indented part of some screw.

As testers, testing to specifications is something we do because finding about the requirements is too hard. We do it because that’s what the testers before us did. We do it because the process might be built around specifications documents, and that’s what managers are tracking to. We might reasonably test to spec if our job is just to test a software component that’s on its way to be integrated with something else. However, testing to spec can’t tell us that the system is going to yield the desired benefits.

In the case of the screwdriver, requirements apply to the handle – how it fits your hand, and the hands of others. They apply to whether it gives you enough grip and whether it has a switch to make it only screw or unscrew. They apply to whether it fits enough hands in the world, and whether it can be built to a price that someone is willing to pay. Requirements apply to whether the pointy end of the screwdriver will make do for unscrewing (or screwing in) a screw that is not of the size covered by the specification for the pointy end of our screwdriver.

Requirements are about utility and real human problems, and are fuzzy, and messy, and never fully understood. When our stakeholders ask us questions about testing, though they often don’t phrase it this way, they are usually interested in information with respect to requirements, both explicit and implicit.

Test to see how the product measures up to requirements, and to learn about what the requirements are. That’s the value that you can bring to the project.

A few weeks ago, Designer commented on Software testing, art and productivity. The question got lost in amongst the comment spam, so I thought I’d give my answer a bit more prominently than usual. The question was:

…Many people who want to get a web-developed project don’t even understand the details of work. They just want to have a result and not to make a lot of efforts. How do you think – is there any solution? I think it is wide-spread problem.

If you’re talking about smaller web-based projects, I think you’re right. It can be really difficult to engage clients in the necessary up-front work to help reduce project uncertainty to a reasonable level. It’s a problem given that those with less money to spend have much more business risk in any project they undertake.

I think the second aspect of this is that customers (both internal and external) often come to us with a solution, not a problem. As we build the solution they asked for, the problem becomes clearer and dissatisfaction starts to creep in.

The focus of my work is increasingly on trying to help people build the right thing in the first place. I’m lucky. In my current role, my employer has the conviction that it’s important to make sure the project is heading in the right direction. This means making sure the project team has a shared understanding of the product vision, stakholders, those stakeholders’ goals, priorities and (in the case of a consumer product) the market and opportunities. They also think that it’s OK to bring development to a pause while we get our project bearings.

If you can’t choose the projects you undertake, then I don’t see any easy answers to these problems. And if you can’t convince your customer to be involved appropriately, and to place *some* value on just talking about the problem, it’s a hard road ahead for everyone. I guess the desire to do things ‘right’ is what drives many of us to start our own companies and projects. Without this option though, we can still focus on providing service – helping our customers better understand their problems and pointing out the benefits, costs and risks present in their solution(s). But choose your moments well, and don’t stop dreaming that things can be better than they are.

Mike Cohn’s “User Stories Applied” discusses using the INVEST mnemonic as a guide to writing better user stories. I was recently asked to dig up a reference for it, and found this presentation here, with the section on the mnemonic on pages 47 and 48.

As I read it, I noticed that there’s been a change to one of the letters. Whereas the book uses ‘S’ to denote ‘Small’, now it’s become ‘Sized appropriately’.

I think this is a change for the better, as I noticed that every time I talked someone through the acronym, I would have a long-winded conversation qualifying ‘Small’ as ‘Just small enough but no smaller’. This would come about as I tried to explain the tradeoffs between a story being small enough to estimate with some reasonable certainty, small enough to fit within an iteration, and still ensuring that stories are ‘V- Valuable to the customer’, needing to ensure that user stories continue to express clearly a problem or need of a person.

Overly small stories push us further from the original context of the problem, and thus force us to compensate with an increasing heirarchy of ’super-stories’ to help us focus on the bigger picture. These become more noticable when working in shorter iterations than might be common on a Scrum project, so three cheers for Mike in spending the effort to come up with ‘Sized appropriately’.

During coffee with Agile-coach and all-round excellent guy Shane Clauson, in sympathy with yet another of my what’s-wrong-with-agile rants, he pointed me to this blog post from Jeff Patton:

Don’t know what I want, but I know how to get it

While my opinions diverge on some of what he says must be true, I think the important message that he (and others – Check Alistair Cockburn’s writing on this) make is that it pays to plan to iterate. That is, if you’re on an agile project and you don’t see anyone planning to rework things in response to feedback from using the product, you’re probably in for some disappointment.

I think we frequently fail to give our customers an appropriate expectation when it comes to (agile) software development. Having them read this isn’t a bad start, but you’ll want to figure out how to make this message your own.

I’ve had a few common rants on most of the agile projects I have worked on. Developers bogged down in the detail of stories, while the critical goals of the system wound up ignored, or realising at the last minute that all of the stories built would do nothing useful.

The ideas I came to as a result of the problems I observed were -

- Compensating by starting with extremely high-level stories that defined the critical user and system goals, then progressively breaking these down into tasks.
- Ensuring that the intent and goal is clear from the story title
- Compensating by coaching the testers to ask the critical questions “Why is this feature being added? What problem does it solve and what value does it add?”. The testers would try to ensure that the context was made clear in the story. Links to the higher-level stories in Jira (the tool I most commonly encounter) also helps to provide context.
- Writing high-level acceptance criteria at the top level of stories to help define alternate paths for each goal. These often provided clear boundaries with which stories could be broken down into sub-tasks or sub-stories.
- Evaluating frequently against the high level goals.

I’ve only had a few chances to really try this out, but from a recent project experience with a little more involvement at the early stages of the project, it seemed I was on the right track.
At some point, being aware of Alistair Cockburn’s work, and being a regular reader of his blog, I realised Alistair had probably written about most of this, and I picked up one of his books on use cases. I expected that he’s figured most of this out and more.

Then before I got around to reading the book, it was confirmed for me with this blog post, which I highly recommend taking a look at. Even if you’re not going to apply use cases, it’s worth asking his questions of your project and seeing what you can learn from them.

If you are using use cases, are you avoiding the problems described there?

I work with James, and he emailed this quote around the office -

“UNIX was not designed to stop you from doing stupid things, because that would also stop you from doing clever things.” (attributed to Doug Gwyn)

I had forgotten about my response until I re-subscribed to his blog just now.

“Unix needs an undo key”

‘Capturing the essence’, or ‘core’, has been a key theme in some of my work recently, and in several of the books I’ve been reading. So over a drink with Michael Ruschena tonight a couple of these came out as we linked ‘the core’, ‘agile’, and haiku – poetry that captures the essence. I’ve been coaching a team around testing on agile project, so I thought it might be fun to try to capture the essence of that…

I thus present agile testing haiku.

Note: I’m just going to keep adding new ones to this post so that they’re all in the one spot. And increasingly I’m finding these are not just about agile development, but software development generally.
————————————————–

Method it is not
Apply values, principles
To meet your own needs

Planning game is key
How do we know when we’re done?
Check the product goals.

Can’t make it all fit?
You’ll need to take something out
or get more money.

This user story
Does not express your intent.
Why, why, why, why, why?

Push the risk up front
Shorten the feedback cycles
Highest value first

Development, test
Are all part of the same team.
Try working closely.

Good acceptance tests
Set goals but skip one main thing -
Implementation.

We got a green bar,
Why is the customer sad?
Just add a story.

I didn’t do it.
It wasn’t in the story.
Look, new offshored jobs!

Exploratory tests
Help find problems most quickly
but make extra work.

Skilled software testers
Use many different models
Try many angles

Though up-front tests help
We can get stuck in a rut
Go and have a beer

Embrace fuzziness
Precise is not accurate
Band your estimates

Some we can know now,
but some we must build to learn.
Best tell the owner.

Start with a small team
When the hole size becomes known
let everyone dig

Unit, story tests
Why is our build getting slow?
More techniques will help.

How are we going?
If you can’t tell at a glance
have visible charts

Rough iterations?
Preserving uncertainty
may help things improve

Estimate fully
But to manage greatest risk
Take away two thirds

The test strategy…
What information and how,
tied to the vision

Left values or right?
What’s the pairing’s principle?
You’ll need skill at both.

The grind of agile
Sprinting then taking a break
helps you sustain pace

What is knowable?
Predicting or reacting?
Find your middle ground

The requirements.
Stories are only a third.
What about the rest?

Who’s your customer?
You have no ******* idea?
Find out quick or stop.

Having worked on the trial project mentioned in Michael Ruschena’s recent blog entry on Project Manifestos, I can say it’s well worth trying out. And when I say ‘trying out’, I mean having the conversations and going through the thought processes required to figure out how you might work together as a team. I find my recent test strategy documents sharing much of the spirit of these documents, being about figuring out how we are going to give ourselves a shot at achieving our aims.

Thanks Michael!

Michael has moved to a new city, and is obviously free of social distractions. That, or there’s something physically stopping him from playing World of Warcraft. Anyway, he’s blogging again and his latest entry (http://www.ruschena.org/michael/?p=107) on writing technical requirements is well worth a read.