Presstimate:

The number you give to get a manager off your back when you’re being hassled to give an estimate; Your best guess of what estimate of effort management will accept, not how long the work will actually take.

I’ve had past conversations with others on the topic of intellectual property rights of employees in Australia, and had always been under the belief that pretty much anything I did in a creative capacity would belong to my employer.  Today I was pointed to this article, which suggests that’s not really the case.

Of course, for simplicity and to avoid any legal action, most people will still take care to separate their entrepreneurial activities as much as possible from their work.  Colleagues frequently leave their current role to go off and fully develop something they’ve been working on secretly.

So if you’re interested, this article has a good discussion of the state of play for employees, employers and their IP rights - 

http://www.findlawaustralia.com.au/articles/default.asp?task=read&id=16492&site=CN

This tweet was forwarded to me yesterday:

martinfowler: Manual scripted testing should be a human rights violation

It bothered me on a few levels.

Firstly, the simplicity of phrasing around manual and scripted testing. Secondly, that agile developers might view themselves as the saviour of oppressed testers everywhere, and the perpetuation of the concept of tester as helpless victims. And thirdly, criticism of scripted testing is hardly new, and it’s a bit sad that one of the key proponents of test-driven development and design has taken so long to come to this idea.

Still, his comments make a nice soundbite, and what he states is often true, but until he says *why*, I’m not sure if he knows what he’s talking about. It just seems a perpetuation of simplistic binary models of automation and prescriptive testing.

We can do better. There’s much more we should be talking about when discussing manual testing and the conditions of testing in corporate life.

I start with these ideas -

- Let’s put under computer control and evaluation the elements of our tests which are well suited to manipulation and/or evaluation by computers.
- Let’s put under human control and judgement the elements of our tests which are well suited to human manipulation and evaluation.
- Let’s prefer exploratory approaches when we want to find new things and adapt to the information at hand.
- Let’s make sure that we record the details for important things that we don’t want to forget to do, for things that are confusing that we can’t fix.
- Let’s be aware that any test have elements of exploration and prescription.
- Let’s be aware that the testing model we work from biases the kinds of problems we’ll find and the kinds of things we’ll miss.
- Let’s choose appropriate levels of abstraction to help manage the cost of our test design.
- Let’s have as many ways as possible to model our test designs so that we can effectively solve the problem at hand.

In particular, the first two points suggest different ways of doing things that blur the lines between traditional automated and “manual” tests, as discussed by Kaner, Bach and Kohl:

http://www.kohl.ca/blog/archives/000193.html
http://www.satisfice.com/articles/boost.shtml
http://www.satisfice.com/blog/archives/118

On the dehumanising of work, a lot of testers pulled out of frontline business roles were probably already having their human rights violated for far less money than they now make as testers. That doesn’t excuse things, but there aren’t as many testers complaining about the status quo as one might expect. My observation is that the nature of testing as practiced in most places drives the majority of those who might want to take a more creative approach elsewhere.

Standard business practice continues to be to throw money at staff retention problems in skilled roles rather than consider how work might be made to suck less. This isn’t uniquely a software testing problem, and the world is some way from tackling this as an issue.

There have been a number of threads I have followed in a few different forums recently where people have discussed requirements, what it means for requirements to be ‘good’, and what it might mean for requirements to be unambiguous. What usually follows is a long-winded back and forth, with no resolution.

At the heart of this deja vu is the fact that one person’s requirements are not necessarily another’s. I resolve this by drawing a clear distinction between specifications and requirements. The distinction may be obvious to some, but in practice it seems to be something we struggle with.

In “Software for dependable systems: Sufficient evidence?”, Daniel Jackson and his co-authors take care to highlight this issue:

“Software systems that are developed specially for a particular client are typically built to meet preagreed requirements, but these requirements are often a long and undifferentiated list of detailed functions.”

“The requirements of a software system should describe the intended effect of the system on its environment and not, more narrowly, the behavior of the system at its interface with the environment, which is the subject of the specification.”

They also take care to point out that the environment of the system includes the software product, plus the humans that use them, and other environmental factors external to the software.

The specifications are not about things that anybody *needs*. Specifications represent the end result of negotiations, conversations, politics and expediency that some group of people thinks represents an understanding that is good enough for now. The specification is at best, our best guess of what’s going to make the world a better place for the numerous people who have a stake in the thing that we’re building. Specifications are a waypoint on the path to something else.

Specifications are never equivalent to requirements in the case of things that will be used by humans.

Specifications apply to the pointy end of a screwdriver that needs to fit into the indented part of some screw.

As testers, testing to specifications is something we do because finding about the requirements is too hard. We do it because that’s what the testers before us did. We do it because the process might be built around specifications documents, and that’s what managers are tracking to. We might reasonably test to spec if our job is just to test a software component that’s on its way to be integrated with something else. However, testing to spec can’t tell us that the system is going to yield the desired benefits.

In the case of the screwdriver, requirements apply to the handle – how it fits your hand, and the hands of others. They apply to whether it gives you enough grip and whether it has a switch to make it only screw or unscrew. They apply to whether it fits enough hands in the world, and whether it can be built to a price that someone is willing to pay. Requirements apply to whether the pointy end of the screwdriver will make do for unscrewing (or screwing in) a screw that is not of the size covered by the specification for the pointy end of our screwdriver.

Requirements are about utility and real human problems, and are fuzzy, and messy, and never fully understood. When our stakeholders ask us questions about testing, though they often don’t phrase it this way, they are usually interested in information with respect to requirements, both explicit and implicit.

Test to see how the product measures up to requirements, and to learn about what the requirements are. That’s the value that you can bring to the project.

“In the universe, nothing can be said to be automatic, as nothing can be said to be without design. An imperfect parallel may be found in human inventions; springs may move springs, and wheels, indexes; but the motion and the regulation must be derived from the artist;”

From Elements of Chemical Philosophy Part 1, Vol.1 By Humphry Davy

Of course, the second step will be taking my automation or testing tools course (plug, plug).

In the Yahoo software-testing list, Shrini Kulkarni stated

“…productvity as a term is “bad” for creative work like “testing” or “art”. That makes me to feel that I am a low skilled labour on a manufacturing assembly line (not that – it is a bad profession but that does not represent the kind of job I do or I would to do or I am capable of doing)”

It triggered some thoughts from recent projects, and my reply follows.

What bad things happen when we apply the word ‘productivity’ to testing?

Like it or not, unless we are on a fixed price contract with no deadline (and when is there *not* a deadline?), the person who pays is going to be evaluating the value that we provide relative to the money they spend. This seems to be a productivity measurement, and things don’t usually go well when we ignore it.

We are often forced to balance the artistic/creative part of our work with giving our employer the impression that we are being ‘productive’. My understanding of history is that most artists through history have also had to concern themselves with both productivity and art, especially if they want to make a living.

To me, productivity is the key to defensible testing. It’s the idea that I should be striving to get the best coverage that I can for the least cost.

I think where we go wrong is when we define productivity as output or effort, not ‘value’. Unfortunately, the former definitions are common, and I think this is a huge problem in many places that I work. That is, there is a focus on appearing busy and creating artifacts, and not on gathering the information that would most help the project right now.

There are negatives to the project in terms of our personal productivity if we are forced to ignore those parts of our work which give us the greatest satisfaction. For me, this does include the aspects of testing that I consider creative and artistic, but it also includes other things, such as the satisfaction I get from face-to-face communication, developing relationships, working ethically, collaborating and helping the development of others.

It’s not all about art, but we all have days when our inner artist is challenged. Some days slamming the lid down on the piano and storming off the stage is what’s called for. But mostly, we respect the audience…and the show must go on.

During coffee with Agile-coach and all-round excellent guy Shane Clauson, in sympathy with yet another of my what’s-wrong-with-agile rants, he pointed me to this blog post from Jeff Patton:

Don’t know what I want, but I know how to get it

While my opinions diverge on some of what he says must be true, I think the important message that he (and others – Check Alistair Cockburn’s writing on this) make is that it pays to plan to iterate. That is, if you’re on an agile project and you don’t see anyone planning to rework things in response to feedback from using the product, you’re probably in for some disappointment.

I think we frequently fail to give our customers an appropriate expectation when it comes to (agile) software development. Having them read this isn’t a bad start, but you’ll want to figure out how to make this message your own.

Many moons ago, when I was a young tester, I worked on the first third party game for Microsoft (Please don’t look for it, it’s terrible). But there was some good to come out of the experience. Windows ‘95 was new shiny, and fraught with danger. To help address some of this, I began collecting test ideas and describing the expected behaviour of ‘good’ Windows applications.

Much of the older documents we created were designed to support an approach that was mostly exploratory. Exploratory testing was supported by coverage models that provided loose charters, and the feedback on tester sessions was verbal.

For the more prescriptive parts of our test effort, documents such as this provided support. Bug catalogues, risk catalogues and checklists were used extensively to ensure that important things were not left untested.

So enjoy the first of these checklists, my Windows ‘95 testing cheat sheet. For anyone testing a Windows game or application, much of this still applies. If you’re testing web applications, this may give you some ideas for how your application should handle events for user-interface controls.

You can find the checklist at http://www.quinert.com/documents/Win95AcceptanceChecklist.pdf.

The concept of inattentional blindness has been, if you’ll excuse the unintended pun, brought to the testing world’s attention lately by Cem Kaner and James Bach. Sajjadul Hakim has written recently on an exploratory testing experience in which he attributed failure to observe a bug to inattentional blindness.

While this may have been the cause, it is worth pointing out that our attention is a fragile thing. Inattentional blindness is but one of the ways that our brains can let us down while testing. Here are a few more:

Attentional blinking

If something grabs our attention, then something else happens almost simultaneously, or just after, we may miss it.

Inhibition of return

If something happens somewhere where we were just looking, we are less likely to see it.

Negative priming

Our brain applies filters to suppress things that are not relevant to the task at hand. These filters can arise through conscious effort or can happen if we perform a task long enough. An example of this is ‘banner blindness’, where things appearing in the spots where website advertising usually appears are highly likely to be ignored.

I became acutely aware of this effect last year. Steve Irwin had just died, and the news came to our team via a phone call to a teammate. I immediately jumped on CNN’s website, looking for confirmation. “There’s no news of it here,” I confidently claimed to the rest of the team. I looked back at the screen, and scrolling across the top of the page was a ticker, proclaiming Irwin’s death as ‘Breaking News’. It’s position at the top of the page, where advertisements usually appear, had caused me to skip straight over it.

Our brain is able to put filters in place to help optimise our performance of common tasks. These effects can last for more than a month after we have stopped performing the task that caused the filters to be put in place, just in case we need to do it again.

Our brain’s processing power is limited. In addition to the above problems, sometimes there’s just too much for it do deal with. It has developed tricks to help us survive lapses of attention, but these are also things that can catch us out as testers. The act of operating the product in order to test it is something which focuses our attention. By doing so, we blind ourselves to some problems.

I play video games with a kind of unfocused stare, and I suspect this actually helped me detect different kinds of problems when testing games, while probably making me blind to others. I find I can’t usually test applications with that protracted stare. I can however observe the application this way by putting the application under computer control, or by observing a playback of a recorded session. Video recorders and Watir scripts are great for this, and give you the chance to find different problems. James Bach also frequently recommends Spector for recording testing sessions.

Strategies

There are plenty of strategies for reducing attentional failures. Mix these up over the lifetime of the project for best effect. Some of them I’ve described already, but others I leave for you to Google -

- Rotate people through projects and tasks to avoid negative priming.
- Negative priming effects are reported to be reduced by alcohol consumption. Have a drink at lunchtime. Tell your boss science says it’s OK!
- Pair up…Maybe even with non-testers or people who have nothing to do with the project.
- Use recording tools and go back and review your testing sessions.
- Automate operation of the product, sit back and observe. I’ve used this technique, and Jonathan Kohl has success stories to tell as well.
- Blink testing. This is a defocusing strategy popularized by Michael Bolton and James Bach. Find different problems in test results, fast, by taking the detail out of them and scrolling through them quickly or zooming out.

If you would like to find out more, check out the book Mind Hacks: Tips & Tricks for Using Your Brain (Hacks). It’s a great primer on your brain, how it works, and how it can let you down. The ‘Developing Intelligence’ blog is also another great source of information and ideas.