I know I don’t write that frequently, but I have an excuse for the extended break this time – a five week visit to South Korea. I had thought about advertising my absence, but it occurred to me that if there is some criminally minded person paying attention to my blog, they could quite easily find out where I live and rob me of my meagre possessions.

Perhaps I’m just projecting onto my readers…I apologise, but I assume that all good testers are potential criminal masterminds!

There are a few important things I took from my Korean trip -

• I really don’t know much Korean.

• Immersion is a great way to learn.
• The Korean yahoo homepage (www.yahoo.co.kr) isn’t as crazy as I thought it was.
• Korean game testers are different (from the game testers that I know).
• Don’t buy from a Buddhist
• We have great coffee in Australia, but Koreans have Poka-Yoke coffee.
• It is still possible to play arcades for 20c (or less).
• Small differences can keep you on your toes.
• Amusement parks are kind of freaky when you’re the only person in them.
• Don’t touch the windows.
• King Sejong was a smart guy.
• Marrying your wife twice is twice as good.

More on these later…

After extending my stay, I also managed to attend the Korea Game Conference and GStar tradeshow. I think I had expected the audience to be a little more international. See if you can spot the lone caucasian in the crowd here.

I attended the two sessions on testing ” QA Operation and Process by the game producing step”
by SinAe Kim and “Game QA organizing and Process set-up cases”. These presentations described the experiences of a large Internet portal’s game development studio as they attempted to improve the quality of their published games. They test a mix of in-house and externally developed titles.

I was somewhat surprised at the testing approach described, as it seemed quite corporate, and not product-like at all. This is explained to some degree, I believe, by the relative youth of the Korean games industry. Most likely, testing expertise is more easily found in business environments. My impressions from the conference and brief conversation with Sinae suggested that she and her team were learning a great deal about the ways in which the testing processes of banks and other corporates are unsuited to mass-market product development.

The areas on which their testing focused were -

• Specification testing

• Compatibility testing
• Performance testing
• Security/Exploitation/Abuse testing

Interestingly, they don’t do gameplay testing before completing functional testing, whereas it is more commonly performed the other way around. Tune the fundamentals, then fix the bugs.

I must add that I don’t yet fully understand the Korean games market. There is a lot that you can play for free, but I don’t have a full understanding of where all the money comes from.

Having seen a number of testers posting their heuristics on Testing reflections, I thought I might chime in with one that popped into my head. I’m calling it the SEP heuristic, which is probably all you need to know about it if you are familiar with Douglas Adams’ Hitchhiker’s Guide books.

It is simply this -

- Anything which project members or stakeholders appear to be treating as “somebody else’s problem”, is quite likely not so, and highly likely to bite us firmly on the backside if we continue to treat it as such. Ignore at your peril.

As SEP fields can be beaten by specifically looking for that which is concealed, the countering strategy is simple, but not necessarily easy. The SEP heuristic tells us to look for problems or tasks that are being ignored or thrown in the ‘too hard’ basket and focus energy there sooner rather than later.

I think that early in one’s career, it’s easier to let some things be somebody else’s problem, with minimal personal consequences. I’m finding as I have increasing responsibility, this heuristic becomes increasingly important.

I came across someone asking for an answer to the old “How do you test a stapler” question, and in light of my new role, I thought this was a good opportunity to start taking up James Bach’s methodology challenge, using the stapler example as a starting point.

I’m starting with the meta-questions, those which I need to answer *before* I can begin to think about what and how I’m going to test it.

• Who is the stapler for? Who is it *not* for?
• Why do they want a stapler?
• How might the different stapler users know that the stapler has fulfilled their needs? What stapler features are unimportant?
• When does it need to be tested by? What will happen if it’s not done by then?
• How much can I spend to test the stapler?
• Who else can help me test the stapler?
• Does it need to be tested by anyone else before it can be considered ‘ready’?
• Does anyone else need to check the stapler, or approve my work before the stapler can be used by the person who wants the stapler?
• Is there anyone who doesn’t want the stapler to be had?
• What tools am I allowed to use to test the stapler? What tools do I know how to use that might help?
• Who might need to test the stapler at a later date? Who is going to read my tests? What documents do I need to create? Where do they need to live?
• Who shouldn’t be able to use the stapler?
• How critical is the success of the stapler? What might happen if it isn’t tested adequately, or fails to meet the needs of the stakeholders?
• Are there any standards our testing needs to comply with?
• Are there any legal constraints?
• What kind of staples does it need to work with?
• What kind of materials should it be able to staple?
• Does it need to work in conjunction with anything else?

I’m sure I will continue to add to this list and organize it, but it’s a start.

I’ve also found I need to develop a checklist to hit the ground running and ensure that I look far enough forward when my involvement with a project begins. For this, I’m grouping some questions under the heading “Start from the end”. This is a subset of the above questions, but are the questions I think I’m most likely to leave until too late if left to my own devices.

• What does success look like?
• What are the release processes/signoffs? Who are the stakeholders who need to sign off?
• When does it need to ship by? What is depending on it being released on that date? What else does our product depend on?

And so, it begins. If only I’d started earlier!

Developing personas is a well-described technique (see Alan Cooper’s ‘The Inmates are Running the Asylum’ and Mike Cohn’s ‘User Stories Applied’) for considering the different kinds of users of the system we are developing. On a recent project, we began considering the different users who might want to user our product. In the process, I was able to connect it to another recently acquired trick and gain further insight into the user of our system.

The initial features added to our system seemed to be really only considering one type of user. Most of the team felt strongly that there were other important users who were being ignored, as many of the features didn’t seem to work the way we expected them to. They made sense for the targetted user, but made little sense when we thought about how *we* would want to use the system.

This suggested to us two user types – The bargain hunter and the traveller. We imagined the traveller to be someone planning a future trip. They were looking for discounted goods or services that would be available for them to use on holiday, some time in the not-immediate future. We also imagined a bargain hunter, someone for discounted goods and services, with a sense of urgency. This is the kind of shopper who hates to miss out on a bargain. Our system forced us to consider this kind of user as the primary customer.

It seemed that the two had some common objectives, but different needs. We tried to model this -

This begged the question -

What about these customers? Who might they be? We wondered if we could give them a name, and if they were worth considering?

We named them. They were important customer types whose needs had to be taken into account.

The Special Occasioner - A person who is looking for a good deal, but is prepared to take time to find it. Perhaps for a wedding anniversary, or someone who is planning a holiday, but hasn’t firmly decided.
The Last minuter – A person who leaves things to the last minute, has to unexpectedly travel, or just acts on the spur of the moment.

It was at this point that I flashed back to my recent reading of Howard Becker’s ‘Tricks of the trade‘, and his writing on ‘Substruction’. He describes substruction as -

‘…the logical converse of reduction. Reduction puts combinations together for the sake of simplicity. Substruction takes them apart, in the interest of discovery.’

As it turns out, this is exactly what we had just been doing. From analysis of our initial two personas, we were able to derive two more, and on further analysis we expanded this even further.

Here’s the process -

1. Determine the important attributes of a persona. In our case, we started with two, but you may have more.
2. Write down all the combinations of presence and absence of the attributes. This may be a truth table, but Becker’s book provides examples for non-binary attributes.
3. Work out which combinations of present or absent attributes correspond to the personas you have already thought of.
4. See which combinations don’t have a name. Try to name them.

Personas are a great tool. Enhancing the approach with the techniques of substruction gave us a much richer model of our system and the kinds of people who may have an interest in using it, and points to possibilities we may not have considered. Howard Becker’s book, though targetted at sociologists, is a great read for anyone looking to improve their thinking.

I was responding to a posting on usenet on the trials of transitioning from a hardware and device testing role to a software testing role.  To me, this breaks down to creating a resume which focuses on the critical, non-domain related skills of testing.  I think this advice is applicable to anyone seeking to move into a different testing field, or simply struggling to find a job.  In both cases, we need to focus on the skills of testing that cross over domains.

Finding a job consists of two parts (once you’ve found a job that looks interesting) ?

- Getting to interview
- Succeeding at interview

Getting to interview should be reasonably straightforward, as long as there is no cause for an agent to reject you when they hear you speak on the phone.  If speaking English is a problem, I can’t imagine a quick fix, although others have advised the use of voice coaches to improve pronunciation.

Here’s the trick to getting an interview.

To find suitable candidates, most agencies simply match keywords in resumes.  You should troll through all of the jobs on Seek, MyCareer, CareerOne and Jobserve and look at the keywords that agents and companies are using in their advertisements.  I am not advising that you falsely add keywords to your resume.  I am advising that you look to see which of those keywords apply to you, then make sure that they appear in your resume, preferably on the front page, in a list.  This is about making sure that you are using the same language as your customer, who in this case is a recruiter and an HR person at the prospective employer’s office. 

Keep in mind though that there are two audiences – The HR person/Recruiter who is not a tester, and the test manager or team lead at the end who is a tester.  You will need a mix of concrete descriptions for the first party (the skill sumary) and more abstract skills for the technical audience.

Succeeding in the interview, and making the descriptive parts of your resume compelling both rely on your understanding of testing.

For the descriptive parts of the resume, make sure that you have some descriptions of your experience in terms of abstract skills, and not the activities you performed.

In the company I am currently working at, I’ve hired testers primarily for their thinking ability.  If you are able to capture this ability in your resume, you can greatly increase your chances of finding a role and crossing over to software testing.

When I receive resumes from hardware/device testers, they commonly imply a tester who simply reads a specification, then implements some automated test to verify that the specification was met.  It is rare that I see language which describes the interactions with people.  So try to think about the reasons someone might have for not hiring you for a software tester role.  Try to address these in your resume.  Try not to express your abilities in terms of the physical things you have done, but in terms of the less tangible skills of a tester.

For example -

- What have been the inputs to your decisions on what it is that you should test?
- What have you done to understand the needs of the various stakeholders?  How have you worked with the business?  With beta-testers?  With analysts?
- What approaches have you applied when designing tests or modelling the system?

Lastly, read about testing. Improve your knowledge of what we actually do when we say we are “testing”.  If you can get to interview and talk clearly and persuasively about the craft, many previously closed doors will be opened.

Hope this is of some help.

One more word on XP as methodology (well, a few more actually).

Any methodology seems to me to be a snapshot of a solution to a particular problem that somebody solved at some point, with a particular set of people and skills in a specific context. There are occasional statements flying around the agile-testing group which suggest to me that the intial applications of XP were either in similar contexts or recreated the important aspects of the original context.

One of the things that strikes me about the thoughtful practitioners Michael describes (and that I’ve been able to observe) is that they are actively seeking to solve a problem. They evaluate a development approach that they hope will solve the presenting problems they have, optimised for the skills that the team possesses and the organisational context. Then they attempt to solve their problems through the application of the designed approach.

My snapshot of some of the important attributes of XP’s context is this -

- Poor or non-existent testers and business analysts.
- Developers who were quite good at testing their own code and focusing on the business domain.
- Knowledge residing in customers’ (or analysts’) heads struggling to be free.

I’m convinced that there are many applying XP who have never really given thought to the problems it is designed to solve, the skills that are required to apply it successfully, the people required to make it work, how it might affect those who don’t fall within the XP developer circle and whether it is suited to their organisation and its culture.

We may then witness some of the following -

- Alienation of those who are not developers while the developers ‘do XP’ (by the book).
- Sub-optimal use of the skills of non-developers.
- Disappearance of team members who get less than two paragraphs of description in the XP book.
- Practices not solving the problems they were designed to when the underlying values or principles are not present.

It’s hard work to tweak XP to fit a team that isn’t just customers and developers.
So think about it. What skills do you have in your team? In whose hands do they lie? How will you work with the people around you to get the most out of your project, not just the developers?

Time to kick off my new blog at my self-indulgent, self-promotional home. You can find my old blog here.

Update – 2009-08-06: In the migration to wordpress, I’ve moved some of that old content here as well.

I’ve shared some ‘perils of XP’ conversations with Michael of late, so I wanted to consider my experiences on the topic.

One thing that strikes me about software methodologies is that like many things which are written down, recorded and/or institutionalised, the reasons are often long forgotten. The early adopters are typically passionate, thoughtful, and (I believe) seeking to solve a very specific problem.

Those that come to the team later may learn the practices that the current team is using, but never understand the problem that those practices are intended to solve. Indeed, the existence of the practices may cause them to never witness the problem at all. This may then lead to the Routine behaviour that Michael describes.

I’m also thinking that while writing down our values and principles as a guide for others is helpful, it may be more helpful to explain the journey that led us to our current thinking. Of course, we are usually slow to learn from the experience of others. I’m not yet sure how we instil just enough fear and understanding into those who haven’t experienced the pains of yesterday, but it seems like an important thing to do.

Before applying any software development approach (or modifying an existing one), it might help us to ask a few questions -

- What problems are we trying to solve?
- How might all of the team members best work together to solve these problems?
- How will we know if we’re succeeding?

I’ve been (very) slowly collecting names and email addresses of enthusiastic testers in Melbourne, and have been hoping to organise some kind of meetup or regular drinks night. Is anyone from Melbourne reading? If so, drop a comment. If there is enough interest, I will post a venue/time.

Erik, you must come!

Note: This now happens monthly via MAST – http://melbournetesters.ning.com

- that you’re going to get it right first time.
- that nobody knows better than you do.
- that it is going to be faster if you do it all at once (tightly coupled to the first point).

Another possibility might be that you’re afraid of negative feedback, which, in the context of working on a software team, is delaying the inevitable.

I’m sure there are more possibilities.

Problems with our assumptions

That you’re going to get it right first time

How might we behave if we assume that we are not going to get it right?

That nobody knows better than you do

How might we behave if we assume that everyone might know something we don’t?

That it is going to be faster if you do it all at once

How do we know that ‘faster for me’ is necessarily faster for the project overall?