Why Automated Testing Enables a Project, But Doesn’t Always Get the Job Done: A Discussion With John Stevenson and Ray Grieselhuber – Part II

Welcome back to part two of our interview with veteran software tester John Stevenson and Ray Grieselhuber from Functionize.

In part one both John and Ray discussed more of the technical side of test automation – what it is, when and how it should be utilized and why it’s important to understand the overarching business value you want to provide.

For part two, we’ll dive a little deeper as they share with us their insight and unique views on why the human behavior is important in testing and cannot be replaced by automation testing, as well as how testers’ roles within an organization are changing and where they see the future of test automation heading.

What human behaviors and valuable insights does manual testing offer that automated testing simply cannot replace, especially when considering the context of the test?

John:

Answering your question goes back to this. This is a psychological. This is where the automation testing debate goes further. Knowing social context, and how socially we interact, it’s part of being a human. That helps a great deal with the testing of software manually. Trying to do that with a machine is difficult.

An example of this, could you design an automated system to reliably tell me or know what a chair is?

Ray:

That’s a really good question. Yeah, my gut reaction is I could not, and if someone could, it would probably be a lot smarter than I am. It brings up the many theories about computer vision and recognition, and understanding what objects are and what things mean. You could describe a chair and all of its components, but then you get into the question of what does it mean when you say something is a chair? What is that and how does it relate to the rest of the world? That would be very challenging.

John:

That’s interesting because even a three-year-old can tell you what a chair is.

Ray:

Yeah. It’s going back to AI. This is a classic, but still very true - differentiation, the difference between humans and computer innovation to this point. We have this inherent ability to recognize objects in the world, and immediately assign meaning to them and tell you what those things actually are. What they are means you’re getting down to the potential question of whatever it is in question because it’s just describing it.

John:

There’s a great social science book by Harry Collins that talks about tacit and explicit knowledge. If we talk about company, big or small, 90 percent of the knowledge inside a company is tacit, is held in people’s heads.

The hardest part, and this goes back to the knowledge, is how you convert that tacit knowledge into explicit. Sometimes you can’t, and this goes back to your original point about automating against a specification. I talk about this in my blog - sport and automation and why up to now we do not have automated referees for any sporting game.

We can take American football (soccer). You’re familiar with that one. If you go online, have a look for the laws of the game. They do not call them rules. They will be called laws, and there’s a particular reason why they are called laws. It’s the same reason why we have lawyers. The interpretation of those laws, in the same way the interpretation of requirements and interpretation of specifications, is open to whoever is reading it.

Ray:

Right. It basically comes down to the question of who wrote the parser of that law? How are you parsing that law out?

John:

Perfect example. You can be sent off in a game of soccer for ungentlemanly conduct. What does that mean to you? How do you interpret that? How does a referee interpret it? That’s where computers have great difficulty in social interaction and social behavior. We talk about this in automation.

That’s why automation can be used in this way. It connects together. You can use automation to support your testing. When you’re doing some testing you can use the testing to support your automation. It’s not one or the other. Each has a value to the business and to the team at a certain time. The skill of a tester is knowing which and when.

How are testers roles within organizations changing and how do both testing strategies (automated and manual) fit within that?

John:

The problem we have is testers no longer are testers. They are testing mentors. They’re the quality advocates for the team.It may mean they don’t do as much manual, hands-on testing. They’re there to support the rest of the team… Testers are normally noisy, vocal people, so they need to be more slightly more noisy and vocal.

Ray:

Definitely, especially if you’re at the point in your career where you know how to articulate these things and be more of an advocate. You can position yourself within a company or within an industry as someone who’s much more of a strategic thinker about these things.

This actually brings up another topic, which is the way people push off testing toward the end. They don’t typically value the role of testing or quality as much as other members on these engineering teams.

I, personally, perceive that’s starting to change now because companies are realizing the business impacts of making that relegation of testing to calling it QA or testing. They hadn’t been taking it seriously enough. Now they’re realizing that quality equals new revenue, new growth, and retention for customers and repeat visitors. It’s probably one of the most important things you can get.

Have you seen the shift in that in these last few years? If you have, I’d love to hear about what you think is behind that.

John:

Yeah. It’s quite interesting because I personally have never experienced it. Maybe it’s because of the kind of person I am. I think there is an onus on the people out there who are testers to stand up and say what their value is. If all you are doing is something that a machine can do, very well and a lot quicker than you, then you need to see where you are going to add value.

Going back to the point, do I see the role of testers improving? Only if they help themselves to do that. They need to stand up and say, where do I add business value? What is my value to this project, to this team, to this company? If they cannot answer that, then I’m afraid that the view that persists and it is still persisting, as you rightly said, is gradually getting less will continue. It’s up to the individuals.

We need more people at a higher level within these corporate companies to stand up and support testing as a skilled, very difficult profession. Not anybody can do it. Some people haven’t the mindset. Some people haven’t the capabilities. It’s different from being a coding person.

I’ve had lots of debates with people about should testers learn code. I don’t think they need to. They should learn syntax, learn a bit of scripting if you class that as code. I don’t know, but if you’re looking for someone to do automation, and they need to code, then get a developer to do the coding. It’s what I do. With teams I have, I know I have developers I work with who are far better and can write something quicker than I ever could. What they’d take an hour to do, it would take me two weeks.

Where’s the value in me doing that, when someone else can do it? To summarize this, when we talk about automation, we should start thinking about automation-assisted testing. What automation can we do, tool-wise, otherwise? We should use it to support the testing we are doing.

… I don’t think we’re just talking about software engineering. We’re talking about a global shift in the workplace. Work is changing because a lot of these roles have been automated. We know that. There’s now machines that in court can automatically do all the documentation for you. You don’t need a group of secretaries typing everything up. It’s automatically done now.

More and more of these roles are being squeezed out, so we’re moving away from a manual sort of working repetitive industry to more a knowledge industry where the sharing of knowledge becomes the value. That’s the biggest change.

What are your concluding thoughts on the future of automation and its place within organizations and software testing?

Ray:

With automation, and not just testing automation, but automation in general, every day there are new products that come out, and things that used to be very a routine part of people’s jobs are being automated away.

To figure out the next space of what work is going to look like for the next 50 to 100 years, we’re going to have to get back to whatever everyone needs to be given and get better at those things. The things that computers really no matter how smart they get can do (unless they pass the Turing test, but that is a different conversation).

That’s what’s interesting about automation - how it relates back to this psychology and question of what it means to be human and what is the role of humans versus technology in all of this? I think that’s going to be something for people to really rediscover in order to make sure that we have a healthy economy that can support people in the type of work they want to do.

John:

There’s a great film on that called “Bicentennial Man”. It talks about what it means to be human from an android point of view. That’s quite interesting.

To conclude on that, when we talked about automation and testing, and especially manual testing, I’ve looked at it from this point of view: Automation ‘checking’ and testing should be seen as partners working together rather than as enemies, and as separate things and separate entities.

Those testers out there need to embrace automation in a way that can support and assist their testing. Those automation people creating the automation stuff need to embrace being human. I think that summarizes the conversation for the last hour.