The following blog post could be demonstrated in the real world with a short field trip to your local Ikea.
Let me start by saying: I love Ikea. When the one in the Denver area opened, I waited in line to be one of the first in the door.
Now, if you’ve ever gone to the iconic chair section at Ikea, you’ll likely have seen something like this:
Pretty great, right? There’s this robot whose job is to stress test their POÄNG Chair™. This robot tells you the chair is robust and will stand up to any amount of sitting and standing.
And while I do care that my chairs last a long time, at the end of the day, the thing that matters most to me is the chair’s level of comfort. I happen to find these chairs comfortable; however, many people who visit my house are in hearty disagreement. Why? Because of people’s different body sizes and shapes, as well as their personal preferences, their use of a chair is different.
In the past few years, many companies have adopted an increased interest in automating all of QA, and trusting that their customers will report failures when they occur—often to just put out a new feature instead of fixing the problem.
Don’t get me wrong, new features can breathe life into the user’s experience in your application. But at the end of the day, if you didn’t like sitting in the POÄNG Chair™, you’re not going to be happy playing games in it, even if it’s now more convenient.
Some companies have a monolithic application with plenty of automation, but the user experience leaves a lot to be desired. These applications end up confusing, and no one knows how to use them. From this is born an entire new job title—the [SaaS Product Expert], who attended classes to understand how to use the application correctly. The rest of us curse the fact these applications are pretty much the best we can hope for in their corner of the market, but they’re still full of bugs and frustrating to use.
Now, I come from a background making video games. Most often in video games, if your game is too complicated to play, people won’t bother. So, game designers, UX experts, and UI artists strive to hook you with some quick win and by showing off the cool features of their game. If the game has too many bugs or is too complicated to use, it won’t make it.
When I jumped to the world of making web applications, I was astounded at the choices people made and the focus many companies had on automation, while failing to consider the user’s experience. Thankfully, not all web applications are driven by these motives.
I once worked with a VP of product who had remarkable insights into UX and the importance of making a good impression on your user—be they the CEO, or just a casual person interacting with your product. He once told me he strove to make sure his users were never disappointed and often delighted to use the application he worked on. And, you know what? It showed through.
But, having people who design good UX isn’t always enough. You also need someone in the trenches, looking at your application as a user would. Be sure your users are delighted. The chair robot cannot gauge the delight (or chagrin) of the person sitting in the chair, it can only tell you if the chair functions as expected. This is a valuable step in the process, but you cannot forget about how it feels.
Consider our chair again for a moment. I buy an Ikea chair, I get it home, and it’s awful. What is my recourse as a customer? Well, at the very least, I can return the chair. This isn’t really something users can do with most SaaS products when they find a bug. Monthly subscriptions to these products usually cost more than a few chairs, yet customers feel OK forgiving the bug and continuing to use the product because it solves a problem. Most of those same people wouldn’t keep an uncomfortable chair just because it solved the problem of no longer wanting to stand.
Manual QA is all about doing things to your software which the engineers and designers hadn’t considered. It’s about sitting in the chair, and seeing if it’s good for watching TV, chatting with friends, or playing video games. They might stand on it, or tip it upside down, or slide it across the floor. The things a good exploratory tester can think of to do to this chair will boggle the layperson’s mind. And yet, I’ll bet some regular chair user (or their bored teenager) has already tried it.
But, your manual QA can also answer the simple question, “is it comfortable?”, as well as expounding on what’s comfortable (or not) about the chair. They can compare it to other chairs and give insights into what might be done to improve the chair. So, when the chair hits the Ikea floors, not only would they know the chair could stand up to robotic sitting for hours, but also a person playing an especially engaging game of FIFA.
While manual QA might not be as fast or exact as test automation, it too plays an important part in making quality software people enjoy using. There should be a harmony between test automation and manual testing when ensuring a quality experience for any software product which people are using, just as there should be a harmony between robots and humans sitting on IKEA furniture.