In today’s blog post, you’ll find out about the QA role here at Auroch, and the differences between being a QA Tester and QA Project Lead. You’ll also learn about the daily duties of a QA Tester, from smoke testing to compliance checks, to verifying, and everything in-between, as well as what kind of software and hardware is used by our department. Last but certainly not least, we’ll explain why QA is so essential to game development.

So, if you’ve ever wanted to become a QA Tester, or want to know more about what they do, this is the article to read!

Hi! I’m Tom Clatworthy, a QA Associate at Auroch Digital. I've been at the studio for nearly three years and have learned so much about Quality Assurance (QA) and the wider video game industry during that time.

At Auroch, I’ve worked across multiple games, with the most recent being Warhammer 40,000: Boltgun - Words of Vengeance – taking the lead on all things QA. If you’d like to learn more about the game’s development, we have written a great article on that very subject.

What is QA?

QA testing in the video game industry is about ensuring the delivery of a high-quality product that meets or exceeds the end user's expectations. Part of QA testing is finding defects in a game, ranging from game-breaking bugs and progress blockers, to minor visual issues, and defects that only the people who worked on the game would be aware of.

We evaluate a game’s mechanics, visuals, audio, and performance daily using multiple types of testing, such as regression, usability, and exploratory testing.

QA TESTING OUT DARK FUTURE: BLOOD RED STATES

Why is QA important?

QA is essential to ensuring a game’s overall quality before its release, requiring us to be very analytical in our work as well as having confident communication skills to concisely document defects we find.

In a lot of ways, QA represent the expectations of the people who buy and play our games, so part of our work includes playing the game just like someone would at home. Any issues we encounter get documented and processed into tickets so they can be investigated and fixed by the development team. I’ve already mentioned a few of these, but we use different types of testing to find bugs and defects, such as Smoke Testing (a form of “scripted” testing, in which you follow a set list of things to test) or Adhoc Testing (a form of “exploratory” testing, where you play through the game like a player, discovering and documenting bugs as you play naturally). Once an issue has been fixed, we follow-up to verify that the fix has worked. In a nutshell, then, we’re here to assess the overall quality of a game and provide visibility to the rest of the development team of the status of the game, overall, and ensure every team’s individual contributions work together as expected.

What does a typical day look like for a QA Tester, and how do we work?

At the start of each day, we attend a meeting to learn about what new features or elements have been added into the game. Sometimes, we also work with external QA teams, so part of that morning ritual would be to liaise with them and make sure everyone’s aligned on what to test.
Depending on the needs of the project, each day we might run through the daily Smoke Test which includes tests such as “does the game boot”, “do the splash screens show correctly”, “can you view the main menu screen” and so on. If all of these work as expected, they ‘pass’, but a ‘fail’ is any area of a game that doesn’t work at all.

As mentioned, if we encounter a bug or defect, it gets reported as an internal ticket, which can be investigated later by a member of the development team. This also provides information, so we know which area of a game is currently failing (particularly useful when we’re conducting a Smoke Test).

screenshot of warhammer 40,000: Boltgun - Words of Vengeance

What information do we put in a ticket?

Class of Bug

Firstly, we write a precise description of what the issue is and give a classification to the bug. At Auroch, this is defined as follows:

Class A – Game-breaking bugs which prevents a player from playing, such as a game crash.

Class B – An issue which disrupts the enjoyment of a player. This could be something like a broken pop-up window that cannot be dismissed and significantly obscures the player's visibility.

Class C - Where the player can notice an issue, but it won’t affect their enjoyment to the point of preventing them from playing. A spelling mistake is a good example of this.

Class D - Bugs that the player wouldn’t notice, but as developers know is not working as intended. For example, a text box is dark red, but the intent is for it to be a light red colour.

Repro

We would then write the ‘repro’ steps so that when someone else tests the issue, they can follow the same steps to reproduce it. A very basic example of this could be:

Step 1: Boot the game

Step 2: Select level 1

Step 3: Pick up the gun

In the ticket title, we also include the ‘repro rate’ (how many times we tried it, and how many times the issue occurred). So, if a bug occurs every time you try it, and you try it three times, we would put “3/3” (it’s happening three times out of the three times we tried it). If we’re unable to replicate the issue once encountered, we will mark the task as such. Trickier scenarios are when the repro rate is, say, “2/3” - we can replicate it following our steps, but not every time, which means there is likely some nuance we’re missing.

Evidence

An additional step is to attach useful evidence of the issue, such as a screenshot or video of the defect in question, as well as logs (technical records of events in the game engine) or save data.

Verifying

Once a defect or issue has been fixed, it then needs to be verified by us. We follow the repro steps outlined in the ticket to see if the problem is still occurring.

If the issue is no longer occurring, we make a note and close off the ticket. But! If the issue persists, it is noted as a failed fix, and the ticket becomes an ‘unrefined’ task, so that it can be discussed at a future date.

The team working on brewmaster: beer brewing simulator

What is a QA Project Lead at Auroch, and what do they do?

As a QA Project Lead, you are also the main point of contact for the rest of the studio and our external partners, so you need a vast knowledge of the game and a broad overview of what the other departments are working on, as well as all things that are going into the game daily. As a Project Lead, you’re also a liaison between the internal and external teams, ensuring they are aligned.

While a QA Project Lead often has similar duties to a QA Tester, there are additional role requirements: Reviewing changelogs daily and seeking information on any changes going into the game that we don’t understand.

Publishing internal changelogs (along with additional explanations for fixes) in communication channels, so that everyone, including the external QA testers we work with, can get a full overview of what’s going into the build that day.
Coordinating and planning with the external QA and what they are focused on.
Representing QA in meetings, particularly in the ‘Backlog Refinements’ meeting, where various disciplines work their way through unrefined tasks to understand more about them, when they should be done and who might address them.

What sort of tools do you use?

Some of our tools are universal across all projects, while others depend on the game.

On PC, we use Steam to distribute builds for testing, but on Xbox, PlayStation and Nintendo, we use special hardware, known as “devkits” provided by those companies.

For capturing game footage, we often use OBS as the software has lots of settings for video quality, file size, and more. It can also have overlays added to the recording, which capture our keyboard and mouse or controller inputs at the same time, which is helpful for knowing which inputs we were using when an issue occurred.

There’s even a handy feature that allows us to record the last three minutes of gameplay – perfect when a pesky issue suddenly appears and we need to capture evidence of it.

How does QA fit within the development cycle?

It goes without saying, but QA is important throughout the whole development cycle – and not just when the game launches! QA should be involved in a project from the early stages, all the way through to the release of the game and beyond.

Being able to see the game through the eyes of a player from the earliest stages, allows QA to bring up ideas and opinions about the game they think players might have, addressing issues before they become issues.

We can also think about development tools we would find useful and bring value to the development for later testing. The earlier we define these, the better – giving us time and budget to get them implemented.

For example, on Warhammer 40,000: Boltgun – Words of Vengeance, I brought up that I was not the fastest at typing, so I would need to have a “God Mode” in the dev tools so that I wouldn’t always die when testing the harder sections, and that, as a player, I would prefer to have different difficulties in the game so I could play and enjoy an easier mode – initially the game only had one difficulty setting which I found very challenging to complete.

QA are vital once a game launches, too. Alongside player reports, this is usually where DLC, patches and hotfixes are added to the game, and all of that needs QA testing.

One of the reasons patches are so essential nowadays is that QA testers at a studio can only test the game on so many combinations of PC specs, but once the game has gone into the world and is being played by so many people (on so many different machines and setups), some of those players might notice issues we couldn’t have found without their exact setup.

After finishing a project, we run a retrospective, discussing what went well, and what didn’t. Every team and discipline is involved to ensure we get different perspectives of the development lifecycle. This is a crucial meeting, and we learn so much from each other during this discussion, which we can then take forward with us into our future projects.

QA are part of the project pre and post-launch

To conclude...

It’s a common misconception that QA just plays video games all day, so I hope this article has revealed more that goes into the myriad of activities a QA Tester does, and skills they have - from routine daily duties to the ways we align as a department.

I’m a massive fan of video games, but I never knew how to get into the industry until a few years ago, when I discovered the role of QA Tester. Ever since starting the job, I've absolutely loved it! Being able to view a game from the player’s perspective helps us understand what is needed to ensure the game is at the highest quality possible when the game launches.

If you were to ask me my favourite part of the job, I would tell you it’s when I spot an unusual issue that no one else could find and try to figure out how to make it happen again. It’s like solving a puzzle!

Thanks for reading!

Blog

The Life of a Video Game QA Tester - Auroch Digital Blog