A Beginner’s Guide to Mobile Game Testing

Developing mobile games has become more enjoyable than ever thanks to advancements in gaming engines. Unity3D, for example, has the ability to generate cross-platform code. This trait addresses the heavy lifting required to optimize graphics natively on different platforms. The healthy eco-system of third-party assets (playmaker, anyone?) supplement Unity even further to create one of the most powerful development environments out there.

Despite the fact that Unity makes building for multiple platforms easy, it doesn’t obviate the need for the game developer to actually test their app on multiple devices. Rendering issues can still occur on various devices despite using dynamically sized viewports and leveraging all cross-platform capabilities of the environment. But it’s not just rendering issues. 

Apps can unexpectedly quit (i.e. crash) on certain device and operating system combinations. From the user’s perspective, this is an untenable bug. For the developer, the best consquence would simply be app abandonment by the user. In the worst-case scenario, your app gets a scathing review in the app store.

So, what can single developers, small teams, and even enterprise level companies do to ensure the success of their mobile apps?

Real-Device Testing is Essential for Mobile Game Testing

Testing on just the top few devices or just a single device actively harms your app’s success. But many developers – especially small teams and single creators – may be daunted by the added obstacle of robust testing. Additionally, the most mainstream tools may not be suited to their budget, needs, and app type.

Cloud based “real device” testing options exist, but many of them are plagued with latency (think Amazon). For testing “business applications”, this latency is often unnoticeable or an acceptable variable. Games, however, are an entirely different animal. The slightest latency will cause a drop in FPS, menu animation lag, and other problems.

In order to test mobile apps such as games, it’s crucial to identify why you need to test in the first place, develop a plan of attack, and adopt a system that allows for early bug detection and quick resolutions.

As with operating system-based programs, testing mobile applications requires thoroughness and consistency. Mobile games are no different and, in fact, have more to them when it comes to testing. Catching game-breaking bugs is important, but there are many different kinds of bugs that can negatively affect a user’s experience.

You can perform basic tests to ensure functionality, have other tests identify graphical glitches, and even conduct A/B testing to get user feedback on features for better fine-tuning. Let’s explore the common bugs in mobile gaming and what kinds of tests you’ll need to perform to up your mobile game app experience.

Common Bugs You Can Find with Mobile Game Testing

Localization Bugs

Popular apps need to operate in many different languages – some of which are character-based like Chinese instead of letter-based like English. Localization ensures that your app is accessible and understandable for a broader audience across the whole world. As such, it isn’t the easiest process to conduct effortlessly. The primary problems with localization include:

• Truncation – where a translation may be longer or shorter in another language
• Incomplete or Inaccurate Translation – missing text happens, but a segment that is badly translated or totally missing can mean the difference between having a happy or disgruntled user
• Inconsistent Terminology and Over Localization – certain things don’t need to change such as trademarks, logos, and even some words that remain the same in the language you are translating into
• Encoding Problems and Hardcoded Elements – some systems aren’t as flexible and cannot accommodate characters or move elements around when translating

Technical Bugs

Every user will have a different device with a different network connection. Technical bugs relate to when a user might have a poor internet connection, switches from a Wi-Fi connection to mobile data, etc. The external variable outside control of your app’s development team can cause issues for the user.

Device Compatibility Issues

If you haven’t tested your app across dozens of devices, you’ll likely have device compatibility issues. These issues can be as simple as graphical to as dire as crashes. Leveraging test environments that emulate many different combinations will allow you to catch compatibility issues early and avoid these cumbersome and annoying bugs.

Logic Bugs

Is your game not completing tasks and delivering expected results? Then you’ve got yourself a logic bug to be addressed. The best-case scenario is lagging or glitching with the worst scenario of this being a total app crash. Usually, what causes logic bugs can be inefficiently written code, poor resource management, using too much memory, and other culprits.

UI/UX and Interface Bugs

Glitches with User Interface (UI) and User Experience (UX) can be the quickest ways to make a bad impression on users. These kinds of bugs relate to anything that negatively impacts the user’s interaction with the app. They can include:

• Overlapping icons, menus, or other graphical elements
• Flickering logos or images
• Overflowing content, incorrect text-wrapping, or justification issues
• Elements not loading at all
• Animation lag

While not an exhaustive list, these bugs alone should be priorities as they can be addressed via testing. In order to catch these early and mitigate negative user experience, companies need to test their apps across many different devices.

Understanding the Game Development Cycle

The software development lifecycle (or SDLC) is the process by which a person or team develops, tests, deploys, and operates a program or application. Every SDLC differs in terms of scope, budget, schedule, etc. So, a “one size fits all” testing solution won’t work for mobile games; these apps have different issues to prioritize over things such as a bank app or music library.

Games have tons of animation, user inputs, effects, and possible combinations of elements to consider. Your test suite needs to be robust to save your team time and money while also working more efficiently and effectively. Here’s a basic rundown of what tests you should be running and why.

Types of Tests to Run

Functional Tests

You want your app to produce the expected results regardless of how a user interacts with it. From the simplest button press like opening a menu to more robust game mechanics, functional tests ensure that your app works as intended. They are a non-negotiable segment of a test suite and help teams identify bugs early to save time, money, and headaches in the process.

Functional tests can be automated and executed manually; both offer different benefits. Automation with Kobiton can be done without complex coding skills, create test scripts visually, and help testers better understand test results, too.

Visual Tests

Various devices all have different screen sizes, resolutions, color palettes, and more to consider. Running tests on a device can capture snapshots of the UI and compare them against the expected UI. Then, the test can validate the current UI state via the comparison and log any differences that need to be addressed.

Artificial intelligence and machine learning also plays a large role in the automation process for better visual testing in mobile games. Kobiton has integrated Deep Figma to further enhance UX testing, design, and maintenance.

Performance Tests

For mobile gaming, performance tests generally focus on evaluating game speed. After all, mobile games need to be responsive – not just to provide a sleek user experience, but to maintain quality from an aesthetic perspective, as well. Users who encounter lag in their mobile games have a higher chance of abandoning the app. Avoid that by conducting performance tests to monitor things like CPU, network, memory, device temperature, Energy drain, etc.

Making use of crash diagnostics can also be immensely helpful for developers – something Kobiton prioritizes along with 3rd party load test integrations.

Combinatorial Tests

These tests leverage multiple sequences of inputs to test for bugs and overall app performance. Historically, studies have found that more failures occur thanks to one or two parameters vs. three or more. Combinatorial testing allows teams to find faults more effectively as they cover a broader range with fewer test cases.

An example is hard to provide as combinatorial tests tend to be very specific and revolve around input parameter combinations. But you can identify these kinds of combinations and their variables for your app such as username and password.

Play Tests

Play tests are integral to the game design, development, deployment, and maintenance process. It allows users to provide direct feedback based on their experience with your current build and product. Additionally, though you can plan many thorough test case scenarios with the best QA team out there, users find new ways to challenge developers during play tests.

These opportunities allow you to re-evaluate your test suites and approaches while also engaging with your community. It’s a win-win situation: the users feel heard and appreciated while your team gets valuable and actionable feedback.

Tree Tests

Serving as a task-based method, tree tests involve hierarchical menus or nested branches users can navigate in your app. Think of it like how you might navigate a shop app on your smartphone: Menu > Account Information > Billing Address. The testing process is as below:

1. A developer creates a tree to be tested
2. Tasks are written for the tester to perform
3. Users attempt the tasks
4. Developers review their findings

One of the biggest benefits to tree tests is that they can be useful even when using a small number of testers. If you want quality or quantity, just design your test to your desired results.

Soak Tests

This non-functional testing method evaluates an app’s performance at peak load for a longer duration of time. The objective is to determine how the app performance at these high volumes and usage rates and to see where the weak points are. Tests like this expose things like memory and resource leaks, degradation, etc. It’s best to conduct these soak tests on mobile games with new releases, peak use periods, and if there is a major hardware change. With how many new phones come out every year, that can seem daunting. Thankfully, it is more of an incremental improvement than a monumental upgrade. 

Additionally, soak tests can highlight slowdowns or lags users might encounter during long gaming sessions. Reviewing hardware metrics can help your team better discern how to make the app run more efficiently.

Regression Tests

Every change you implement should be followed directly by regression tests. These tests help prevent old bugs from resurfacing with new changes. New code or logic changes can conflict with existing code, so a version control system can assist with regression testing, as well. Additionally, regression tests can include different types of tests such as the ones described above.

Functional, performance, security, UI, API, integration, end-to-end – all of these and more can and should be included in regression testing. Your team will be able to identify bugs quickly and deal with them just as swiftly with the right tests and tools.

How Kobiton Helps You Level Up Your Game

Kobiton leverages test environments that you can customize to your mobile game testing needs. With both manual testing and automated testing integration, you can address a variety of common mobile game testing aspects such as device fragmentation and varied network conditions and with Kobiton you can convert the metrics you get into actionable insights. Additionally, we have partnered with GameDriver to redefine excellence in mobile game testing.

The Kobiton and GameDriver partnership revolutionizes mobile game testing for developers and testers, merging advanced automation with real-device testing capabilities. This integration offers the unique benefit of enhancing testing efficiency, ensuring your games not only captivate with their superior quality but also excel in performance across all devices.

How it Works

Leveraging Kobiton’s Turbo Test Execution, our integration with GameDriver ensures high-speed, low-latency testing on real devices via our mobile testing platform.

Simply enable the Turbo Test Execution on your Kobiton subscription, integrate the GameDriver agent into your game app, and prepare your GameDriver test project. Then launch tests effortlessly using Kobiton API credentials and command line execution, blending Kobiton’s cloud prowess with GameDriver’s testing efficiency for unparalleled gaming experiences

Elevate Your Unity and Unreal Engine Game Testing

By blending GameDriver’s powerful automated game testing technology with Kobiton’s comprehensive mobile testing platform, we are facilitating efficient testing for Unity and Unreal Engine applications while also introducing Turbo Test Execution for GameDriver within Kobiton.  Our integration accelerates the mobile game testing process, reduces time-to-market, and significantly improves the player experience. 

Click here to learn more about how our partnership with GameDriver is establishing a new gold standard in mobile game testing. Or if you are headed to GDC 2024 stop by and see us! We’ll be in booth S1369, it’s in the South Hall against the wall!