4 Tips for Better Mobile App Testing

With over 4,000 mobile device types, over 100 mobile operating system versions, and hundreds of carriers around the world, you would have to test your mobile app with hundreds of millions of device configurations to account for every scenario. Obviously, that’s impossible. And yet, you want to test your apps on as many real mobile devices as possible to make sure your app works flawlessly for the most people.

For developers, this always poses a big problem. How do you know when you’ve tested on enough mobile devices? While your time and budget may seem to severely limit your testing options, free and paid mobile testing tools exist that allow you to test apps on more real devices than you previously thought possible.

These four tips will help you narrow down your number of device configurations while also making sure you’re not unnecessarily limiting your testing options.

1. Know your audience. 

First, it helps to gather information about what devices people are likely to use by performing some basic market research.

• What’s the general popularity of specific devices? Many resources such as Android Developers, Apteligent, and NetMarketShare are available that can help you narrow down your devices by operating system, type of device, manufacturer, and form factor based on popularity and use. 
• How do your audience demographics narrow down the number of mobile device configurations to test? Mobile analytics and secondary research about your target audience’s demographics can help you eliminate device configurations. For example, if your audience is tech-savvy and likes to use cutting-edge devices, then they may not use many older devices or operating systems. Especially examine regional data and trends if you target users outside the United States so that you don’t overlook popular devices in various countries and regions. 
• Is your audience still using older devices and operating systems? It’s often more exciting to focus testing on new and future devices while overlooking older devices and operating systems that your audience still uses. Will your app work with older devices that seem obsolete to you but that many of your users still own and operate?

2. Consider device and operating system lifecycles.

In addition to your market research, keep an eye on lifecycles for devices and operating systems. For example, Android’s Nougat update, released in August 2016, has had a slow adoption rate (currently on only 11.5% of active Android devices). By contrast, iOS 10, released in September 2016, is currently on 86% of Apple devices. This means you may focus on testing Android devices more comprehensively across past operating system versions than with iOS.

And for specific kinds of apps, you may eliminate certain device configurations completely. For example, working with Android Instant Apps eliminates any need to test Android devices with an operating system before version 6.0.

3. Understand your mobile testing options.

Once you’ve whittled down your list of device configurations a bit, you may still have a large list. But before whittling that list further out of time and budget considerations, it’s good to look in depth at a variety of mobile testing tool options such as simulators, emulators, and real devices.

Clear differences emerge when you realize that simulators only mimic software—not hardware. Or that emulators have a long setup process and can work very slowly. And to help you examine of the cost of testing real devices, we offer a calculator that gives you a sense of both device and people costs.

4. Understand what cloud mobile testing on real devices entails.

Cloud mobile testing options have revolutionized the availability and scalability of testing apps on real devices—allowing you to test many device configurations. However, with so many cloud mobile testing providers emerging, it’s good to know that they are not all created equal. If you don’t make a careful assessment early on in your evaluation process, you increase the chance of running into unexpected (and expensive) problems.

To help you assess cloud mobile testing providers, our recent blog post (How Much Should You Be Paying For Real Device Testing?) goes into detail about four factors to consider when evaluating the cost of real device testing:

1. What testing is included in the package?
2. What kind of devices are included?
3. What are the purchase options for testing minutes?
4. What will your access to the devices look like?

At Kobiton, we accommodate these four factors by offering a large combination of devices—including the latest Android and iOS devices—and operating systems for accessing and testing Android and iPhone apps on real phones. Plus, developers and businesses can use our cloud to manage their internal devices from anywhere in the world. Our platform helps scale your investment when you purchase a device and allows your team members to increase productivity by giving them access to devices regardless of location.

Today, we’ve only covered a few of the challenges with mobile testing. Continue reading our blog and white papers to learn best practices for implementing your mobile testing strategy at scale.

To learn more about Kobiton, click here, or get started with a no-commitment free trial today.