April 2021 Product Update: Better ways to create Appium scripts & more
With over 4,000 mobile device types, over 100 mobile operating system versions, and hundreds of carriers around the world, you would have to test your mobile app with hundreds of millions of device configurations to account for every scenario. Obviously, that’s impossible. And yet, you want to test your apps on as many real mobile devices as possible to make sure your app works flawlessly for the most people.
For developers, this always poses a big problem. How do you know when you’ve tested on enough mobile devices? While your time and budget may seem to severely limit your testing options, free and paid mobile testing tools exist that allow you to test apps on more real devices than you previously thought possible.
These four tips will help you narrow down your number of device configurations while also making sure you’re not unnecessarily limiting your testing options.
First, it helps to gather information about what devices people are likely to use by performing some basic market research.
In addition to your market research, keep an eye on lifecycles for devices and operating systems. For example, Android’s Nougat update, released in August 2016, has had a slow adoption rate (currently on only 11.5% of active Android devices). By contrast, iOS 10, released in September 2016, is currently on 86% of Apple devices. This means you may focus on testing Android devices more comprehensively across past operating system versions than with iOS.
And for specific kinds of apps, you may eliminate certain device configurations completely. For example, working with Android Instant Apps eliminates any need to test Android devices with an operating system before version 6.0.
Once you’ve whittled down your list of device configurations a bit, you may still have a large list. But before whittling that list further out of time and budget considerations, it’s good to look in depth at a variety of mobile testing tool options such as simulators, emulators, and real devices.
Clear differences emerge when you realize that simulators only mimic software—not hardware. Or that emulators have a long setup process and can work very slowly. And to help you examine of the cost of testing real devices, we offer a calculator that gives you a sense of both device and people costs.
Cloud mobile testing options have revolutionized the availability and scalability of testing apps on real devices—allowing you to test many device configurations. However, with so many cloud mobile testing providers emerging, it’s good to know that they are not all created equal. If you don’t make a careful assessment early on in your evaluation process, you increase the chance of running into unexpected (and expensive) problems.
To help you assess cloud mobile testing providers, our recent blog post (How Much Should You Be Paying For Real Device Testing?) goes into detail about four factors to consider when evaluating the cost of real device testing:
At Kobiton, we accommodate these four factors by offering a large combination of devices—including the latest Android and iOS devices—and operating systems for accessing and testing Android and iPhone apps on real phones. Plus, developers and businesses can use our cloud to manage their internal devices from anywhere in the world. Our platform helps scale your investment when you purchase a device and allows your team members to increase productivity by giving them access to devices regardless of location.
Today, we’ve only covered a few of the challenges with mobile testing. Continue reading our blog and white papers to learn best practices for implementing your mobile testing strategy at scale.