28 Jun The myths of mobile testing
Mobile app development and testing have matured dramatically in just the last few years. Cross-platform development tools have matured, automated testing frameworks have emerged, and the ability to test on real devices has become more accessible and affordable.
And while we’ve seen this general maturity, and resultant increase in quality, in mobile development and testing, two common misconceptions remain.
Myth 1: “Testing on 2-3 real devices is good enough”.
The first common misconception is that testing on just the 2-3 latest devices is “good enough”. It’s easy to fall into this trap. However, as the chart below shows, achieving proper market coverage requires testing on many more devices:
As depicted above, testing on 28 devices only yields a market coverage of 70%. That means your app may not be functioning properly, and may even be crashing, for 30% of your addressable market.
To demonstrate the extent of device fragmentation, moving to 58 devices still only gives 80% coverage. Testing on a whopping 159 devices puts you at 90%.To reach 95% of your addressable market means testing on 359 device!
So, you still want to only test on 2-3 devices?
For the disbelievers, let Kobiton show you how your current app really functions on a large swathe of real-devices. With our unique “health-check”, we’re able to show you what percentage of the market is being alienated by bad app behavior. Even better, we’ll show you exactly on which devices its crashing. Request your free app health check here.
Myth 2: “Real-Device testing is expensive, complicated and slows down our release cycle.”
During the nascent days of mobile testing, this was likely a legitimate complaint. Trying to procure and manage a device farm was cumbersome and expensive. And worse, it was hard to integrate the device farm into your DevOps processes for automated test execution and deployment.
Thankfully, modern mobile testing platforms have alleviated these constraints. Managing a device farm is as easy as using any cloud-enabled service, shared devices can be utilized to bring down the cost, and the platforms integrate cleanly with DevOps and CI/CD processes.
It has never been more convenient or affordable to incorporate real-device testing.
Can you really afford not to incorporate real-device testing?
Clients are often surprised when they see the results of our health-check. Organizations are often unaware that no only do their apps not work as expected, but at the high percentage of users affected by those crashes. These crashes or UI issues are rarely reported directly to the organization. Even worse, they're reported indirectly via bad app store reviews and high abandonment. In today's hyper-competitive environment, you really can't afford to not be testing on real devices.
Why not try it out? Request a free health-check of your app.