Performance Testing in Mobile

Is your testing team overlooking critical forms of testing?

Mobile app software testing often revolves around hitting a handful of benchmarks that are seen as priorities to getting the app off the ground and on the market. This tends to have a slant towards functional testing first and foremost, concentrating efforts on ensuring that a new mobile app is capable of handling the baseline requirements to function in specific and expected manners.

However, a critical non-functional form of testing shouldn’t be overlooked by testers that want to determine the ability of their product to execute under a wide scale of scenarios, including strategizing a product’s capability of handling a large user presence. This is known as performance testing, a mobile application software testing strategy that focuses on aspects of app stability, scalability, speed, and responsiveness given a specific workload placed on the app. For many testing teams, performance testing is one of the final steps in the testing cycle, executed once code is finalized and ready to release.

For a successful testing team, setting key goals when performance testing that evaluate the output of an application, the app’s processing speed, network bandwidth usage, as well as metrics like the maximum number of concurrent users, will ultimately serve the team to accurately determine expectations for the app to perform under a range of possibilities so that post-market delivery is seen with as few surprises as feasible.

With that in mind, we’re going to look at the various types of performance testing, whether or not tests need to be automated, and what the differences are between performance and load testing so that a proper understanding of performance testing can be reached and its benefits can be weighed by your testing team.

Types of Performance Testing

Performance testing requires a range of steps to execute, depending on the type of performance test utilized. Teams will generally identify the test environment and tools available to them, make a determination about acceptable performance criteria goals, plan and design the tests, prepare a range of test environments and tools, run the tests, and then meet with dev teams to resolve any issues and retest.

Within these parameters, we can examine four different types of performance tests to direct our attention towards. These tests are…

• Capacity testing: Similar to an elevator weight capacity, this form of testing examines how many users can be on the app at a time without performance levels dipping below acceptable levels
• Soak testing: Much like the name implies, soak testing imagines how an app will be functioning once it’s “soaked” in high traffic for an extended period of time. This type of test will simulate high bandwidth environments and their impact over hours and hours on a network, determining choke points within the code of the app.
• Stress testing: Not dissimilar from the revolutionary testing concepts used by restaurant industry pioneer, Jon Taffer, on his hit reality television series Bar Rescue, to intentionally push the limits of bar and restaurant staff by overwhelming them with patrons at a single time, stress testing is a form of performance testing designed to measure how many users can operate on a mobile app before it completely crashes.
• Volume testing: Similar to capacity testing, volume testing measures the limits on data that can be processed at a single time without slowing down the servers or losing data along the way.

Familiarity with these types of tests will enhance a testing team’s ability to run performance tests that truly determine the rigors of their mobile application. With that in mind, let’s turn our attention to how the question of manual versus automated testing factors into the performance testing realm.

Manual vs. Automation in Performance Testing

The question of manual versus automated testing always springs up in any testing discussion. They both have their place in a testing environment, and performance testing is no different.

Typically performance testing falls under an automated umbrella. Most capacity or volume testing requires incredible amounts of user and data simulations that aren’t meant to be conducted under the auspices of individual, manual users. Automation has made most forms of performance tests possible when they weren’t before.

There are cases where manual tests may resolve testing concerns. For example, a tester may run a quick sanity test to see how an app page loads or to measure response times. These short-window tests that aren’t reliant on computing power can be cases in which manual testers may find that it’s just easier to set up and execute a performance test in a manual setting.

Differences Between Performance & Load Testing

If you’re familiar with load testing, you may be asking yourself at this juncture how it differs from performance testing. They seem fairly identical in what they’re prescribing a tester to do, but they are different concepts for a tester to understand.

You can think of performance testing as a more categorical term that explains a variety of test types, from capacity testing to load testing. It’s an umbrella term for any type of test that evaluates the system’s overall performance and examines a multitude of metrics, from response times to mobile app stability.

This all-encompassing concept includes load testing, which is a specific mobile app testing strategy that verifies whether a mobile app will be able to handle an expected load. This specific testing strategy is typically employed by testers to survey the mobile app for potential bottlenecks in the code, identifying areas where heavy user traffic can slow the app to a screeching halt. This obviously has a tremendous effect on the overall performance of an app, and so we consider load testing to be a vital strategy, along with the four aforementioned performance testing types, in measuring mobile app performance.

With this assessment of performance testing in mind and an understanding for how specific performance tests are used to enhance the efficiency of a mobile app, testers can optimize how they view their post-coding testing prior to the app hitting the market. Performance testing is a vital, though often overlooked, component to a healthy mobile app launch. Implementing these strategies into your mobile app testing routine will result in more work on the front-end, but fewer headaches post-launch and a better user experience, helping testers sleep at night knowing their app is flexible and malleable to an explosion in popularity and user base overnight.