Mobile App Testing on Real Devices: Types, Tools, and How Teams Get There

Mobile App Testing on Real Devices: Types, Tools, and How Teams Get There
Where you test

Where you test

Real devices. Private, public, cloud, or behind your firewall.
4 topics

What you verify

What you verify

Functional, performance, accessibility, security, and the advanced features your app depends on.
8 topics

How you test

How you test

Manual sessions to automation, with an AI layer on top.
5 topics

Where you test

Physical iPhones and Android phones with real OEM firmware, sensors, and constraints, plus the advanced features your app depends on: biometrics, image and audio injection, Bluetooth, and real phone calls and texts over a SIM.

“It worked in automation but failed on the real device. The bug only hits four percent of users, and they are all on one phone model we do not own.”

Real devices
Real devices

Dedicated, secure devices only your team can touch.

“We cannot upload this build to a public cloud. Security will not approve shared devices.”

Private devices
Private devices

A shared cloud of hundreds of devices for coverage you do not need to own.

“How do we cover every iPhone and Android model our customers carry without owning a drawer full of phones?”

Public devices
Public devices

Cloud, behind the firewall, or hybrid. What security can approve.

“Which model will our security team actually sign off on?”

The deployment spectrum

Two variables generate every model. Public, Private and On-Premises are the three you ask for; the tiers below are how they split.

Model

Where the portal runs

Where the devices sit

Best when

Public cloud

Shared portal
portal.kobiton.com

Kobiton data center
shared pool

No constraints; wants breadth and speed

Private cloud

Shared portal
portal.kobiton.com

Kobiton data center
dedicated pool, reserved 24/7

Wants exclusive devices, no wait times

Private cloud, dedicated

Dedicated portal
your-name.kobiton.com

Kobiton data center
dedicated pool

Needs an isolated portal + private path

Hybrid

Shared portal
portal.kobiton.com

Your premisesdedicated pool

Devices must sit on your network

Hybrid, dedicated

Dedicated portal
your-name.kobiton.com

Your premisesdedicated pool

Local devices + an isolated portal

On-premises

Customer-hostedyour own server

Your premisesstandalone or air-gapped

Data cannot leave the perimeter

Air-gapped is the fully disconnected form of on-premises. Hosting a dedicated portal in your own cloud account is a variation on the dedicated rows, not a separate model.

What you verify

Does it work, and does it work on the devices your users own.

“Every test passed on every device the team owns. Failures started coming in on a phone model QA does not have.”

Illustrative example

Compatibility coverage: your device bundle against your critical journeys

Build the rows from analytics, not a catalog. Green means proven on that device. Every red cell is a customer you would have shipped to.

Device in the bundle

Share of users

Login

Biometric login

Checkout

Push notification

iPhone 17iOS 27

24%

Proven on this device
Proven on this device
Proven on this device
Proven on this device

iPhone 15iOS 26

19%

Proven on this device
Proven on this device
Proven on this device
Proven on this device

iPhone 13iOS 26, last supported release

9%

Proven on this device
Failed on this device
Proven on this device
Not run yet

Samsung Galaxy S25Android 16, One UI

17%

Proven on this device
Proven on this device
Proven on this device
Proven on this device

Samsung Galaxy A16Android 15, mid-range, 4 GB

14%

Proven on this device
Failed on this device
Failed on this device
Proven on this device

Google Pixel 9Android 16, stock

8%

Proven on this device
Proven on this device
Proven on this device
Not run yet

Motorola Moto GAndroid 14, carrier build

6%

Failed on this device
Not run yet
Not run yet
Not run yet
Proven on this device

Proven on this device

Failed on this device

Failed on this device

Not run yet

Not run yet

Illustrative example. Device shares and results are hypothetical.

Response times, HAR capture, memory, CPU, network, battery, and temperature from real devices.

“It is fast on the developer’s phone. Is it fast on the mid-range Android our customers actually carry?”

Performance
Performance

VoiceOver and TalkBack on real devices, plus automatic IQS checks.

“A VoiceOver user could not get past our login screen, and we heard about it from the complaint, not the test run.”

Accessibility
Accessibility

Does it look right on every screen size and state.

“The test passed. The text wrapped off the edge of the screen on four devices.”

Visual and UI
Visual and UI

How the app protects data at rest, in transit, and on the device.

“Can a signed-out user retrieve cached account data or reuse an expired session token?”

Security

Can real people complete real tasks.

“Can a first-time user complete the app’s main task without help?”

Usability

Biometrics, image and audio injection, QR codes, camera, Bluetooth pairing, and SIM-enabled private devices for real phone calls and text messages.

“Our login is Face ID. Our workflow starts with a QR scan. How do you test that remotely?”

Advanced features

Crash logs, device logs, video, and API traffic captured for the session.

“The tester says it crashed. The developer cannot reproduce it.”

Crashes and evidence
Crashes and evidence

How you test

Manual to automated on Kobiton

One manual session on a real device is the starting point for everything else.

Manual session

Real device, in a browser

Lightning mode, 30 fps
Device in hand, anywhere
Private or public device
Every action is captured

Session capture

The exhaust from the session

Video recording
Every action and step
Device logs, API logs, HAR
Crash logs and metrics
Session Explorer timeline

Scriptless test

Session becomes a repeatable test, no code. Part of IQS.

Appium Script Generator

Session becomes a fully written Appium script, ready to run.

Device bundle in CI

Every build, every device, in parallel

Appium, XCUITest, Espresso, Maestro
Jenkins, GitLab, or straight from Xcode
Session artifacts kept for every run

The AI layer and the MCP server

Work from the tools your developers already use. Tell it what to test in natural language.

  • Claude Code
  • Copilot CLI
  • Gemini CLI
  • Codex CLI
  • Antigravity

Generates and runs

Generate a test from your project

Execute it on Kobiton real devices

Results and artifacts

Pull the artifacts, analyze, iterate

Flex Correct self-heals scripts

Inspect the generated test, then run it across your device bundle in CI.

Device in hand from anywhere, at up to 30 fps, with every action captured.

“Our testers are in three countries and the phones are in one drawer.”

Manual sessions
Manual sessions

Turn a manual session into a test and run it across a device bundle.

“We do not have automation engineers, but we need a smoke suite on every build.”

Scriptless test automation
Scriptless test automation

A manual session becomes a fully written Appium script.

“Automating this app took 18 months last time.”

Appium Script Generator
Appium Script Generator

Appium, XCUITest, Espresso, Maestro; Jenkins, GitLab, Xcode.

“We run automation on emulators in the pipeline and it does not catch what customers hit.”

Automation frameworks and CI/CD
Automation frameworks and CI/CD

Generate, execute, and analyze tests from Claude Code, Copilot CLI, Gemini CLI, Codex CLI, or Google Antigravity.

“Our developers already live in Claude Code or the Gemini CLI. Why should they leave it to test on a device?”

AI and the MCP server
AI and the MCP server
Where does your team sit on the Mobile Testing Maturity Model?

Size your device fleet

How many real devices do you need?

Assumes tests run independently and devices stay available for the whole window. Setup, cleanup, retries, and queue time are not included. Tests and spares are entered per test target.

Why real devices

Kobiton is real devices in a variety of deployment models: public cloud, private devices in the cloud, or behind your firewall. On top of the device lab sits manual and automated testing, and on top of that an AI layer that takes your team from manual to automated very rapidly.

Success Stories

Mobile testing has to account for the device, not just the app: OS versions, carrier networks, sensors, and OEM firmware all change the result. The app is one piece of the system, and many mobile defects come from how it meets the hardware it runs on.

Use both, but real devices decide the release. Emulators and simulators are cheap and fine early in development, but they cannot validate OEM firmware, physical sensors, camera, biometrics, or real battery, thermal, and memory behavior. Those flows need real devices before you ship, which is why Android’s own testing guidance treats on-device instrumented tests as their own layer.

There is no fixed number: start from the devices your own users actually carry, then add public cloud devices for the long tail of OS versions you do not need to keep on hand. The calculator above turns that into a baseline fleet size from four inputs.

Judge a tool on real device access, private versus public devices, framework support for Appium and the native frameworks (XCUITest, Espresso, Maestro), CI/CD integration, the session artifacts it returns, and how fast it takes you from manual to automated.