How banks test mobile apps on real devices securely

Every mobile security lead has the same moment. The build passes on the emulator, the scan comes back clean, and then someone runs it on a physical iPhone with Face ID enabled and a real carrier network. Suddenly the Keychain behaves differently, the biometric prompt fails a step-up flow, and the device integrity check returns a verdict nobody planned for.

Emulators don’t share the threat model of real hardware. Secure keystores, biometrics, device attestation and live network conditions only show up on physical devices. That’s why banks and financial-services teams test there, and why the device lab itself becomes something you have to secure.

Here’s the approach I’d use to do both: test the app hard, and keep the lab from becoming a new risk. For more on the hardware gap, see real devices vs emulators.

Start with OWASP MASVS and MASTG so every test maps to a control

Two OWASP documents do most of the work. The OWASP Mobile Application Security Verification Standard (MASVS) defines what “secure” means. The OWASP Mobile Application Security Testing Guide (MASTG) explains how to verify each control. MASTG is maintained as a live web resource, with its v2.0.0 refactor moving away from a static checklist, so link your test cases to the current pages instead of a stale spreadsheet.

Three MASVS areas matter immediately for banking apps:

• Storage. MASVS-STORAGE-1 states, “The app securely stores sensitive data.”
• Cryptography. MASVS-CRYPTO-1 requires the app to employ “current strong cryptography” used according to industry best practices.
• Key management. MASVS-CRYPTO-2 says the app “performs key management according to industry best practices.”

For governance, NIST SP 800-163 Rev. 1, “Vetting the Security of Mobile Applications” (published April 2019, superseding the 2015 baseline), gives you the planning, testing and approval mindset. It covers how to plan a vetting process and develop security requirements for evaluating apps. On the payment side, PCI DSS applies pressure too. Requirement 6.3.3, for example, expects critical and high-severity patches deployed within 30 days of release, which means your regression evidence has to be fast and repeatable. For a deeper treatment of how scans become proof, read our guide to mobile app security testing from scan to proof.

Treat the device lab as in scope

A device lab is infrastructure that handles your builds, credentials and screenshots. Put it in the threat model. The surface includes test runners, device pools, uploaded builds, logs, videos, screenshots, and the network paths to staging backends.

AWS Device Farm’s own documentation is a useful reference here. It publishes security best practices covering network isolation and access controls, and it notes that data can persist between sessions in some cases and needs appropriate handling. Whatever platform you pick, ask the same questions.

Baseline controls to expect:

• Session isolation and a reset between runs, so one tester’s data never reaches the next session.
• Least privilege access, with roles scoped by team and app.
• Encryption in transit for every path between runners, devices and staging.
• Artifact retention rules that say who can view logs, videos and HAR files, and when they’re deleted.
• Deployment choice. Cloud works for many teams, and on-prem fits stricter data residency needs. Our comparison of cloud vs on-premise device lab management covers the trade-offs.

Then write down the “do not” list. Never use real customer or production accounts. Never log secrets or tokens. Never leave sensitive screenshots and recordings unmasked. Synthetic test data is the rule, and it’s the heart of secure test data for banking apps.

Run six security checks on real iOS and Android devices

Map each check to a MASVS theme so the result doubles as audit evidence.

1. Storage and backup safety. Confirm tokens and account data land in the iOS Keychain or Android Keystore, not in plain preferences, logs or backups.
2. Cryptography and key management. Verify keys are hardware-backed where the device supports it, and that the app rejects weak algorithms.
3. Authentication, MFA step-up and session management. Test step-up prompts for high-risk actions, biometric fallback, and session invalidation after logout, timeout and password change.
4. Network protection. Validate TLS and certificate pinning, and confirm the app fails closed when a proxy presents an untrusted certificate. Exporting a HAR file helps with network traffic validation and review.
5. Device integrity and anti-tampering. On iOS, Apple documents DeviceCheck and App Attest as services that help reduce fraudulent use of your services. Apple’s documentation also states that not every device supports App Attest, so you need physical hardware to see real behavior. On Android, the Google Play Integrity API supplies integrity verdicts that your app sends to your backend for a decision.
6. Supply chain. Verify third-party SDKs and dependencies against an approved list before each release.

One rule covers the integrity checks: the backend must make the risk decision. A client-side check that an attacker can patch out isn’t a control. Your test should prove the server rejects a bad attestation or integrity verdict, not just that the app displays a warning.

Make it repeatable so security doesn’t become the bottleneck

A one-time pen test ages out the day you ship. Treat security as secure regression, tied to every release, and the 30-day patch expectations in PCI DSS stop feeling like a scramble.

The bottleneck is usually authoring. Teams can’t script every flow by hand, so security cases get skipped. Record a validated flow once, such as a biometric login or a step-up transfer, and turn it into reusable automation. Kobiton’s Appium Script Generation converts recorded manual sessions into Appium scripts, and tests can run in parallel across selected real devices. Session Explorer then lets engineers replay a failed run and find the defect with logs and device context.

Keep the evidence clean. Use synthetic data, sanitize artifacts before they leave the lab, and record only results you can verify. An audit trail that shows what was tested, on which device, by whom, and when is worth more than any claim about speed.

Write the policy before the next release

The teams that do this well don’t have a secret tool. They have a short, agreed document: which MASVS controls get tested, what data is allowed, who can see artifacts, and how long they live. Security, QA and compliance should all sign it.

Draft that policy this week, then pressure-test it against your next release candidate on real devices. If you’d like to see how a real-device lab fits into that workflow, explore Kobiton’s real device testing best practices and bring your security team along.