Article

Mobile Accessibility Testing: What Automation Still Misses

14 min read
Mobile Accessibility Testing: What Automation Still Misses

Mobile accessibility testing verifies that a mobile app can be used by people with disabilities, including people who rely on assistive technology, adjusted display settings, alternate input methods, captions, clear navigation, or extra time to complete tasks.

But mobile accessibility testing is not only about whether an app passes a checklist. It is about whether a person can perceive, understand, navigate, and complete a task when the screen, controls, audio, gestures, text, or navigation do not work for them in the default way.

Automated accessibility checks can identify important technical issues, such as missing labels, contrast failures, touch target problems, and some structural issues. They cannot fully determine whether the mobile experience works for a real person using real device settings and assistive technology.

WCAG 2.2 provides the central accessibility framework many teams use, organized around making content perceivable, operable, understandable, and robust. But WCAG also notes that following the guidelines does not address every user need. Accessibility testing has to account for that gap.

WCAG begins with the principle that information and user interface components must be presentable to users in ways they can perceive. That principle is simple, but mobile testing makes it complicated. A tool may confirm that content exists on the screen, but people perceive content through vision, hearing, touch, assistive technology, device settings, context, and cognitive load.

Apps are not designed for checklists. Apps are designed for people.

Accessibility is more than compliance

Accessibility is more than compliance

Who are mobile apps designed for?

For many users, the obvious answer is “me.” The app fits their hands. The text is readable. The colors make sense. The gestures feel natural. The buttons are easy to tap. The audio, motion, and navigation all work the way they expect.

For many other users, the answer is often “not me.”

A person may have low vision, color blindness, limited mobility, hearing loss, tremors, chronic pain, cognitive differences, temporary injuries, or age-related changes in vision, dexterity, or memory. Someone may use a screen reader, voice control, switch control, magnification, larger text, captions, reduced motion, or one-handed navigation. Someone may simply have larger fingers, a cracked screen, a small device, or a hard time adapting to a new interface.

Those users should not have to fight their apps too.

Accessibility standards matter. They give teams a shared way to identify barriers and improve app quality. WCAG guidance is often used as a foundation for digital accessibility, and mobile accessibility guidance helps teams apply accessibility principles to mobile web, native, and hybrid apps. But compliance is not the whole experience.

A screen can meet several technical requirements and still feel difficult to use. A button can have a label and still appear in an illogical place. Text can technically resize and still break the layout. A flow can pass automated checks and still become exhausting for someone using a screen reader, switch control, voice control, magnification, or one-handed navigation.

Mobile accessibility testing should help teams find those gaps before users do.

Accessibility is also a market service

Accessibility is often discussed as a compliance requirement, but it is also a way to serve more people well.

Accessible apps can reach users who are too often left out of design decisions. That includes disabled users, aging users, users with temporary injuries, users in distracting environments, and users whose needs change from one moment to the next.

A puzzle app may pass a checklist, but can a great-grandmother enlarge the text, understand the controls, and play with her granddaughter from half a continent away?

A banking app may have labeled buttons, but can someone using a screen reader move through the transfer flow without confusion?

A map app may display the right route, but can someone with low vision actually follow it?

These are not abstract questions. They are the moments where mobile apps either welcome people in or remind them that the product was not built with them in mind.

Accessibility helps users feel considered. It builds trust. It expands who can participate. It makes the app better for people who already experience unnecessary friction in the physical world and should not have to encounter more of it on the device in their hand.

Customer examples show why automation is only part of accessibility testing

In one healthcare customer conversation, a small mobile team described using Kobiton while completing WCAG 2.1 accessibility testing across multiple patient-facing mobile apps in a matter of weeks. The team used automated findings to identify non-compliant items, then separated issues they could address in the app from issues tied to third-party web shells or platform dependencies. That split helped the team assign ownership, validate fixes, and understand which accessibility issues required deeper follow-up beyond the initial automated finding.

That distinction matters. Automated checks can help teams find and track accessibility issues faster, but human review is still needed to decide what the issue means, who owns it, and whether the user experience is actually improving.

In another customer conversation, a digital accessibility engineer compared screen reader behavior on a handheld device with behavior during remote device testing. The team noticed possible differences in the screen reading flow, including focus movement that seemed to skip parts of the interface, and planned to retest and record the behavior. A scanner can flag labels, contrast, or touch target size, but a person still has to verify whether the focus order and assistive technology experience make sense in the actual flow.

What automated accessibility checks can catch

Automated checks are useful for issues that can be evaluated against clear technical rules. For example, tools can help identify contrast problems related to WCAG 1.4.3 Contrast Minimum, touch target concerns related to WCAG 2.5.8 Target Size Minimum, and focus-order issues related to WCAG 2.4.3 Focus Order.

They can help identify issues such as:

  • Missing accessibility labels
  • Low color contrast
  • Small or crowded touch targets
  • Images or icons without useful descriptions
  • Form fields without clear labels
  • Inconsistent heading or navigation structure
  • Elements that may not be reachable by assistive technology
  • Layout issues caused by text scaling

These checks are valuable because they reduce obvious barriers. They also help teams find repeated issues across screens, builds, and releases.

They can also catch things a non-disabled tester might not think to check at first, not because that tester is careless, but because people move through technology in different ways. A person who does not rely on a screen reader may not immediately notice a confusing announcement order. A person who can easily tap small controls may not notice how difficult those controls are for someone with limited dexterity. A person who can distinguish certain colors may not notice that color is doing too much of the work.

Those checks matter. But they do not answer every accessibility question. A technically valid focus order still needs to make sense in the user’s task flow. A target may meet a minimum size and still be hard to use in context. A contrast ratio may pass while the experience remains difficult under real mobile conditions.

What mobile accessibility testing still needs from humans

Automation can identify many technical accessibility issues, but it cannot always understand context.

It may know that a button has a label. It may not know whether the label makes sense in the flow. It may know that text exists. It may not know whether the user can understand what to do next. It may know that an element is technically reachable. It may not know whether reaching it takes too many gestures or sends the user through a confusing path.

A test can detect the “Pay now” button, but can a person find it, understand it, and activate it confidently?

What if the button depends on two similar shades of green? What if the text is technically present but too small on a particular screen? What if the modal can be opened but not dismissed by someone using assistive technology? What if one wrong letter changes the meaning of a message, and the keyboard makes correction difficult?

These are not edge cases. They are part of whether the app works for the people using it.

Automation can help make an app technically more accessible. Human review helps teams understand whether the experience is actually usable.

What automation can check vs. what humans should review

Use this comparison to understand where automated accessibility checks are useful and where human review is still needed. Automation can help teams find common technical issues quickly, such as missing labels, contrast problems, touch target concerns, and some structural issues.

Human review is still needed to validate whether the mobile experience works in context. That includes screen reader flow, cognitive load, task completion, timing, gesture behavior, and whether someone using assistive technology can actually complete the task.

What automation can check vs. what humans should review
Automation checks Human review validates
Automation finds important signals. Human review validates the experience.

Automation struggles with cognitive accessibility

Automated checks are weakest when the question is not “does this element exist?” but “can a person understand and use this experience?”

Cognitive accessibility often depends on clarity, memory load, timing, help text, error recovery, layout consistency, and whether the user can complete a task without becoming confused or overwhelmed. Those are not simple pass/fail conditions.

W3C’s guidance on making content usable for people with cognitive and learning disabilities includes supplemental guidance beyond WCAG requirements and emphasizes involving users in research, design, and testing activities. It also notes that automated testing often focuses on technical areas and cannot fully assess whether people with cognitive or learning disabilities can use the content. (W3C)

For mobile teams, that means automated accessibility checks can help identify important issues, but human review and user testing are still needed to understand whether the app experience works for real people.

Why mobile accessibility testing needs real devices

The State of Mobile App Accessibility report found that nearly three-quarters of tested mobile user journeys included accessibility barriers that resulted in a Poor or Failing experience. That kind of finding reinforces why accessibility testing has to evaluate complete tasks, not just isolated screens.

Mobile accessibility happens where the user meets the app. That makes real-device testing important. Users interact with mobile apps through actual screens, operating system settings, assistive technologies, haptics, audio, gestures, orientation changes, and physical device conditions. W3C’s mobile accessibility guidance explicitly addresses mobile web content, native apps, and hybrid apps, which makes it a better authority for this article than generic web-only accessibility guidance. 

A flow might look fine in a static review but become difficult when VoiceOver or TalkBack announces the controls. A layout might fit on one device and crowd important actions on another. A button might be large enough in one orientation and awkward in another. A pop-up might interrupt the flow and leave a screen reader user without a clear path forward.

Device differences matter too. An app can look readable on one screen and shrink into something difficult to use on another. A color combination may appear one way on one device and differently on another. Screen size, brightness, resolution, operating system settings, and assistive technology behavior can all affect the user experience.

Just like teams cannot assume an app will run perfectly on every device because it passed on one, they cannot assume every person uses technology in the same configuration.

This is one of the reasons real-device accessibility testing matters. With support for iOS VoiceOver and Android TalkBack, Kobiton can help teams test how mobile app experiences behave with the assistive technologies users actually depend on.

Testing on real devices helps teams understand how accessibility behaves in context. It also helps reveal problems caused by device fragmentation, operating system differences, screen sizes, and app interruptions.

Accessibility is not only a design property. It is a lived interaction.

Final takeaway

Automated accessibility checks are useful, but they cannot feel the path.

The goal should not be simply to pass a check. The goal should be to make the app usable when someone needs it to work differently than it works for you.

Mobile accessibility testing needs both systems and people. Use automation to catch repeatable issues. Use real devices to test actual behavior. Use human review to understand whether the app can be navigated, understood, and completed by people with different needs.

People want to feel seen, not sold to. They want to use their apps without being reminded of all the ways the world already makes them adapt. Mobile devices are part of daily life. They hold conversations, maps, games, photos, finances, routines, memories, and small moments of independence.

They are not just screens. They are personal worlds shaped by habit, need, and care.

Everyone deserves to feel welcome in their own.

FAQ

What is mobile accessibility testing?

Mobile accessibility testing verifies that people with disabilities can use a mobile app with assistive technology, adjusted device settings, alternate input methods, captions, clear navigation, and enough time to complete tasks. It combines automated checks, real-device testing, assistive technology review, and human evaluation.

What can automated accessibility checks catch on mobile apps?

Automated accessibility checks can help identify issues such as missing labels, low contrast, small touch targets, some focus-order problems, and certain structural issues. These checks are useful because they can run consistently across builds and help teams find common problems earlier.

What do automated accessibility tools miss?

Automated tools often miss issues involving task completion, cognitive load, unclear instructions, confusing focus order, screen reader meaning, gesture difficulty, and whether a real person can use the app independently. A tool may confirm that an element exists, but a human still has to verify whether the experience makes sense.

Does WCAG 2.2 apply to mobile apps?

Yes. WCAG 2.2 applies to web content and is commonly used as a foundation for mobile accessibility testing, including mobile web, native, and hybrid app experiences. W3C also provides mobile accessibility guidance to help teams apply accessibility principles to mobile-specific behaviors such as touch, orientation, small screens, and device settings.

Why does mobile accessibility testing need real devices?

Mobile accessibility testing needs real devices because assistive technology behavior, screen size, gestures, orientation, audio, haptics, performance, and device settings can change the user experience. Testing only in a simulated or automated environment can miss issues that appear when someone uses VoiceOver, TalkBack, larger text, reduced motion, captions, or other settings on an actual device.

Is accessibility testing only about compliance?

No. Compliance matters, but accessibility testing is also about whether people can actually use the app. A mobile app can meet some technical requirements and still create barriers if the flow is confusing, the screen reader experience is unclear, the timing is too strict, or users cannot complete key tasks independently.

Related reading

Manual Mobile Testing Still Matters

What Automated Accessibility Testing Can and Can’t Catch

Can AI Help With Mobile Accessibility Testing?

AI Mobile Testing Still Needs Real Devices

Tiffany Smith
About the Author Tiffany Smith Technical Content Strategist at Kobiton Tiffany Smith is the Technical Content Strategist at Kobiton, specializing in mobile testing documentation and content architecture. She focuses on turning complex systems into clear, usable guidance that engineers can actually rely on. Her work centers on reducing friction, improving clarity, and helping teams build better testing practices.
Follow LinkedIn