Automation and Artificial Intelligence in Mobile Testing

In an era where mobile applications are not just a luxury but a necessity, the 2024 State of Mobile Webinar by Kobiton offered invaluable insights into the current and future landscape of mobile app testing. As technology evolves at a breakneck pace, so do the expectations for mobile app functionality, usability, and reliability. 

The webinar was hosted as a companion to our 2024 State of Mobile Report and took a more indepth look into how advancements in Artificial Intelligence (AI) and automation are shaping the future of mobile application testing, highlighting the opportunities and challenges that lie ahead.

Developments in the world of automation and artificial intelligence (AI) technology are allowing us the opportunity to deliver on accelerated development timelines, and it’s also enabling more robust testing strategies.

Last October, Kobiton surveyed a hundred developers and software testing professionals who were directly involved in the development and testing of mobile apps. We gathered this data, analyzed it, and created our report and webinar to help you understand the direction mobile developers and testers are moving towards concerning AI and automation. 

Our Survey Methodology

We began with reviewing our survey methodology for the  2024 State of Mobile Report. We had a broad coverage of industries and only spoke to developers and software testing professionals directly involved in testing at organizations that had at least 100 employees. From October 9 – 24, 2023, these mobile development and testing professionals answered 30 multiple choice and multi-select questions that informed our research report and webinar.

Then on January 25, 2024 we held our in-depth review of this data via webinar. This engaging and informative session was led by our Head of Product Marketing Cara Suarez, as well as our CRO, Matt Klassen and featured family feud style polling that worked to not only keep the audience involved but also acted as a secondary data source that was then compared live on the webinar to the data we gathered in October.

The Critical Role of AI and Automation in Mobile App Testing

The evolution of mobile app testing is significantly marked by the integration of AI and automation technologies. These advancements are transforming the testing landscape by offering solutions to traditionally time-consuming and resource-intensive processes. The webinar discussion began with reviewing how AI-driven tools and automated testing frameworks are enabling organizations to achieve faster testing cycles, higher accuracy, and broader test coverage across numerous device types and operating systems. “There is really more of a trend to continuous app delivery and automation” remarked Cara Suarez, Head of Product Marketing at Kobiton.

Today, AI allows organizations of every size to automate and test code across more devices, faster than ever before and survey results confirmed, that although some development and QA teams express reservations about certain aspects of AI in the software testing process, many organizations are already using AI to accelerate processes like updating scripts and analyzing test results. These same individuals see even more opportunities for AI, and plan to expand their own use of AI.

The Impact of Mobile Test Automation and AI on Business Operations

Our webinar also emphasized the symbiotic relationship between DevOps practices and the need for rapid software delivery cycles in the mobile app development realm. Illustrating how AI and automation are critical in enabling organizations to meet the demands for quicker release schedules without compromising on quality. By integrating AI-driven testing tools into the DevOps pipeline, companies can achieve continuous testing and integration, leading to faster time-to-market and improved application performance. 

When remarking on the benefits of incorporating AI and automation into the mobile development and QA process, Cara said, “Stability is something that can be challenging dealing with the brittleness and flakiness of tasks.” Highlighting some of the benefits and impact of using more advanced technologies in the development and testing of mobile applications.

Despite the clear benefits, the adoption of AI and automation in mobile app testing is not without its hurdles. The webinar reviewed key challenges such as budget constraints and the technical complexities associated with implementing advanced testing methodologies. However, it also outlined how these obstacles can be overcome through strategic investments in AI technologies that offer scalable, efficient testing solutions without the need for extensive manpower.

A third of the organization representatives we heard from told us that they lose revenue due to slow mobile software release cycles, and 40% reported diminishing rates of customer satisfaction.

Given that more than three-quarters of businesses surveyed depend on mobile apps for at least 25% of their total revenue, slow mobile development has clear consequences for their bottom line. According to our survey, it’s estimated that a third of companies today lose more than $500,000 annually as a result of slow release times. AI and automation can end or at least greatly reduce those loses. By allowing QA engineers to run tests faster, automation delivers dramatic turnaround time. While in the past manual tests would take a usual 3-5 days to execute, automated tests are completed in hours.

Role of Automation in Accelerated Mobile Software Delivery

More than half of the survey respondents reported that test automation has decreased time to market by at least 25% and 58% of organizations told us they release mobile app updates or bug fixes at least once per week. Automation is making a real difference in the speed at which these releases and fixes are available. 

“Ok this one may be controversial, do you believe AI- driven test automation can eventually replace manual testing on mobile applications?”

Cara Suarez, Head of Product Marketing at Kobiton

Looking forward, Kobiton believes in an optimistic picture of the role AI is expected to play in mobile app testing. It predicts a significant shift towards the increased use of AI not just for automating repetitive tasks but for making intelligent decisions based on real-time data analysis. This includes predictive analytics for identifying potential issues before they occur and enhancing the overall quality and user experience of mobile applications. “I would look at AI as a way to help move to higher levels of productivity,” said Matt Klassen. “It’s the difference between a writer who sees ChatGPT as a threat vs a writer who sees it as a tool to help them achieve more.” 

The Career Impacts of  AI and Automation 

Although some of our survey respondents expressed concern about how AI might negatively impact careers and software quality, a significant number of teams also expressed a degree of excitement about the potential of AI to increase software development productivity. There’s also a consensus about the potential for these technologies to free up obligations toward routine and mundane tasks.

While the webinar and report were largely positive about the advancements in AI and automation, it does not shy away from addressing the concerns and ethical considerations that accompany these technologies. We discussed the fear of job displacement, issues of bias and fairness in AI algorithms, and the importance of maintaining a human oversight to ensure that the benefits of automation are realized without unintended consequences.

When weighing in on this, CRO Matt Klassen said, “The one interesting thing about this is the concern AI tools may impact the quality of software – it’s probably related to trust. I think it’s healthy to have some level of insight”

AI and Automation Will Continue to Evolve

Automation and AI are going to evolve right alongside us, giving us new solutions, opportunities – and yes, some concerns, too – about how we learn to work with the new technology.

As the mobile app market continues to expand, the need for efficient, effective testing strategies becomes increasingly critical. By embracing these technological advancements, organizations can navigate the challenges of modern app development, ensuring their products meet the high standards of today’s digital consumers.

Are you confidently ahead of the curve, or do you worry about being left behind? Kobiton can help put you in the driver’s seat. Learn more, see The 2024 State of Mobile App Delivery, Test Automation, and AI webinar on demand here.