Empowering Testers with Visual AI

Martin Kowaleski, Director of Sales Engineering at Applitools led a powerful session at MTES 2024 illustrating the power of Visual AI automation testing. He gave a brief overview of visual testing and its importance, then provided an in-depth demonstration of how Kobiton and Applitools integrate to empower testers by automating visual testing. 

Kowaleski had been in the industry for over 20 years. He has a background in dynamic storytelling, problem solving, and is a proven pre-sales leader in technical solutions achieving real business outcomes.

Applitools sets the industry standard for AI-powered test automation, embedding AI throughout the testing lifecycle. The Applitools Intelligent Testing Platform is remarkably intuitive, enabling every team member—from design and development to QA, operations, marketing, and product teams—to contribute to quality assurance, regardless of coding expertise. With Applitools, companies can accelerate their testing processes, perform comprehensive quality checks, and ensure superior product delivery through proactive, continuous testing.

What is Visual Testing?

Visual testing, as defined by Martin, is “the process of validating all the visual aspects of an application’s UI on all platforms.” This testing extends beyond Appium, Espresso, or XCUI. It ensures that the content, images, and buttons appear correctly. 

Since functional testing can balloon test scripts if you use it to validate the size and position of visual elements, visual testing can be useful for developers. There are two forms of visual testing: manual and automated. Each has advantages and disadvantages; you must decide what works best for your system. 

Manual Testing

Manual testing consumes a lot of time and effort. This form of testing is done by comparing two screenshots: a baseline and the latest version of the app. Each screen must be tested individually on each device and with each unique screen orientation. Kowaleski did the math and to conduct a manual test for one screen on your app, there would be 40 different screen orientations to test using Android and IOS operating systems. That is a lot of testing!

Automated Testing

Automated visual testing can be a good option to get your whole team involved in the testing process. This form of testing compares the visual elements of each screen through Appium, XCUI, Espresso, or Webdriver. By automating your visual testing, your team can reap these benefits:

• Single Capture Assertions
• Catching more bugs
• Easier maintenance 
• Involves more teammates in Quality Assurance 

Different Types of Visual Testing

There are two types of visual testing: Pixel comparison and Visual AI. Pixel comparison involves taking bitmap screenshots of the new screen and comparing them to a baseline bitmap to detect subtle differences. However, it has limitations. It cannot accurately test overlaps, affecting dynamic content testing. Additionally, it struggles with mobile apps due to dynamic elements like date, time, battery status, and display settings, which can result in false positives. 

Visual AI testing uses algorithms to test for differences. Visual AI can be used in both static and dynamic environments. There are many use cases Visual AI testing covers that pixel does not. These include:

• Cross Browser Testing
• Account Balances
• Mobile Device Status Bars
• News Content
• Ad Content
• User-submitted content
• Notification Icons
• Content Shifts
• Mouse Hovers
• Cursors
• Anti-aliasing Settings
• Browser Upgrades

Benefits of using Visual AI in native mobile automation

Visual AI in native mobile automation offers more testing coverage for less work. It is able to test a range of visual elements on OS, browser, screen orientation, and resolution combinations. 

Kowaleski gave an example of pixel comparison vs Visual AI testing on an e-commerce app. The pixel comparison would have failed the test because users sort products differently, and each time a user searches for a product, the products show up differently. In this case, Visual AI testing would be a better option as it tests using an algorithm. 

Demonstration: Manual to Automation with Visual AI

Kowaleski demonstrated his workflow through a manual testing session on the Kobiton platform. Watch the full demo here, but in the meantime, here is a quick summary:

1. Manually navigated through your application
2. Convert it to an automated script via Appium automation
3. Created a Visual AI Automated testing script
4. Generate JAVA script
5. Run the test on a go-forward basis (This converts the test from manual to automated session) 
6. Open code to add visual validation from Applitools to existing Appium script
7. Create Baseline by running an inside automation first test on Kobiton platform
8. Complete Project 

Kobtion’s integration with Applitools allows for greater collaboration within the testing environment. Martin Kowaleski verified that Applitools and Kobiton’s integration is a

“Powerful capability to run a mixed session for Kobiton.”

Unlock the Full Potential of Visual AI

Martin Kowaleski masterfully illustrated the game-changing potential of Visual AI in testing. Don’t miss the chance to dive deeper into this innovative integration. Watch the full session here and discover how the combined power of Kobiton and Applitools can elevate your testing strategy to new heights!