5 Methods to Utilize ChatGPT for Test Automation

No one really anticipated that Generative Pre-trained Transformer or ChatGPT would have such a huge impact on almost every sphere of the technological landscape. However, what has never been in doubt was the anticipated impact of Artificial intelligence in shaping the future of technology. Today, QAs, developers, and other stakeholders in the software and application development industry finally have a chance to take full advantage of ChatGPT.

As a generative AI, ChatGPT has unique capabilities that can be utilized in code testing in a way that will make testers and QAs’ work easier. One of the greatest challenges with automated testing to date remains the time spent before release. The testing lifecycle has often experienced repeat delays due to bottlenecks in critical procedures such as text generation.

Today, testers and QA experts see the entry of ChatGPT as a timely solution. But can ChatGPT live up to these expectations? This article assesses the role of ChatGPT test automation in shaping the future of mobile app testing.

What is ChatGPT?

Anyone who is keen on modern trends in technology has certainly googled ChatGPT. This breakthrough in Large Language Models has taken the world by storm given its current impact and its future prospects. The model is designed to recognize language patterns using machine learning and the vast databases available on Google.

With ChatGPT, it is now possible to get large texts of sequenced information in a grammatically correct and logical way. Interestingly, the same concept of the Large Language Model now also applies to the code application ecosystem.

How to Utilize ChatGPT in Test Automation?

As the uses of ChatGPT evolve, testers and QAs are also finding new uses for this tool.

1.   Generation of Automated Test Cases

A common reason for the delays in test results has always been the tedious process of test case generation. Testing in the past always depended on human beings to undertake language descriptions. Of course, this dependence on testers resulted in errors and time constraints. With ChatGPT test automation, QAs, and testers can now rely fully on AI to generate test cases generated through natural language descriptions.

The best part about using ChatGPT in automation testing is that it can generate as many tests for all possible chatbot testing scenarios. In the past, this was difficult due to various constraints. With this approach, it’s possible for the system to suggest additional requirements that help improve the overall security.

The more structured the input, the more useful the output. Paste in the user story and its acceptance criteria, then ask ChatGPT for three categories of test case:

• Positive: the expected path works as specified.
• Negative: invalid input, missing permissions, expired sessions, and failed network calls.
• Edge: boundary values, empty states, interruptions, and orientation changes.

Asking for a fixed format — ID, preconditions, steps, expected result, priority — makes the output easy to import into your test management tool. You can also turn the question around and ask ChatGPT which scenarios the acceptance criteria don’t cover. It’s often better at spotting gaps in a spec than at writing the final test.

Test data works the same way. Describe the field rules and ask for valid, invalid, and boundary values — phone numbers in several national formats, names with non-Latin characters, or dates on either side of a validity cutoff. Ask for CSV or JSON so the data drops straight into a data-driven test.

ChatGPT can help testers analyze and generate relevant test data and cases once the desired outcomes are well laid out. The approach significantly improves the testing speeds and release.

Even then, a constant concern for ChatGPT users currently is the risk of errors. Generated results from AI may require additional checks from human standards to ensure the elimination of any errors in the content.

2.   Refactoring Codes

The modern developer is highly productivity-oriented. Given the time and effort required, they don’t want to spend significant time trying to refactor codes. With ChatGPT in automation testing, developers can now add comments and refactor codes quickly. Over time, developers have also confirmed that AI produces cleaner, more maintainable codes. This means there can be better collaboration among developers, testers, and QAs due to the extra time available to counter-check and review such codes.

Writing new automation scripts

Refactoring is only half the story. ChatGPT can also draft new scripts from scratch, which is where many testers start. Give it a test case, the framework, the language, and the locators you want it to use, and it will return a working skeleton for Appium, Espresso, or XCUITest.

Example prompt: “Write an Appium test in Python for an Android app using the UiAutomator2 driver. The test opens the app, taps the element with accessibility ID ‘login_button’, enters ‘qa_user’ into ‘username_field’, and asserts that ‘welcome_banner’ is displayed. Use explicit waits, not sleeps.”

Treat what comes back as a first draft. Before you commit it, check the places where AI-generated mobile scripts most often go wrong:

• Locators. ChatGPT can’t see your app, so any locator it invents is a guess. Supply real accessibility IDs or resource IDs, and prefer them over long XPath expressions.
• Library versions. Models often mix old and new APIs — for example, older imports such as MobileBy, which newer versions of the Appium Python client replaced with AppiumBy. Check against the client version your project uses.
• Capabilities. Confirm that platformName, automationName, and the device and app settings match your environment. Appium 2 expects non-standard capabilities to carry a vendor prefix, such as appium:deviceName.
• Waits. Drafts sometimes fall back on fixed sleeps. Replace them with explicit waits so the test doesn’t become flaky.
• Platform differences. A script written for Android won’t run unchanged on iOS. Ask for each platform separately, or ask for a shared page-object structure with platform-specific locators.

Then run it on real devices before you trust it. A script that passes on one emulator can still fail on the devices your users actually carry.

Today, ChatGPT can quickly assist developers and testers in refactoring, renaming, or modifying codes based on the provided instructions. Further, unlike previous bots, which faced problems when dealing with complex requirements, ChatGPT continues to show incredible ease when undertaking most of the writing and refactoring procedures.

3.   Identifying Best Practices

Developing best practices is critical in the process of app testing. These best practices act as a guide for all subsequent procedures. Even then, there is so much involved in the process of mobile testing that it may be tempting to overlook the process of identifying best practices.

The AI helps developers generate different chatbot testing scenarios and develop best practices in the testing process to suit the different scenarios. Normally, testers apply different chatbot testing scenarios and also adhere to varying best practices for different architectural data models.

The fact that ChatGPT is a product of machine learning means that it can easily regenerate these best practices in cases where the scenarios are similar.  This helps save time and improves the overall testing landscape.

4.   Explaining Test Article

The fact that ChatGPT can write text cases and explain the test results is mind boggling. Testers can simply post the error messages received when running tests. The AI is tailored to assess such results and provide responses for each test.

Another major win for testers is the conversation format inherent in the platform. It’s possible for developers, QAs, and testers to ask the AI follow-up questions regarding the errors found in the testing process.

What to include when you paste a failure

Give ChatGPT the same context you’d give a colleague sitting next to you:

• The full stack trace or the relevant section of the Appium server log, not just the last line.
• The device model, OS version, and whether it ran on a real device or an emulator.
• Framework and driver versions, for example Appium 2 with the UiAutomator2 driver.
• The test step that failed and the code around it.
• Whether the failure is consistent or intermittent, and on which devices the test passes.

For example: “This Appium test fails on a Samsung Galaxy S23 running Android 14 with NoSuchElementException at the ‘checkout_button’ step. It passes on a Pixel 8 and on iOS. The screen shows a promotional banner before the button loads. Here are the log and the test code. What are the likely causes, most probable first?” A prompt like this lets ChatGPT reason about timing, overlays, and device-specific layouts instead of guessing.

Treat the answer as a hypothesis to check on the device, not as a diagnosis. Screenshots, video, and device logs from the failing session are still the evidence.

Over time developers have also noted that ChatGPT can provide feedback about potential bugs in the various coding procedures. With such a tool, it’s possible to achieve better quality and timely tests at all times.

5.   Providing Test Summaries for Testers

It can be such a daunting task trying to compile test summaries for thousands of tests using manual processes. The risk of errors and inaccuracies can affect the overall quality of the tests. However, with ChatGPT, testers can now obtain test summaries with incredible ease.

ChatGPT summarizes the content in a narrative format that makes it easy for any audience to follow and utilize. Whether the tests are technical or non-technical the AI will still provide useful summaries.

At the end of every testing process, the testers and QAs have an obligation to provide a comprehensive report that details the scenarios and results. But with the entry, you don’t need to worry about generating large amounts of reports, which could be unstructured information. With ChatGPT, you can now generate report summaries with ease and speed.

ChatGPT prompts for mobile test automation

What you get out of ChatGPT depends almost entirely on what you put in. Ask it to “write a login test” and you’ll get a generic answer. Give it the platform, the framework, the behaviour you expect, and the format you want back, and you’ll get something you can actually work with. A strong testing prompt usually includes five things:

• Context: what the app does and which feature or screen you’re testing.
• Platform and framework: Android or iOS, and whether you’re using Appium (with the UiAutomator2 or XCUITest driver), Espresso, or XCUITest, plus the programming language.
• The input: the user story, acceptance criteria, real accessibility IDs, or the error log — pasted in, not described.
• Constraints: your conventions, such as the Page Object Model, explicit waits only, or no hard-coded test data.
• Output format: a table with named columns, Gherkin scenarios, CSV, or code only.

Here are prompts QA teams use for each of the five methods above. Swap in your own feature, framework, and data.

Prompt	What it helps with	Method
“Here are the acceptance criteria for our Android checkout flow. List positive, negative, and edge-case test cases as a table with ID, preconditions, steps, expected result, and priority.”	Test case design	1. Generation of Automated Test Cases
“Generate 20 test addresses for a shipping form, including invalid postcodes, non-Latin characters, and fields at maximum length. Return CSV.”	Test data	1. Generation of Automated Test Cases
“Convert these three test cases into an Appium test in Java using the UiAutomator2 driver and the Page Object Model. Use these accessibility IDs as locators: [list].”	Script generation	2. Refactoring Codes
“Refactor this Appium test to replace Thread.sleep calls with explicit waits and move all locators into a page object.”	Refactoring	2. Refactoring Codes
“Suggest a regression checklist for a release that changes the login screen and push notification settings.”	Coverage and best practices	3. Identifying Best Practices
“This Appium test fails only on Android 14 with the error below. It passes on iOS. Suggest likely causes, most probable first. [log]”	Failure analysis	4. Explaining Test Article
“Summarise these test results for a product manager: what failed, which user flows are affected, and what is blocking release. [results]”	Test summaries	5. Providing Test Summaries for Testers

Benefits of ChatGPT

Now that we have seen some of the ways through which ChatGPT can help in testing, the next part should be the benefits that will accrue from the use of ChatGPT in the process of testing.

·  Saves Time

The need to introduce AI in the entire process of mobile app testing is to meet specific inherent gaps in human beings. One such gap is the ability to work on bulk processes while saving time. ChatGPT has saved significant time over the last six months since its launch.

Today, testers can conduct multiple tests and get a full summary for each report I record time. Testers can use the extra time to plan and execute more test cases. Depending on the project’s complexity, it may take a few hours to conduct such tests.

·  Saves on costs

Hiring testers, QAs, and other personnel during testing can be strenuous cost-wise.  In some instances, tests have returned false results due to limited work capacity. Rather than waste money trying to hire more staff to do the same repetitive task, ChatGPT works ten times better than the average testers. This means that companies seeking to use AI will notice significant cuts in current budgets.

·  Process Bulk Orders

ChatGPT makes it easy for developers, QAs, and testers to quickly undertake bulk testing for various scenarios. The fact that the AI can generate codes, write and reprint codes means that it can be a valuable source for bulk test cases. The best thing about AI is that it can achieve the same results for bulk orders as for one order.

·   Shorter Release Time

The test process is unpredictable when it is entirely human-led. The turnaround time around time for delivery has remained a matter of concern over the year due to the long wait time noted. However, ChatGPT is a game changer. Now, you can enjoy a shorter release time with greater flexibility.

Cons of ChatGPT

It’s important to note that everything has the other side of the coin.

·  Prone to Error

While less significant than human error, ChatGPT may have notable errors, mainly due to originality. In some cases, it’s clear that the AI cannot present a clear and concise line of thought. It may take a keen review of the entire text for you to remove any underlying ambiguities.

·  Future Prospects

Current text	Suggested replacement
“The model is designed to recognize language patterns using machine learning and the vast databases available on Google.”	“Developed by OpenAI, the model is trained on large volumes of text and code, which lets it recognise language patterns and respond to natural language prompts.”
“ChatGPT has saved significant time over the last six months since its launch.”	“Since its launch in late 2022, ChatGPT has become a routine part of many testers’ workflows, cutting the time spent on repetitive preparation work.”
“Rather than waste money trying to hire more staff to do the same repetitive task, ChatGPT works ten times better than the average testers.”	“Rather than adding headcount for repetitive preparation work, teams can use ChatGPT for first drafts of test cases, data, and documentation, freeing testers for the judgement-heavy work that still needs a person.”
“The ChatGPT is still new, and the available versions are on trial. This raises questions about the future of AI, the possible cost of the other version, and how well testers embrace it.”	“ChatGPT and the models behind it change frequently, so prompts and workflows that work today may behave differently after an update. Re-check generated output after model changes, and keep an eye on pricing and data-handling terms for the tier you use.”

The ChatGPT is still new, and the available versions are on trial. This raises questions about the future of AI, the possible cost of the other version, and how well testers embrace it.

• No view of your app. ChatGPT has never seen your screens, element hierarchy, or device behaviour. Everything it writes about your app is inferred from what you paste in.
• It doesn’t execute tests. It can draft a script but can’t run it, so it can’t tell you whether that script passes — on one device or on fifty.
• Invented methods and outdated APIs. It can confidently use functions, capabilities, or library calls that don’t exist or have been deprecated. Code that looks right can still fail to compile.
• Blind to mobile runtime conditions. Gestures, interruptions such as calls and notifications, network changes, permission dialogs, and differences between OS versions and manufacturers are exactly where mobile apps break — and exactly what a language model can’t observe.
• Data exposure. Pasting proprietary source code, production logs, or customer data into a chat tool may breach your organisation’s policies. Check which tools and account tiers your company has approved, and mask sensitive data before you share it.
• Inconsistent output. The same prompt can produce different answers from one day to the next, so generated tests need to be reviewed and version-controlled like any other code.

ChatGPT can	ChatGPT cannot
Draft an Appium script	Confirm the locators exist in your app
Explain a stack trace	Reproduce the failure on a device
Suggest edge cases	Know which ones your users actually hit
Summarise test results	Verify that the results are correct
Write pipeline configuration	Run the pipeline

A practical workflow: from prompt to real device

ChatGPT is most useful when it sits inside your testing workflow rather than beside it. Here’s how the five methods above fit together in a typical sprint.

1. Start from the requirement. Paste the user story and acceptance criteria and ask for test cases, including negative and edge cases. Review the list and cut what doesn’t apply.

2. Generate test data. Ask for data sets that match those cases, including invalid and boundary values. Mask or replace anything that resembles real customer data.

3. Draft the scripts. Request Appium, Espresso, or XCUITest code that uses your real locators and your project’s conventions.

4. Review it like any pull request. A person checks the locators, waits, assertions, and library versions before the code goes into the repository.

5. Run on real devices. Execute across the device and OS combinations that matter to your users. This is the step ChatGPT can’t do for you, and it’s where most AI-generated scripts reveal their problems.

6. Add it to the pipeline. ChatGPT can draft the CI configuration — a GitHub Actions workflow or a Jenkinsfile stage — that triggers the suite on every build. Review it before merging.

7. Feed failures back. Paste failure logs, along with the device, OS version, and framework version, into ChatGPT for likely causes. Fix, then rerun on the device.

8. Summarise for stakeholders. Ask for a summary of the results written for the audience that needs it — developers, QA leads, or product managers.

Notice where people stay in the loop: reviewing what ChatGPT generates, and deciding what gets committed and released. ChatGPT speeds up the preparation. The results on real devices decide whether you ship.

ChatGPT vs. AI-powered mobile testing platforms

ChatGPT is a general-purpose assistant. Mobile testing platforms with built-in AI are designed around your app, your devices, and your test runs. The two overlap in places, but they solve different problems.

Capability	ChatGPT	AI-powered mobile testing platform
Draft test cases and scripts	Yes, from what you paste in	Yes, often from sessions recorded against the real app
See your app’s real UI	No	Yes, during sessions on devices
Run tests on real devices	No	Yes
Keep locators working as the UI changes	No	Self-healing, on platforms that offer it
Capture device logs, video, and screenshots	No	Yes
Explain concepts and answer general questions	Strong	Varies by platform
Data governance	Depends on your account tier and company policy	Managed within the platform’s security controls

Most teams end up using both: ChatGPT for thinking, drafting, and explaining, and a testing platform for running, maintaining, and proving the tests. Kobiton’s AI capabilities, such as Appium script generation and self-healing, sit on the second side of that line.

AI is the Future

As a multiplicative AI, ChatGPT has unique abilities that can be utilized in code testing to make testers and QAs’ work easier. One of the extreme trials with automated testing remains to turn around time spent before release. The testing lifecycle has often experienced repeat delays due to tailbacks in critical procedures such as text generation.

With ChatGPT automation testing, such concerns are behind us as we look into the future with great optimism that AI will drive the industry. Kobiton is among the few platforms that now allow for integrating this technology into most routine processes. 

FAQ

Can ChatGPT write Appium tests?

Yes, as a first draft. Stress supplying real locators and checking library versions and capabilities; link to Gap 2 content.

Can ChatGPT run automated tests?

No. It drafts and explains; execution needs a framework and devices. One sentence on why real devices matter for mobile.

Will ChatGPT replace QA testers?

No — it removes preparation work but can’t see the app, judge risk, or verify results. Keep it measured, not dismissive.

Is it safe to paste code or logs into ChatGPT?

Depends on company policy and account tier; mask customer data and secrets; check with security before sharing proprietary code.

What are the best ChatGPT prompts for test automation?

Point to the prompt table; summarise the five ingredients of a good prompt (context, platform/framework, input, constraints, format).

How is ChatGPT different from AI testing tools?

General-purpose assistant vs. tools built around your app and devices; reference the comparison table in Gap 7.