How to automate end-to-end testing with AI

By the end of this guide you will have a small suite of automated end-to-end tests that exercise your most important user journeys, run on every pull request, and are kept up to date with help from AI. The tools do one or more of three jobs: generate tests, run them, and repair them when the interface changes.

AI test automation is not one product. Some tools write tests from plain English, some record real user sessions, and some wrap an open-source framework with agents. This guide shows how to choose, how to roll the tests out in order, and how seven options from our Developer platforms category compare.

Key takeaways

• Start with three to five critical journeys such as sign-up and checkout, not the whole app. Get them running in CI before you widen coverage.
• The main decision is how tests are authored: code you own (Playwright), plain English (testRigor, agent-qa), recorded flows (KushoAI), or a managed service (QA Wolf, mabl).
• Many commercial tools publish no prices. mabl, for example, is a custom quote, so compare on a trial before you commit.
• AI-generated and self-healed tests still need human review, because a healed test can pass while checking the wrong thing.

What you need before you start

• A staging environment. Tests should run against a stable copy of your app with test accounts and test data, never against real customer data.
• A list of critical journeys. Write them as short sentences, for example "a new user signs up, adds an item and pays".
• A CI system. The tools below connect to pipelines such as GitHub Actions or Jenkins, depending on the tool.
• Someone to own the suite. A person must review failures and approve repairs.
• Time. A first test running in CI takes under a day. A dependable suite takes weeks of tuning.
• Difficulty. Easy to moderate for plain-English tools, moderate for code-first frameworks.

Which kind of AI tool fits this job

A code-first framework you own. Playwright is an open-source Microsoft framework that drives Chromium, Firefox and WebKit from one API, with auto-waiting, parallel runs, sharding for CI, an HTML report and a Trace Viewer. It has a test generator that records clicks into code, plus a CLI and an MCP server for AI coding agents. The tests are your code, so there is no vendor lock-in on the tests themselves.

Plain-English test authoring. testRigor lets you write end-to-end tests in plain English, covering web, mobile, desktop and API, and has a free public plan. agent-qa is a source-available agent that runs plain-English YAML tests on web, Android and iOS apps through Playwright or Appium. It re-observes the screen when a step fails and tries another route.

Generation from specs and recordings. KushoAI generates tests for APIs and web UIs from an OpenAPI spec, a Postman collection, a cURL command or a recorded UI flow, and has a free Developer Edition.

Managed or agentic platforms. QA Wolf pairs AI agents with QA engineers, writes tests in open-source Playwright and Appium, and bills per run. mabl builds, runs and self-heals tests for web, mobile, API, accessibility and performance, with a 14-day free trial. These suit teams that want less maintenance work in house.

Visual checks. Applitools uses Visual AI to flag meaningful visual changes rather than pixel differences, and lists more than 30 SDKs for adding visual checks to coded tests.

Step 1: Choose critical journeys and success criteria

You end this step with a short list of journeys and a clear pass condition for each.

1. List the three to five journeys whose failure would hurt most, such as sign-up, login, checkout or an account change.
2. For each, write the starting state, the actions and the visible result that proves success.
3. Note any external dependencies, such as email, SMS or two-factor login. testRigor lists email, SMS and two-factor login support, which matters if your journeys include them.
4. Decide which devices and browsers you must cover.

Check it worked by having a teammate perform each journey by hand using only your notes.

Step 2: Pick an AI test automation approach

Choose by who will maintain the tests and what you already use.

1. If your developers write the tests, start with Playwright and use AI to draft and repair them.
2. If non-developers need to write tests, shortlist plain-English tools such as testRigor or agent-qa.
3. If you need API tests from existing specs, look at KushoAI, which accepts OpenAPI, Swagger, Postman and cURL inputs.
4. If you want someone else to build and maintain the suite, compare QA Wolf's per-run pricing with mabl's custom quote.
5. Check that your platforms are covered: web, iOS, Android, desktop or API.

Check it worked by writing down which tool you chose, who will own it and why.

Step 3: Create the first test

Start with one journey and make it pass reliably before adding more.

1. Author the test in the way your tool supports: a recorded flow, a plain-English description, code, or a generated draft.
2. Use stable test data and a dedicated test account.
3. Add assertions about what the user should see, not only that a page loaded.
4. Read the generated steps. Generation tools can include irrelevant steps or miss a key check.

Check it worked by running the test five times in a row and getting the same result each time.

Step 4: Review what the AI produced

AI can draft tests quickly, but you decide whether they check the right things.

1. Compare each generated step with your Step 1 notes.
2. Add negative cases, such as an invalid card or an expired session.
3. Treat generated tests as drafts. A person who knows the product should review every test before it joins the suite.
4. Remove duplicate or low-value tests, since every extra test adds run time and maintenance.

Check it worked by deliberately breaking the app, for instance by disabling a button, and confirming the test fails.

Step 5: Run the tests in CI

Tests only help if they run automatically.

1. Add the test run to your pipeline so it runs on each pull request.
2. Run in parallel where supported. Playwright lists parallel runs and sharding, and QA Wolf says it runs tests in parallel on its cloud.
3. Check CI support for your tool. KushoAI documents CI/CD execution as an Enterprise feature, so confirm your plan includes what you need.
4. Save reports, traces or recordings from failed runs. Playwright's Trace Viewer records DOM snapshots, network requests and console logs.

Check it worked by opening a pull request that changes a button label and seeing the pipeline report on it.

Step 6: Let AI help with maintenance, with limits

Self-healing reduces upkeep, but it can hide real bugs.

1. Turn on auto-healing or repair features where offered. mabl lists auto-healing and runtime recovery, and agent-qa re-observes the screen after a failure.
2. Require review of every healed or repaired step before it becomes permanent.
3. Treat repeated healing on the same step as a signal. Either the interface is unstable or the test is too loose.
4. Keep a record of what changed and why, so you can trace odd passes.

Check it worked by changing a label in staging and confirming the tool either adapts correctly or fails with a clear message.

Step 7: Add visual checks and widen coverage

Once core journeys are stable, add the next layer.

1. Add visual checks for pages where layout matters. Applitools lists cross-browser and device checks and accessibility checks.
2. Add edge cases and less common journeys, one at a time.
3. Review flaky tests weekly and fix or remove them. QA Wolf lists up to 3 free attempts per run to clear flakes, but reruns do not fix underlying instability.
4. Track simple measures: run time, flake rate and bugs caught before release.

Check it worked by comparing the bugs your tests caught this month with the bugs reported by users.

Tools for this job compared

Tool	Good fit for	Pricing	Stated limits
Playwright	Code-first web tests across Chromium, Firefox and WebKit	Open source	No vendor-hosted paid plan listed; browser binaries must be updated with the framework
KushoAI	API and UI test generation from specs or recordings	Freemium	CI/CD execution and adding more tests are Enterprise-only; Developer Edition is for teams of 2 to 5
mabl	Agentic web, mobile and API testing with auto-healing	Free trial	Custom quote; cloud runs use credits starting at 500 per month
QA Wolf	Managed Playwright and Appium suites on web, mobile and desktop	Paid, from $1, usage-based (checked 24 Oct 2026)	Up to 5 minutes per test; longer tests billed in 5-minute blocks; cost scales with runs
testRigor	Plain-English tests across web, mobile, desktop and API	Freemium	Free Public plan makes tests public; parallelization limited to one on free plans
agent-qa	Plain-English YAML tests in your own repo, with CLI and MCP	Free	Bring your own model credentials and infrastructure; source-available licence
Applitools	Visual AI checks for web, mobile and API	Free trial, paid from $667/mo (checked 24 Oct 2026)	Starter: 100,000 component or 1,000 page checkpoints; Professional and Enterprise prices not listed

Prices and plan limits change, so confirm the current terms on each vendor's site before you subscribe.

Common mistakes to avoid

• Automating everything at once. Large suites are slow and flaky, and nobody trusts them. Fix it by starting with a few critical journeys and expanding only when they are stable.
• Trusting self-healing blindly. A healed step can click the wrong element and still pass. Fix it by reviewing healed changes and keeping assertions on visible outcomes.
• Using production data. Tests that create orders or send emails can affect real customers. Fix it by using staging with test accounts, and check how tools such as testRigor handle email and SMS in tests.
• Comparing tools only by price. Many vendors publish no prices, and per-run, per-credit and per-parallelization models are hard to compare. Fix it by estimating your monthly runs and asking for quotes on the same assumptions.
• Forgetting licensing and data terms. A free public plan, such as testRigor's, makes tests visible to anyone. Fix it by reading the terms before you put real test data or internal URLs in.

How we put this together

This is an evidence-based guide built from each tool's listing in our directory, which is compiled from the vendor's own site and pricing page. It is not a review based on using the tools. We don't rank tools, and nobody paid to be included. Some outbound links may be affiliate links; see our disclosure.

Frequently asked questions

Is there a free way to automate end-to-end tests with AI?

Yes. Playwright is open source, agent-qa's vendor says it has no paid tier (you supply your own model and infrastructure), KushoAI has a free Developer Edition, and testRigor has a free public plan where tests are visible to anyone. mabl and Applitools offer free trials. Free options trade away some features, hosting or privacy.

Can AI write my end-to-end tests for me?

It can draft them, from plain English, specs, recordings or captured sessions, depending on the tool. You still need to review what it wrote. Generated tests can miss important assertions, so compare them with your list of journeys and break the app on purpose to see them fail.

Is Playwright an AI testing tool?

Playwright itself is an open-source testing framework, not an AI agent. It adds a CLI and an MCP server for AI coding agents, and tools such as QA Wolf and agent-qa build on it. That makes it a common base for AI-assisted testing.

How long does it take to set up?

It depends on your app and how many journeys you cover. Getting one test running in CI is the quick part; a suite you trust takes longer, because you need stable test data, reviewed assertions and a plan for flaky tests. Managed services such as QA Wolf aim to shorten that by building the suite for you.

Can I keep my test data and code private?

Code-first and self-run options give you the most control: Playwright and agent-qa run in your own environment, though agent-qa uses a model provider you configure, so check what that provider receives. With hosted platforms, review each vendor's data terms. Avoid free public plans for anything sensitive.

Compare more options in our Developer platforms category.