Automating QA with AI-assisted testing

By Marco Lopes · May 5, 2025 · Web Development

Shipping a web application means checking much more than whether a page loads. We need to know whether permissions hold, forms recover from errors, interfaces remain accessible, and integrations behave when something goes wrong. As applications grow, manually repeating those checks becomes expensive, while maintaining automated tests becomes its own engineering workload.

At Kordspace, we approach AI-assisted testing as a way to reduce that workload, not as permission to remove human judgment. As a remote studio working across software, AI, Web3, hardware, and design, we see quality as a shared responsibility rather than a final handoff.

The useful question is not whether AI can write a test. It can. The useful question is whether we can make its contribution trustworthy, reviewable, and cheaper to maintain than the work it replaces.

Here is how we would build that system with the tools available today.

Start with risk, not generated test volume

Before introducing a model, we map the journeys that matter. For a typical web product, these might include signing in, updating account details, completing a purchase, or accessing content under different roles.

For each journey, we identify the consequences of failure. A misaligned icon and an authorization bypass should not receive the same testing priority.

We then organize coverage across three layers:

• Unit tests for isolated rules, transformations, and boundary conditions.
• Integration tests for interactions among services, databases, and external dependencies.
• End-to-end tests for a small set of critical user journeys.

AI can suggest gaps within this structure. Without it, a model may generate dozens of overlapping browser tests while overlooking a crucial permission rule. More test files do not automatically mean more confidence.

Give AI a bounded role in the workflow

We separate probabilistic assistance from deterministic execution. The model proposes scenarios, drafts code, or summarizes evidence. Established tools execute tests and produce the signals that control merging and deployment.

For a TypeScript application, that might mean Vitest or Jest for unit tests, an integration suite against disposable infrastructure, and Playwright for browser workflows. We choose tools that fit the application rather than replacing a working stack just to introduce AI.

A practical workflow looks like this:

1. Gather the relevant requirements, diff, and existing test conventions.
2. Ask the model for a coverage proposal or a narrowly scoped patch.
3. Review the proposal against expected product behavior.
4. Run the resulting tests in an isolated environment.
5. Preserve failures, traces, and review decisions as ordinary development artifacts.

We do not let a model change assertions, accept new visual baselines, or approve its own repairs automatically. Those actions can hide the very regressions we want to catch.

Generate scenarios from explicit acceptance criteria

AI produces better tests when we provide behavior instead of a vague instruction such as “test this page.” We include supported roles, validation rules, state transitions, known edge cases, and examples of existing tests.

For a password-reset flow, we might use this prompt:

Propose test scenarios for this acceptance criterion: a registered user can request a single-use reset link, and an expired link cannot change a password. Cover successful use, repeated use, expiration, invalid input, and information leakage. Do not invent the expiration duration or response wording. List missing requirements separately.

That final instruction matters. Models often fill gaps with plausible assumptions. If those assumptions become assertions, we risk enforcing behavior nobody approved.

We review the scenario list before requesting code. This gives product, design, and engineering colleagues a readable artifact to challenge, without requiring everyone to inspect a browser automation script.

Make generated tests behave like good engineering

Generated code should follow the same standards as handwritten code: clear setup, focused assertions, predictable cleanup, and minimal dependence on unrelated state.

For browser tests, we prefer role-based locators and accessible names when they express the intended interaction. Stable test identifiers are useful where semantic locators cannot uniquely identify an element. Long CSS chains and positional selectors usually create avoidable maintenance.

We also replace arbitrary sleeps with conditions the test runner can observe. Waiting for a confirmation message is more meaningful than assuming a network request will finish within a fixed delay.

Test data deserves equal attention. We create isolated users or tenants, seed only the necessary records, and clean up reliably. We avoid making one test depend on another test having run first. AI cannot compensate for a suite whose outcomes depend on yesterday’s database contents.

Keep assertions independent of the implementation

One of the biggest risks in AI-assisted QA is circular reasoning. If we ask a model to read a function and write matching tests, it may document the current implementation, including its bugs.

We need an independent source of expected behavior: approved requirements, an API contract, a domain rule, or a reviewed example.

Suppose a checkout calculates discounts. We should supply the intended rounding rule and eligible items, rather than asking the model to infer both from the calculation code. Otherwise, a mistaken rounding choice can appear in the implementation and the test simultaneously.

For suitable logic, we can also use property-based testing. Instead of checking only a few examples, we verify invariants across generated inputs. Mutation testing offers another useful check: if deliberately changing the code does not break a test, its assertions may be too weak.

AI can propose these checks, but we still verify that each invariant is genuinely required.

Use AI to investigate failures, not erase them

Failure triage is a strong starting point because it can save effort without changing the release gate. We can give a model a failed assertion, sanitized logs, a browser trace summary, and the relevant code diff, then ask for ranked hypotheses.

We want a response that distinguishes evidence from speculation. “The trace shows a request returning 403” is an observation. “The test account may lack the required role” is a hypothesis that needs verification.

The output should include the next useful action: inspect the seeded role, reproduce with the same account, or compare the request against the authorization contract.

We treat automatic test repair cautiously. A proposed locator update may be legitimate, but changing a success assertion to match an error message is not a repair. Any patch should explain what changed and why the original product expectation still holds.

Combine visual, accessibility, and functional checks

Web quality includes appearance and usability, but these need different evidence. Screenshot comparisons can reveal layout changes. Accessibility tools can identify certain structural issues. Functional assertions verify behavior. None replaces the others.

We establish reviewed visual baselines and stabilize the capture environment: browser version, viewport, fonts, animations, and test content. Otherwise, harmless rendering differences can bury meaningful regressions in noise.

AI may help summarize screenshot differences or suggest areas for inspection. We do not treat its visual opinion as proof that an interface is correct or accessible.

We pair automated accessibility checks with manual keyboard and assistive-technology testing where appropriate. For example, an automated scan may not establish whether a multi-step dialog communicates its purpose clearly or returns focus to the right place. These remain important review tasks, especially after significant design changes.

Protect data and constrain agent permissions

Test artifacts can contain sensitive information: session cookies, personal details, internal URLs, or credentials accidentally printed to logs. Sending a complete trace to a model without inspection can turn a QA improvement into a data-handling problem.

We prefer synthetic fixtures and redact artifacts before model submission. We also check the selected provider’s retention, training-use, regional-processing, and access-control settings against project requirements. These details vary by service and configuration; we do not assume that every endpoint behaves identically.

If we introduce an agent that can run commands, we confine it to an isolated workspace with short-lived credentials and restricted network access. Repository contents, page text, and logs are untrusted inputs, not instructions that can override the agent’s task.

The agent should not have production access merely because it is helping with tests. For Web3 or hardware-connected applications, we use local chains, test environments, simulators, or controlled fixtures rather than valuable assets or live actuators.

Build a Google Cloud-first execution path

For a Google Cloud-oriented project, we can use Cloud Build to run containerized test stages, Artifact Registry to store runner images, and Cloud Storage to retain reports, screenshots, and traces. Vertex AI can provide the model interface for approved assistance workflows.

We keep responsibilities separate. The CI runner executes tests; a narrowly scoped identity invokes the model; artifact access follows project permissions. Secret Manager supplies required credentials without embedding them in test code or container images.

Browser workloads need deliberate runner configuration. We pin browser dependencies, provide sufficient memory, and check container sandboxing requirements instead of assuming every generic build image will work.

We set artifact expiration policies, model request limits, and timeouts. For advisory tasks such as failure summaries, a model outage should not prevent deterministic tests from reporting their results.

The same boundaries apply on AWS or Azure. We prefer Google Cloud where it fits, while adapting execution, storage, identity, and model services to the infrastructure a project already depends on.

Roll out gradually and measure the trade-off

We start with one painful, contained workflow: drafting unit-test cases, expanding an integration-test matrix, or summarizing browser failures. Introducing generation, autonomous execution, and automatic repair together makes it difficult to understand what actually helps.

For an initial pilot, we record review time, accepted versus rejected suggestions, test flakiness, model cost, and time spent diagnosing failures. We also inspect whether generated tests detect realistic defects, not just whether they pass.

We compare the new workflow with the existing one. If a generated patch takes longer to correct than a handwritten test, we adjust the context or narrow the task. We do not count generated lines as productivity.

In a remote team, we document prompts, test conventions, ownership, and escalation paths alongside the repository. Every failing check needs a clear route to resolution; an AI summary is helpful context, not an accountable maintainer.

A short checklist before adoption

• Identify high-risk journeys and approved expected behavior.
• Stabilize test data, environments, and existing CI checks.
• Choose one bounded AI task with a human reviewer.
• Sanitize inputs and restrict credentials, tools, and network access.
• Keep release gates based on reviewed, executable assertions.
• Track maintenance effort, useful defect detection, flakiness, and cost.
• Preserve a working path when the model is unavailable.

Build confidence, not just automation

AI-assisted testing works best when it strengthens an existing engineering discipline. We can use it to explore edge cases, draft repetitive code, and make failures easier to investigate. We still need explicit requirements, independent assertions, secure infrastructure, and people who own product quality.

At Kordspace, we bring software engineering, AI, and design into that shared workflow. If we can help turn a fragile test suite into a practical quality system, start a project with us, or explore our related services pages to see where we can collaborate.