Test Automation, AI and External Testing Support
Why AI and testing tools alone are not enough
1. The new gold rush: AI in test automation
There are phases in IT when one technology becomes so dominant that it seems to crowd out everything else. In the 2010s it was Agile. Then came DevOps. And now, in the mid-2020s, we are living through the AI gold rush in test automation.
The promises are tempting: generative AI writes unit tests in seconds. AI-powered tools generate Playwright scripts from screenshots. Self-healing test suites adapt automatically to UI changes. And reasoning models analyse requirements and derive test cases from them that would take a human tester hours or days.
All of that is real. All of that works. And all of that is — taken on its own — not enough.
Because between a working AI tool and a working test strategy lies a chasm that no machine, however clever, can build a bridge across. That chasm is called: context, experience and judgement.
2. What AI delivers in test automation today
Before we get critical, let us acknowledge what AI already delivers in test automation today — and does excellently:
Test-case generation: Reasoning models such as Magistral (Mistral AI) or Qwen3-Coder-Next can systematically derive test cases from user stories or requirements documents. In doing so they apply formal methods — equivalence partitioning, boundary-value analysis, state-transition diagrams — faster and more completely than most human testers.
Code-based test automation: Coding models such as Devstral 2 generate unit tests, integration tests and API tests directly from source code. They understand the context of entire classes and modules and can even create edge-case tests that a developer might have overlooked under time pressure.
UI test automation: AI-powered tools can create page-object models, maintain selectors when the DOM changes, and even detect visual regressions — tasks that were traditionally extremely maintenance-intensive.
Test-data generation: Generative models produce GDPR-compliant synthetic test data that is semantically correct — including consistent address data, plausible dates of birth and valid check digits (the latter 100% correct when deterministic validation engines are integrated).
Log analysis and root cause identification: AI models correlate error messages across microservices and identify root causes that a human would only have found after hours of digging through logs.
These capabilities are impressive. And they lead to a dangerous conclusion.
3. The dangerous illusion
Figure 2: AI in testing — opportunities and limits (image generated by AI).
“AI replaces the tester”
In some companies — especially where testing is perceived as a tiresome chore anyway — a narrative has taken hold that goes roughly like this:
“We now have Copilot and a few AI tools. They generate the tests for us. We no longer need dedicated testers. And external support least of all.”
This narrative is seductive. It promises cost savings. It fits the narrative of technological disruption. And it is fundamentally wrong.
Here is why:
AI has no context for your business
An AI model does not know that your insurance product “Comfort Plus” has had different coverage limits since last quarter. It does not know that the cancellation process for tariff X has a special regulatory rule implemented in the code as a hard-coded exception. It does not know the history of the system — why certain workarounds exist, which modules contain “fear code” that nobody touches, and which integration points wobble with every release.
An experienced test expert knows this. Or knows how to gather that knowledge systematically.
AI generates tests — it does not assess them
A reasoning model can derive 500 test cases from a specification. But which of them are important? Which cover the highest risks? Which are redundant? Which fit the available test time? Prioritising tests requires a risk assessment that combines technical knowledge, business understanding and project experience — something no model can deliver.
AI automates — it does not question
If the requirement is wrong, the AI tests the wrong thing — perfectly automated, but off target. An experienced tester asks the question: “Is that really what was meant? What happens if the user does X instead?” This exploratory, questioning mindset is the core of professional testing — and it cannot be automated.
AI does not recognise organisational risks
The most critical quality risks in software projects are often not technical: poor communication between teams, unclear responsibilities, time pressure that leads to shortcuts. An external test expert — with a fresh eye and no operational blindness — recognises these patterns. An AI does not.
4. The human-in-the-loop: not optional, but essential
The term “human-in-the-loop” is ubiquitous in the AI discussion, but is often understood as a formality — a tick on the compliance checklist. In test automation the human-in-the-loop is not a formality. It is the decisive authority.
The role of the test expert in the AI era
Instead of monotonous routine work, the test expert concentrates on what really counts:
Test strategy and test design: Which test types at which levels? Where does automation pay off, where is exploratory testing more effective? How do we distribute the test budget across unit, integration, API and E2E tests?
Risk-based prioritisation: Not everything that can be tested has to be tested. The expert assesses risks and focuses on the areas that cause the greatest damage in the event of a failure — commercially, in regulatory terms or in terms of reputation. Quality assurance of the AI outputs: the test cases generated by the AI have to be reviewed, extended and corrected. In practice we have regularly seen that AI-generated tests are technically correct but functionally pointless — or, conversely, that they ignore obvious business scenarios.
Exploratory testing: The most valuable bugs are not found by automated tests, but by experienced testers who deliberately use the system “differently” than intended. Session-based exploratory testing remains, even in 2026, the most powerful tool against unknown unknowns.
Test-process orchestration: Someone has to configure the CI/CD pipeline, manage the test environments, define the test-data strategies and interpret the results. AI can support parts of this, but the responsibility lies with the human.
5. Why external test expertise makes the difference
And now we come to the third element of the equation: external testing support. Because even when a company has the right AI tools and understands the human-in-the-loop, one decisive ingredient is often missing: specialised test competence at the cutting edge.
The competence problem
Most development teams have developers who “also test.” Some have dedicated QA engineers who concentrate on regression and bug triage in day-to-day work. But the combination of AI-powered test automation, modern frameworks (Playwright, Cypress), agentic workflows and risk-based test design requires a specialisation that is rarely present in internal teams.
Why? Because the landscape is changing so quickly. A year ago Selenium was still the standard. Today Playwright is the de-facto standard for UI test automation. Six months ago hardly anyone knew Devstral. Today it writes better tests than many a senior developer. Anyone who does not work in this ecosystem daily falls behind.
What external test experts bring
Cross-industry experience: An external test expert who has worked on 20 different projects brings pattern recognition that an internal team lacks. They have seen where typical architectures break, which integration scenarios are critical and which test strategies work in which contexts.
A fresh eye without operational blindness: Internal teams inevitably develop blind spots. They test the features they know, in the way they have always done it. An external expert questions assumptions, discovers neglected test areas and brings in new perspectives.
State-of-the-art knowledge: At SEQIS we work with the newest tools and models every day. We have spent hundreds of hours evaluating, optimising and integrating local AI models for test automation into workflows. That knowledge is available to our clients immediately — without a months-long internal learning curve.
Scalability: Before a release, during a migration or after a security incident you need more test capacity. And you need it fast. External specialists can be productive within days — with their own tools, their own methods and their own AI infrastructure.
6. The three-pillar model: AI + human + expertise
The future of test automation does not lie in an either/or decision. It lies in a three-pillar model.
Figure 3: Three-pillar model — AI + human + expertise (image generated by AI).
The model combines the strengths of each component:
Pillar 1: AI-powered automation. Local AI models (as on the rzfz.ai Box) take over the repetitive, data-intensive tasks: test-case generation, code review, test-data creation, log analysis, regression testing. They are the tireless workers producing around the clock.
Pillar 2: The test expert as conductor. The human test expert orchestrates, prioritises, questions and assesses. They define the test strategy, validate the AI outputs, carry out exploratory tests and ensure that the right thing is tested — not the obvious thing.
Pillar 3: External specialisation. External test partners bring the depth and breadth of experience that an internal team cannot build up (and does not have to). They know the newest frameworks, have already evaluated the AI tools and can provide test competence in a scalable way.
None of these pillars works on its own. AI without human control produces technically impressive but strategically pointless tests. Human expertise without AI support does not scale. And internal teams without external impulses ossify in their habits.
7. A real-world example: the symbiosis in action
Figure 4: Real-world example timeline — 4 phases of the symbiosis (image generated by AI).
Imagine a medium-sized company modernising a legacy ERP application. So far it has been tested manually — Excel lists, click by click, 200 test cases before every release.
Phase 1 — Analysis (external test expert): A SEQIS consultant analyses the existing test landscape, identifies the highest-risk modules and defines a test strategy. Result: a prioritised automation plan that shows where automation pays off and where exploratory testing remains more efficient.
Phase 2 — Automation (AI + expert): On the rzfz.ai Box, Devstral 2 generates Playwright tests for the identified core scenarios. The test expert reviews every generated test, adds missing business-logic validations and builds a stable page-object model.
Phase 3 — Operation (internal team + AI): The internal team takes over maintenance of the automated tests, supported by the self-healing capabilities of the local AI. Devstral automatically suggests adapted selectors when the UI changes. Magistral generates test data for new features.
Phase 4 — Continuous improvement (external impulse): Every quarter the SEQIS consultant reviews the test strategy, identifies gaps and integrates new models or techniques. They bring in experience from other projects and ensure that the test strategy keeps pace with business development. The result: 80% of regression tests automated, release cycles shortened from monthly to weekly, test coverage tripled — and yet the truly critical tests are found through exploratory testing by experienced experts, not by the AI.
8. Conclusion: AI is the tool — not the master
The temptation is great to delegate testing entirely to AI tools. The tools are good. They keep getting better. But they remain tools. And a tool — however sharp — does not replace the craftsperson who knows where to cut.
In a world where software keeps getting more complex, regulatory requirements rise and the costs of quality defects explode, we do not need less test expertise — we need different test expertise. Experts who orchestrate AI tools instead of fearing them. Who define test strategies instead of merely running tests. Who know the difference between “all tests green” and “the software really works”.
And we need external partners who bring this knowledge with them — current, cross-industry and at the cutting edge of technology. Not as a replacement for the internal team, but as reinforcement. As the decisive difference between “we test” and “we assure quality”.
AI delivers speed. The human delivers judgement. External expertise delivers depth. Together they deliver quality.
Sources and further reading
- QualityNews H2/2025 — AI Act & Local AI — seqis.com
- rzfz.ai — Local. Open Source. Independent. Secure. — rzfz.ai
- ISTQB Foundation Level Syllabus v4.0 — istqb.org
- Playwright — Modern web testing — playwright.dev
- Mistral AI — Devstral & Magistral — mistral.ai
- Session-Based Test Management (SBTM), Jonathan Bach — satisfice.com
Published in QualityNews H1/2026