Over three weeks, I turned the failures that slipped through a local OpenCode/Qwen loop into six Jmix-specific verification gates: change contracts, tool targeting, authorization, persistence, independent acceptance, and the browser.
On paper, the feature was finished: Gradle passed, the XML looked plausible, and the searchable entity picker rendered in the browser. The problem appeared only when I tried to use it.
The picker was supposed to find a database record as the user typed, but the first search produced:
DevelopmentException:
Parameter 'searchString' is not used in the query
The query never referenced :searchString, and that defect had survived compilation, the agent’s own review, and a basic render check. Although the view existed, the interaction the task was built around did not work.
An earlier Jmix article covered the basic setup: context files, MCP tools, and browser checks. This follow-up starts where that setup leaves off: the false greens that still survive it.
Jmix puts entity metadata, view XML, data containers, transactions, access policies, and a Vaadin web UI in predictable places, which makes failures like this practical to encode as reusable checks. The bug set off a three-week refinement loop driven by one question: what evidence should a coding agent produce before anyone accepts a Jmix feature as done?
I began calling the result a harness: versioned instructions and reusable skills, verifiers that execute checks, and gates that turn the results into pass/fail decisions. The files did not execute or certify themselves; I ran the checks and made the final call on every rule. The Jmix team later packaged the material as a public toolkit.
What “Done” Had to Mean
By the end of the loop, done meant six things: the code compiled, a Spring context test actually ran, the view opened, the interaction worked, authorization behaved correctly for the intended user, and the owner’s checks passed from a clean state.
| False green | Why a build can miss it | Check intended to expose it |
|---|---|---|
An itemsQuery ignores :searchString |
The XML and JPQL remain valid | Type and verify the returned options |
| A role lacks an entity or attribute policy | The role is valid but the operation is unusable | Test positive and negative cases as the intended users |
| A required default is assigned only in the view | UI creation works; direct persistence bypasses it | Save and reload outside the UI |
| A composition editor uses a disconnected collection | Types are valid but changes miss the master entity | Create, edit, remove, save, and reload |
Four failures made the pattern obvious. The value is in the mapping: a change to a Jmix view, role, query, or persistence path already has a matching check. Tool targeting and independent acceptance became two additional gates around that core.
How the Failure Loop Worked
The loop ran for roughly three weeks. OpenCode drove a locally served 235B Qwen model. The task, rubric, and maintainer-authored reference project came from a benchmark fixture built for this experiment. Claude reviewed that disposable fixture; no customer source code or production data entered the loop. I decided which proposed rules to keep, sometimes writing the missing rule myself.
Across successive guideline revisions, the share of rubric checks Qwen passed on the same adaptive task rose from roughly 50% to 84%. That was the harness doing its job: each escaped failure became the next rule or verifier. The six gates below are what survived that loop.
Six Gates a Build Cannot Exercise Alone
Gate 1: Change Contract
“Follow Jmix best practices” gives an agent nothing testable. The root AGENTS.md instead named the artifacts implied by a change, its required checks, and the evidence to provide when a verifier could not run. It organized verification into three broad layers: static, context, and render. The rest of the harness made those layers concrete for six Jmix boundaries.
Here is the core of that contract:
A task is not done after the code compiles.
1. Static layer:
verify new symbols and inspect every changed Java and view XML file.
2. Context layer:
run `./gradlew --no-daemon clean test`;
confirm that a named `@SpringBootTest` executed.
Zero executed tests is a failed check.
3. Render layer:
open every changed view and exercise the changed interaction.
Never report a skipped, ambiguous, or failed check as passed.
Each skill recorded one failure, its correct Jmix pattern, and the check that should catch it next time; the MCP-first revision shows those verification artifacts being added.
The picker failure became this Jmix 2.8 pattern:
<entityComboBox dataContainer="employeeDc"
property="department"
pageSize="30">
<itemsQuery class="com.company.app.entity.Department"
searchStringFormat="(?i)%${inputString}%"
escapeValueForLike="true"
fetchPlan="_instance_name">
<query><![CDATA[
select e from Department e
where e.name like :searchString escape '\'
order by e.name
]]></query>
</itemsQuery>
</entityComboBox>
Under the Jmix 2.8 entityComboBox contract, itemsQuery@class is the fully qualified Java class, while Department in the JPQL is the JPA entity name. An entity declared as @Entity(name = "app_Department") therefore requires app_Department in the query. The acceptance check must type a partial value and verify the filtered options.
Gate 2: Tool Targeting
With several projects open and no projectPath, JetBrains MCP returned one line: No exact project is specified. Nothing had been inspected, yet the agent was ready to interpret the result as “no problems found.”
A later toolkit commit made projectPath mandatory on every JetBrains MCP call and required the agent to confirm that the intended file was actually inspected. The absolute path stays local or is redacted before logs are shared. Any ambiguity fails the gate. I pinned documentation lookups to the application’s Jmix release line because “latest” may be wrong for a Jmix 2.8 project. The lesson was simple: an empty or misdirected tool result must fail loudly.
Gate 3: Intended-User Authorization
Jmix provides server-side UI integration tests through jmix-flowui-test-assist. They load the Spring context, open views, operate the server-side component tree, and persist data without launching an external browser.
The authorization-coverage trap is the test identity: by default, @UiTest uses SystemAuthenticator to act as the built-in system user. In a standard generated application, that user has system-full-access, which means the test does not validate restrictions assigned to an ordinary target user. This is a test setup convenience, not a runtime authorization bypass.
Role-sensitive tests should therefore authenticate the intended user, commonly through a custom UiTestAuthenticator, and exercise a constrained DataManager or the actual security-enforcing service API. Navigation and menu visibility need their own checks, separate from field state and CRUD actions. Custom actions require UI Constraints or explicit permission checks, while row-level roles need both matching and non-matching data assertions.
Gate 4: Persistence Boundaries
Persistence introduced a boundary that compilation would never reveal. A before-commit EntityChangedEvent covers ordinary entity changes processed through DataManager and Jmix data repositories. Jmix’s container-managed EntityManager also participates: JmixEntityManager.flush() dispatches the change, and testContainerEm asserts the event.
Writes that bypass JmixEntityManager sit outside that mechanism; an unwrapped provider-level manager is one example. JDBC, native SQL, bulk JPQL updates, triggers, and external writers likewise bypass per-entity event generation. EntitySavingEvent and EntityLoadingEvent do not fire for EntityManager operations. An after-commit listener that accesses data needs a new transaction, normally REQUIRES_NEW.
Gate 5: Independent Acceptance
Claude’s review meant Qwen was not grading its own work, but Claude also helped change the harness between runs. Its verdict therefore belonged to the repair loop. This gate passes only when a separate runner starts from an immutable revision after coding stops, uses owner-controlled tests and fixtures under a separate service identity, and reproduces the acceptance result from a clean state.
Gate 6: Real-Browser Behavior
Server-side @UiTest exercises Vaadin’s server-side component tree and navigation logic, but it does not validate navigation in a real browser, client-side web components, CSS, or responsive layout.
I saw that gap in a disposable, agent-generated Jmix 2.8.2 Meetups prototype built for the experiment. It had 15 @Route classes, four admin CRUD areas, composition editing, roles, and custom public pages. I used Playwright MCP during development.
The admin side already looked conventional: generated list and detail views handled the ordinary CRUD path, including a nested speaker collection in the Meet editor. That was useful evidence, but not acceptance; save-and-reload still belonged to Gate 4.
Figure 1. The generated Meet list in the running Jmix prototype
Figure 2. The real Meet editor, including its nested speaker collection
The run exposed route-registration failures, MetaClass not found for meet_Meet, detached avatars, and, at one point, a catalog collapsed into raw text and empty space. Qwen proposed repairs, browser checks found what remained, and I made the design decisions.
First stable render: after those failures were fixed, the route was usable, but the hierarchy was still thin and the page looked like a scaffold.
Browser-guided render: another repair pass turned the catalog into a branded page with clear navigation, stronger card hierarchy, status chips, and consistent calls to action.
First stable render
Browser-guided render
Figure 3. Before and after the browser-guided visual repair pass
Both snapshots opened the route, but only the later catalog was usable; duplicated CSS, thin test coverage, and broken entity-name queries elsewhere still kept the prototype far from production-ready.
Moving the Harness to Kilo
Kilo can load the harness, but not the confidence earned by another agent’s runs. It supports a root AGENTS.md, skills in .kilo/skills/ or compatible .agents/skills, and local or remote MCP servers. Project configuration belongs in kilo.jsonc or .kilo/kilo.jsonc, and Kilo does not read .opencode directories. More detailed instructions on migrating guidelines to Kilo are available on Kilo.ai.
Try It on One Real Task
For a first trial, I’d pick one bounded backlog item, three to five acceptance scenarios, and a disposable environment with no production credentials. Treat MCP servers, browser tools, Git access, telemetry, and logs as separate data paths; an NDA alone does not control them. Measure final acceptance, escaped defects, repair cycles, reviewer time, and tool cost, then keep the pinned versions, gate results, final diff, and residual risks.
The Jmix Agent Toolkit is public, so choose the branch for your Jmix version, load the skills into the coding agent you already use, and run the six gates against one disposable task.
The agent can write the diff; your acceptance criteria still decide whether it is done.
