Tests from the phone, coverage criteria included
Six levels of tests, coverage criteria from statements to MC/DC, and why having the results on the phone is an advantage.
The Project → Tests tab is not just a list of what is failing. It is where you commission tests, read their history and click through from a failing case to what it actually verifies — all from the phone, without a single line of prompt.
Levels
Tests are divided into six levels: unit, integration, system, E2E, UI performance and backend performance. They switch at the top and each keeps its own history of Runs.
That division is not cosmetic. A failing unit test means something different from a failing E2E — and when you see both in one list, you lose exactly the information the levels exist for.
Coverage criteria — and why at all
When you have tests written by saying “write tests”, you get tests. The question is what they verify — and nobody has an answer to that until something falls over in production.
A coverage criterion is a rule saying when there are enough tests. It is not a percentage; it is a sentence that can be checked:
- Statement coverage
- Every statement runs at least once. The cheapest and the weakest — the code ran, but that says nothing about its branches.
- Branch coverage
- Every condition comes out true once and false once. This is where tests start finding bugs.
- MC/DC
- For every sub-condition it is shown that it changes the outcome on its own. Expensive, but with compound conditions it is the only way to find out that one of them is redundant.
- Pair-wise
- Every pair of values from different characteristics meets in at least one test. A fraction of the combinations, most of the bugs.
In the app you pick the criterion when commissioning tests and you see how demanding it is — before the Agent starts working. That is the difference from “write tests”: you know what you will get, and the Agent knows when to stop.
Have the Agent draw up an overview of the criteria for your project. It will add how demanding each is right here — they are one thing for a parser and another for a form.
Why having it on the phone is an advantage
- Results read without a connection too. Runs are downloaded, so you can dig through them on a plane.
- From a failing case there is a link to its description in the documentation and to a screenshot from the moment it fell over. Not to a hundred lines of log.
- History across Runs reveals a flaky test — the one that passes once and not the next time. In a terminal you remember it, here you see it.
- Measured values in performance tests have a course across Runs. One number says nothing about a measurement, the direction does.
- Commissioning further tests is a few taps, not writing a prompt. You pick the level and the criterion, the Agent does the rest.
Where the results come from
The app reads JUnit XML, the format nearly every testing tool can spit out. The Agent writes it into the project and the phone downloads it with the mirror — no service in between, nothing uploaded anywhere.
The shape the results are written in is set by an Application skill. Skills are put into the project from the Project tab, the Application skills section; without them you see only what is already in the project.
Stuck somewhere other than what is here? Write to support@coden-app.com.