Record being wrong about which change broke a test
A UI test went red in the same run that adopted three new APIs. The one touching that screen was reverted; the test still failed. A modifier order was corrected; still failed. Simulator data was wiped; still failed. Running it against a clean checkout — which should have been the first move, not the fourth — it failed there too, with none of the work present. The scrolling entry had already been rewritten to explain the fault in confident detail. That write-up was fiction, so it now says what actually happened: the API was convicted on circumstance and remains unproven here, and the one claim that survives is about its two-way binding costing invalidations. xcode-practice gets the general lesson, since it will outlive this particular test: stash and rerun before reasoning about which edit did it, and a note recording a false cause is worse than no note, because the next person believes it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
1 parent
f5a3d932fa
commit
542dd925d0
4 files changed
+46
-5
No files matched your search
@@ -136,6 +136,27 @@ dead when it is passing. Also: do not pipe a test run through `tail` or a
|
||||
narrow `grep` and then diagnose from what survived — filter the saved log
|
||||
afterwards instead.
|
||||
|
||||
## Check a failure against an unchanged tree before blaming the change
|
||||
|
||||
A UI test started failing in the same run that introduced three API
|
||||
changes. The obvious suspect — the one touching that exact screen — was
|
||||
reverted, and the test still failed; a modifier order was corrected, still
|
||||
failed; the simulator's app data was wiped, still failed. Only then was
|
||||
the test run against a clean checkout with none of the work present, and
|
||||
it failed there too. The change had been innocent all along, and two
|
||||
rounds of edits went into explaining a fault that did not exist.
|
||||
|
||||
`git stash` and one run is cheaper than any amount of reasoning about
|
||||
which edit did it. Do it first, not fourth. The same goes for what gets
|
||||
written down afterwards: a note recording a false cause is worse than no
|
||||
note, because the next person believes it.
|
||||
|
||||
Corollary for UI tests: when one goes red without a code change, suspect
|
||||
what the test depends on but does not control — accumulated seed data,
|
||||
`@AppStorage` surviving between launches, a simulator location and the
|
||||
"nearby" sort order it drives, or an element scrolled out of a lazy
|
||||
container and therefore missing from the accessibility tree entirely.
|
||||
|
||||
## Core Data in tests
|
||||
|
||||
Build objects through the context's own model rather than
|
||||
|
||||
Reference in new issue
Block a user