Tests that early-return on an unsupported platform report as PASSED, so the executed count overstates coverage

[monkopedia-coder]

RpcFunctionalityTest is a shared base used by 20 test classes. Each of its five tests opens with a guard that returns before doing anything:

@Test
fun testSerializePassthrough() = runBlockingUnit {
    if (TestType.SERIALIZE !in supportedTypes) return@runBlockingUnit
    …
}

testPipePassthrough, testHttpPassthrough, testWebsocketPassthrough and testServiceWorkerPassthrough have the same shape (RpcFunctionalityTest.kt:73, 89, 132, 173, 203).

A test that returns early reports as PASSED, not skipped. It contributes a green line and a tests="1" to the XML while asserting nothing. So the executed count — the number this repo has been leaning on all evening to distinguish "ran" from "did not run" — overstates coverage here, and no amount of checking that a task ran, or that a filter matched, or that results are fresh will reveal it.

The guard set differs per platform (platformSupportedTestTypes()):

platform	supported	so these no-op and pass
jvm	all but SERVICE_WORKER	1 of 5
native	SERIALIZE, PIPE, HTTP	2 of 5 (WEBSOCKET, SERVICE_WORKER)
js / wasmJs	SERIALIZE, PIPE, + SERVICE_WORKER only if (hasWindow())	2 or 3 of 5

Across 20 classes that is a large number of green test lines that ran no assertions, and the count varies by platform in a way nothing reports.

Worth being clear about what is and is not wrong here. The mechanism is legitimate — a websocket test genuinely cannot run on a platform with no websocket support, and the alternative of duplicating the class per platform is worse. The defect is that skipping is indistinguishable from passing in the output. kotlin.test has no portable assumption/skip primitive, which is presumably why it was written this way.

Options, none free:

• Have the guard fail rather than return when a type is unsupported and the platform claims to support it — catching the case where platformSupportedTestTypes() drifts out of date and quietly disables a test that should run.
• Emit a marker (a log line, or a deliberate expected skip mechanism per platform) so a reader can count no-ops.
• Split the class so each platform's test set is what it actually runs, at the cost of the duplication the base class exists to avoid.

Concrete instance worth resolving first, since it is measured rather than hypothetical: the js suite executed 383 tests on my machine and 379 on a GitHub runner (PR #259), same code. Four tests were not merely no-ops but were not generated at all. hasWindow() differing between a local headless Chromium and the runner's is the obvious candidate, and it means the js suite's SERVICE_WORKER coverage depends on which machine ran it — invisibly, because the missing tests were never counted and the no-op ones passed.

Found by applying sdbus's #227 finding (a suite reporting 7 passing tests with its dependency absent) to this repo.

[monkopedia-coder]

Applying kcsg's variant of this check — a conditional wrapping an assertion with no comment explaining it — turned up six concrete instances, all the same guard:

ksrpc-test/src/jsTest/kotlin/ServiceWorkerTest.kt:32, 44, 131
ksrpc-test/src/wasmJsTest/kotlin/ServiceWorkerTest.kt:32, 44, 131

    if (!hasWindow()) return@runBlockingUnit

Six tests that pass without asserting anything whenever the browser has no window, with no comment saying so. Same mechanism as the RpcFunctionalityTest guards above, but written per-test rather than via platformSupportedTestTypes(), so a reader scanning for the pattern in one place would miss them.

One apparent hit that is not one, recorded so nobody re-flags it: CurrentRpcCallTest.kt:232, if (value == "outer"). That is a dispatch on which invocation is running, and both branches assert — the outer branch checks the call element is non-null, the inner records a fresh id. Not a narrowed domain.

What I have not done, and it is the decisive half of the check: remove the hasWindow() guards and run the suites. RED would mean the condition was hiding a defect; GREEN would mean the guard is unnecessary. My expectation is neither — that the tests genuinely cannot run without a window and the guard is correct but undocumented — but that is a prediction, not a result, and the check costs one run of a suite that is currently red for other reasons (#255, #258).

So the actionable item for these six is the smaller one: a comment stating why the guard exists, so the next reader can tell a deliberate platform limit from an assertion someone quietly narrowed. That is precisely the distinction that let kcsg's difference() defect survive being reported once already.

[monkopedia-coder]

Triaged agent-workable, scoped -- dispatching part of this and deliberately not all of it. Recording the split here rather than only in the dispatch, so it is citable on review.

In scope:

1. The six undocumented hasWindow() guards (ServiceWorkerTest.kt:32,44,131 in both jsTest and wasmJsTest). The actionable item the comment above already identifies: a comment stating why the guard exists, so a deliberate platform limit is distinguishable from an assertion someone quietly narrowed.
2. The 383-vs-379 discrepancy. This is the measured half and the most useful piece of the issue: same code, 383 tests generated locally and 379 on a GitHub runner, with four tests not merely no-op'd but never generated at all. Whether hasWindow() differs between a local headless Chromium and the runner's is checkable, and the answer matters past this issue -- it means the js suite's SERVICE_WORKER coverage depends on which machine ran it.
3. A countable marker for no-op'd tests -- option 2 of the three listed above, the least invasive of them. Something a reader or a script can use to tell "20 classes x 5 tests, all green" from "some of those asserted nothing."

Out of scope, deliberately:

• Making the guard fail when a type is unsupported but the platform claims to support it (option 1). It is a good idea, and it changes the pass/fail behaviour of a base class shared by 20 test classes -- worth landing on its own rather than folded into a reporting fix.
• Splitting RpcFunctionalityTest per platform (option 3). The issue itself notes that the duplication is exactly what the base class exists to avoid.

Neither exclusion is an owner question; both are sequencing. If the review of the scoped work suggests option 1 is the right end state, it is a follow-up issue rather than a decision anyone is waiting on.

One practical note for whoever picks this up: items 1 and 3 are verifiable locally -- Chromium is available and the reproduce line in #255 works -- while item 2 needs a comparison against a runner, since the 379 figure came from one.

[monkopedia-coder]

Checked as a close candidate and deliberately not closed. Labels unchanged (bug,
agent-workable); recording what landed and what did not, so nobody re-dispatches the finished half.

PR #267 merged 2026-08-26 using Addresses #261
rather than a closing keyword. Verified on origin/main: PlatformSkips.kt exists with
SKIP_MARKER / skipUnsupported(test, reason), and the guards in RpcFunctionalityTest and both
ServiceWorkerTest copies route through it.

Landed — items 1 and 3 of the 2026-08-22 scope: every platform guard now emits a countable
marker, and the guard count turned out to be 14 rather than the 6 recorded in the comment above.
The number nobody had: between 8.2% (jvm) and 14.8% (linuxX64) of reported tests assert nothing.

Not landed, and this issue is the tracker for it — the shipped KDoc says so in as many words:

 * Turning a fired guard into a *failure* when the platform claims to support the capability
 * would be a stronger signal, but changes the pass/fail behaviour of a base class shared by
 * 20 test classes — deliberately out of scope here (issue #261).

Still open under this number:

1. Option 1 — make the guard fail when a type is unsupported while the platform claims to
   support it. Explicitly out of the dispatched scope, and explicitly not an owner question: the
   scope comment above calls both exclusions sequencing.
2. The per-class supportedTypes opt-outs Make platform-skipped tests countable instead of silently green #267 surfaced and did not touch. jvm skips
   testWebsocketPath 4 times and testHttpPath 3 times; every platform skips SERIALIZE/PIPE a
   few times. Those come from per-class overrides rather than from the platform — classes opting out
   of a capability their platform does have. That is a new finding from the fix, not part of the
   original filing, and option 1 is what would catch it.
3. Item 2 (383 vs 379) is reframed, not resolved. Make platform-skipped tests countable instead of silently green #267 showed hasWindow() cannot produce that
   symptom — it is evaluated inside the test body, so a guarded test still exists and still counts.
   The live version of this is same-machine nondeterminism (388/395/395 on one unchanged commit),
   which is tracked on First-ever jsTest run: 371 of 383 pass, 12 fail across four classes #255 and referenced from the shipped KDoc. The local-vs-runner comparison was
   never made.

Two downstream findings from reviewing #267 are already filed separately and are not part of
this issue: #273 (the documented counting recipe double-counts on wasm) and #272 (ServiceWorkerTest
reports zero test cases on both browser targets, so 14 of those markers can never reach the XML at
all).