pubnub-illuminate: runtime-correctness guidance (decision false positives, ingestion lag, verify loop)

[tgreene]

Why

The pubnub-illuminate skill is solid on how to build each resource (the Decision 4-step workflow, required-fields-or-500, immutability, cascade-delete, dashboard full-replace PUT are all well covered). The gap is the runtime-correctness layer — why a syntactically valid config still behaves wrong. The most painful instance, hit in real Illuminate work: a Decision that fires on every evaluation window regardless of real activity.

Root cause: a single PubNub channel usually carries several message/event types, and a Business Object ingests all of them. An unfiltered measure metric (AVG/SUM/MIN/MAX) therefore averages across unrelated messages — records lacking the measured field contribute nothing, so the aggregate collapses toward 0. A NUMERIC_LESS_THAN rule then trips on every idle window (0 < X). This was verified end-to-end against a live keyset: an unfiltered < threshold decision fired every minute; adding a STRING_EQUALS event-type filter to the metric and flipping to NUMERIC_GREATER_THAN above baseline made it fire only on genuine spikes.

Changes (docs only)

• decisions-4-step-workflow.md
  • New "Avoiding False Positives (a Decision That Fires Every Window)" section: filter the source metric to one event type; prefer NUMERIC_GREATER_THAN above baseline (an empty window reads 0, which > X does not satisfy).
  • Two decoder-ring rows (fires-every-window; action logged but message never delivered).
  • PUBNUB_PUBLISH note: the target channel must be a valid PubNub channel — an invalid/over-nested channel (a.b.c.d, >3 dot levels) was observed to log as unsuccessful and never deliver.
• metrics.md — reframe filtering as a correctness requirement for measure metrics (not only a cost/cardinality tip), plus a Common Errors row.
• business-objects.md — state that a BO ingests every message type on the channel, so downstream metrics must filter by event type; suggest mapping an event/type discriminator field.
• SKILL.md — add the ~20–30s ingestion/aggregation lag to Constraints (verifying too early reads as "broken"), a false-positive pointer, and a "verify before declaring success" output step (confirm fields populate + the decision actually fired).

All additions are general (not app-specific) and shape-agnostic — they don't change any documented request JSON.

Notes for maintainers (flagged, not changed)

Two things I couldn't confirm against the raw admin-api (my hands-on was via the manage_illuminate MCP tool, same backend) and deliberately left untouched:

1. Metric filter object shape — metrics.md documents { fieldId, operation, value }; the backend I exercised used { sourceType, sourceId, operation, arguments: [...] }. If the documented shape isn't accepted by the versioned API, a copy-pasted filter would be silently ignored (reproducing the very misfire above). Worth a quick verify.
2. Dashboard decision overlay — dashboards.md controls the overlay via decisionIds alone; I always also set showDecisions: true. Worth confirming decisionIds alone renders the trigger overlay.