Skip to content

feat(coretime-alert): optional discord mention and --fake-alert drill - #1528

Merged
enthusiastmartin merged 1 commit into
masterfrom
coretime-alert-mention
Sep 21, 2026
Merged

enthusiastmartin merged 1 commit into
masterfrom
coretime-alert-mention

Conversation

@mrq1911

@mrq1911 mrq1911 commented Sep 17, 2026

Copy link
Copy Markdown
Member

why

Alerts currently post an embed with no top-level content. Discord does not send a notification for mentions inside an embed — only message content does. So today a coretime renewal alert lands silently in the channel and is only seen by whoever happens to scroll past it. The renewal right is genuinely lost at the region deadline, so a silent alert is close to no alert.

There was also no way to see what a real alert looks like without waiting for an actual deadline — --test only proves the webhook is reachable.

what

  • ALERT_MENTION (new, optional env var) — posted as top-level content above the embed, so alerts actually ping. Takes @here, <@&ROLE_ID> or <@USER_ID>. Sets allowed_mentions explicitly rather than relying on the webhook default, so a role not flagged "mentionable" still pings. Unset, the payload is byte-identical to today's.
  • --fake-alert (new flag) — posts one synthetic alert per configured chain (Hydration URGENT, Basilisk WARNING) through the real buildEmbed path. Each is titled 🧪 [DRILL] with a "this is a test alert" line and a synthetic drill footer so it can't be mistaken for a live deadline. Contacts no RPC and does not touch the state file, so it cannot disturb the cooldown of a real standing alert.
  • README: config-table row, the embed-doesn't-ping gotcha, ID formats, drill docs.
  • lark-stack.yml: ALERT_MENTION wired up.

Both chains were already watched — Hydration (task 2034, Polkadot) and Basilisk (task 2090, Kusama); no change to coverage.

testing

Dry-run verified with and without the var — unset produces exactly the current payload; set produces:

"content": "<@...>",
"allowed_mentions": { "parse": ["everyone","roles","users"] }

Not yet run against the live webhook — that happens at deploy.

deploy notes

Needs an image rebuild; autoredeploy on the lark coretime-alert stack picks up :latest, and ALERT_MENTION must be set on the service.

Unrelated, but worth flagging: DISCORD_WEBHOOK_URL is stored in plaintext in the stack env, so it surfaces in any Swarmpit service dump. The script already supports DISCORD_WEBHOOK_URL_FILE if we want to move it to a Swarm secret.

Copilot AI lite review requested due to automatic review settings September 17, 2026 10:34
@github-actions

Copy link
Copy Markdown

Runtime version has not been increased.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Two moderate issues and one documentation nit remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds optional Discord mentions and a synthetic --fake-alert drill to the coretime renewal watchdog.

Changes:

  • Adds top-level mention payloads with explicit mention permissions.
  • Adds synthetic alerts without RPC access or state changes.
  • Documents configuration and wires ALERT_MENTION into the stack.
File summaries
File Summary
scripts/coretime-alert/README.md Documents configuration and drill behavior; cooldown wording needs to distinguish test modes (nit, 3 votes).
scripts/coretime-alert/lark-stack.yml Wires ALERT_MENTION into the deployment stack.
scripts/coretime-alert/check.mjs Implements mentions and fake alerts; moderate issues remain with dry-run compatibility and configurable fake-alert shortfalls (1 vote each).
Review details

Suppressed comments (2)

scripts/coretime-alert/check.mjs:231

  • With ALERT_MENTION unset, this changes the --dry-run output from the previous { embeds } shape to { username, embeds }. That contradicts the stated byte-identical compatibility claim and the dry-run verification; either preserve the old diagnostic shape when unset or update the claim to distinguish the HTTP payload from the dry-run output.
    console.log(JSON.stringify(payload, null, 2));

scripts/coretime-alert/check.mjs:300

  • This fixture assumes the configured target is exactly three: it hard-codes one secured core and two pending renewals but derives shortfall from desiredCores - 1. With a valid override such as HYDRATION_DESIRED_CORES=1 or 4, the drill displays a zero/incorrect shortfall while still emitting an alert. Derive the synthetic core lists and shortfall from the configured target, or make the fixture explicitly independent of that setting.
      shortfall: chain.desiredCores - 1,
  • Files reviewed: 3/3 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +87 to +90
It applies to **every** alert the watchdog posts, including the throttled
`🔌 check failed` notice, `--test` and `--fake-alert`. Sends are already
rate-limited by `ALERT_COOLDOWN_HOURS`, so this is at most a couple of pings a
day per condition.
@mrq1911

mrq1911 commented Sep 17, 2026

Copy link
Copy Markdown
Member Author

drilled and deployed. --fake-alert posted both DRILL embeds through the live webhook with top-level content set, and the ping landed — so the mention path works end to end. lark now runs sha256:31e5f36e with ALERT_MENTION on the stack, re-confirmed from inside the running container.

basilisk was sitting at URGENT when I started (0/3 secured, 5.5d to region begin) and got renewed mid-deploy, so both chains are 3/3 now and no real alert is pending.

@github-actions

Copy link
Copy Markdown

Quick benchmark at commit 80b7e09 has been executed successfully.
View results

@enthusiastmartin
enthusiastmartin merged commit 1e57084 into master Sep 21, 2026
11 of 12 checks passed
@enthusiastmartin
enthusiastmartin deleted the coretime-alert-mention branch September 21, 2026 22:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants