Skip to content

docs(weave): Add preset signals to monitors page#2342

Draft
captainshar wants to merge 1 commit intomainfrom
scampbell/weave-signals-docs
Draft

docs(weave): Add preset signals to monitors page#2342
captainshar wants to merge 1 commit intomainfrom
scampbell/weave-signals-docs

Conversation

@captainshar
Copy link
Copy Markdown

Summary

  • Adds a new "Enable preset signals" section to the existing monitors page
  • Documents the suggested classifier monitors feature from PRs #40741 and #40742
  • Positioned before the custom monitors section so users discover the simpler option first

What's covered

  • 13 preset signals: 7 Quality signals (hallucination, low quality, user frustration, jailbreaking, NSFW, lazy, forgetful) and 6 Error signals (network error, ratelimited, request too large, bad request, bad response, bug)
  • Enable workflow: One-click from signal cards or batch via "Add signals" drawer
  • Manage workflow: View and remove active signals via "Manage signals" drawer
  • How signals work: Trace selection, prompt construction, LLM scoring, result storage, batching
  • Comparison table: Signals vs custom monitors

Open questions for reviewer

  • Should "signals" be the public-facing term, or "classifier monitors"? (Draft uses "signals" based on UI terminology)
  • Should this be a standalone page at signals.mdx or a section on the monitors page? (Draft adds it as a section — can be split later)
  • Is the feature ready for public docs, or should this wait for the weave_trace_analysis_frontend feature flag to be removed?
  • Should we mention the underlying model (gpt-oss-20b) by name, or keep it as "W&B Inference model"? (Draft uses the latter)

Sources

Source Purpose
PR #40741 Core CRUD infrastructure, classifier config, hook logic
PR #40742 Suggested signal cards UI, one-click enable
Existing monitors.mdx Page structure, style conventions
Slack/Glean internal discussions Feature naming, positioning, roadmap context

Test plan

  • Verify Mintlify renders the page correctly (tables, links, section anchors)
  • Confirm the cross-reference link to #how-to-create-a-monitor-in-weave resolves correctly
  • SME reviews signal descriptions against actual classifier prompts in defaultClassifiersConfig.ts
  • Verify the W&B Inference link /inference/models resolves

🤖 Generated with Claude Code

Adds documentation for the new suggested classifier monitors (signals)
feature introduced in wandb/core PRs #40741 and #40742.

Covers: 13 preset signals (7 quality, 6 error), enable/manage workflow,
how signals work under the hood, and comparison table vs custom monitors.

Inserted before the existing "How to create a monitor" section so users
see the simpler signals option first.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@captainshar
Copy link
Copy Markdown
Author

Resume prompt

I'm working on PR #2342 in wandb/docs (branch scampbell/weave-signals-docs). This adds documentation for the new "signals" feature (preset classifier monitors) to the existing monitors page at weave/guides/evaluation/monitors.mdx. The feature was implemented in wandb/core PRs #40741 and #40742 by jtschoonhoven. The draft covers all 13 preset classifiers (7 quality, 6 error), enable/manage workflows, and a comparison table. Key open questions: should "signals" be the public term, should this be its own page, and is the feature flag (weave_trace_analysis_frontend) removed yet for GA? Check PR comments for reviewer feedback and iterate.

@mintlify
Copy link
Copy Markdown
Contributor

mintlify bot commented Mar 23, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
wandb 🟢 Ready View Preview Mar 23, 2026, 8:01 PM

@github-actions
Copy link
Copy Markdown
Contributor

📚 Mintlify Preview Links

🔗 View Full Preview

📝 Changed (1 total)

📄 Pages (1)

File Preview
weave/guides/evaluation/monitors.mdx Monitors

🤖 Generated automatically when Mintlify deployment succeeds
📍 Deployment: bb8d500 at 2026-03-23 20:01:20 UTC

@github-actions
Copy link
Copy Markdown
Contributor

🔗 Link Checker Results

All links are valid!

No broken links were detected.

Checked against: https://wb-21fd5541-scampbell-weave-signals-docs.mintlify.app

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant