Create alert system with customizable rules #255
Description
Description
Build alerting system for monitoring agent performance and triggering notifications based on conditions.
Alert Rule Features
- Metric selection (cost, latency, error rate, token usage)
- Operator configuration (>, <, ==, !=, contains)
- Threshold settings
- Duration requirements (sustained for X seconds)
- Alert actions (email, webhook, UI notification)
Alert Types
- Cost threshold exceeded
- Error rate spike
- Latency degradation
- Token limit approaching
- Agent failure
- Generation timeout
Implementation
- Background job for alert checking (using SolidQueue)
- Alert state machine (pending, triggered, acknowledged, resolved)
- Alert history tracking
- Cooldown periods to prevent spam
- Alert grouping and deduplication
Notification Channels
- In-app notifications with Turbo
- Email notifications (Action Mailer)
- Webhook integrations
- Slack/Discord webhooks (optional)
Acceptance Criteria
- Alerts trigger based on conditions
- Notifications are delivered
- Alert history is maintained
- No duplicate alerts within cooldown