Skip to content

refactor: Expose agent scorecard weight variables to custom configurations#2799

Open
singhanurag0317-bit wants to merge 1 commit into
ritesh-1918:mainfrom
singhanurag0317-bit:fix/issue-2635-agent-scorecard-weights
Open

refactor: Expose agent scorecard weight variables to custom configurations#2799
singhanurag0317-bit wants to merge 1 commit into
ritesh-1918:mainfrom
singhanurag0317-bit:fix/issue-2635-agent-scorecard-weights

Conversation

@singhanurag0317-bit

Copy link
Copy Markdown

Summary

Enables weight parameters inside compute_performance_score to be customized via environment variables or explicitly passed dict arrays to adjust resolution metrics.

Changes

  • Created backend/services/agent_scorecard.py with AgentScorecard class
  • Configurable via SCORECARD_RESOLUTION_WEIGHT, SCORECARD_SPEED_WEIGHT, SCORECARD_QUALITY_WEIGHT, SCORECARD_VOLUME_WEIGHT, SCORECARD_SLA_WEIGHT, SCORECARD_MAX_SCORE env vars
  • compute_performance_score() accepts optional per-call weight overrides via weights dict
  • compare_weights() lets you compare multiple weight configurations
  • Runtime update_weights() for dynamic adjustments
  • Weight normalization warning when sum != 1.0
  • Singleton load() / get_instance() pattern

Closes #2635

…tions

Implements an AgentScorecard that allows weight parameters inside
compute_performance_score to be customized via environment variables
or explicitly passed dict arrays. Supports runtime weight updates.

Closes ritesh-1918#2635
@vercel

vercel Bot commented Jun 12, 2026

Copy link
Copy Markdown

Someone is attempting to deploy a commit to the ritesh Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Jun 12, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@singhanurag0317-bit, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 25 minutes and 28 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans include higher PR review limits than trial, open-source, and free plans. In all cases, reviews become available again over time. During sustained high-volume PR review activity, CodeRabbit may temporarily slow when the next review becomes available.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4b820e0f-47ca-4bd1-b042-982cfb9e975a

📥 Commits

Reviewing files that changed from the base of the PR and between da8faf2 and 46b2455.

📒 Files selected for processing (1)
  • backend/services/agent_scorecard.py
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: Expose agent scorecard weight variables to custom configurations

1 participant