forked from LayerLens/stratix-python
-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathgithub_actions_gate.yml
More file actions
38 lines (34 loc) · 961 Bytes
/
Copy pathgithub_actions_gate.yml
File metadata and controls
38 lines (34 loc) · 961 Bytes
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# AI Quality Gate for GitHub Actions
# Runs evaluation against a golden dataset using the LayerLens Python SDK.
# Blocks PRs if pass rate drops below threshold.
#
# Prerequisites:
# - LAYERLENS_STRATIX_API_KEY stored as a GitHub Actions secret
#
# Usage:
# Copy to .github/workflows/ai-quality-gate.yml
name: AI Quality Gate
on:
pull_request:
branches: [main]
paths:
- "src/**"
- "prompts/**"
- "agents/**"
env:
LAYERLENS_STRATIX_API_KEY: ${{ secrets.LAYERLENS_STRATIX_API_KEY }}
PASS_RATE_THRESHOLD: "0.85"
jobs:
evaluate:
name: Run AI Evaluation
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install layerlens
- name: Run evaluation gate
run: python samples/cicd/quality_gate.py --threshold $PASS_RATE_THRESHOLD