Conversation
Enrich analytics by integrating Prompture usage tracking across backend, storage, CLI and frontend. Backend: /api/agents/stats now includes cost_by_model and daily/monthly totals when tracker is available; new endpoints added for /stats/models, /stats/providers, /stats/today and /stats/session/{id}. Generation pipeline uses prompture.infra.session.UsageSession (graceful fallback if missing) to build combined usage and per-model breakdown; tracker contexts are cleaned up after runs. Storage/models: AgentRun gained session_id, DB migration adds session_id column and repository persists it; cost backfill now uses prompture.infra get_model_rates for rate-based cost calculation. CLI: show per-model cost breakdown when available. Frontend: new API calls (today/models), analytics hook merges tracker totals with stats, and Analytics page displays Today and This Month KPIs and adjusts layout. Also added two .md files to .gitignore.
There was a problem hiding this comment.
Pull request overview
Integrates Prompture usage tracking to enrich analytics across the backend, storage, CLI, and frontend, adding session-level attribution and new KPIs (Today / This Month).
Changes:
- Backend stats endpoints enriched with Prompture ledger data and new
/stats/*endpoints (models/providers/today/session). - Agent run persistence extended with
session_id, including DB migration and repository insert/backfill updates. - Frontend analytics hook + page updated to display new cost KPIs, with a fallback call to
/stats/today.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/pages/AnalyticsPage.jsx | Adds “Today” and “This Month” KPI cards and adjusts KPI grid layout. |
| frontend/src/hooks/useAnalytics.js | Fetches /stats/today and merges cost fields into existing stats response. |
| frontend/src/api/agents.js | Adds API helpers for new stats endpoints (today, models). |
| agentsite/storage/repository.py | Persists session_id, updates cost backfill to use model rates, maps DB rows to AgentRun. |
| agentsite/storage/database.py | Adds session_id to schema and incremental migration to add the column if missing. |
| agentsite/models.py | Adds session_id to AgentRun model. |
| agentsite/engine/pipeline.py | Introduces a Prompture UsageSession-based combined usage summary and assigns session_id to runs. |
| agentsite/cli.py | Prints per-model cost breakdown when available in usage payload. |
| agentsite/api/routes/agents.py | Enriches /stats response and adds new stats endpoints (models/providers/today/session). |
| .gitignore | Ignores two Prompture tracking markdown files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| input_tokens=row["input_tokens"], | ||
| output_tokens=row["output_tokens"], | ||
| cost=row["cost"], | ||
| session_id=row["session_id"] if "session_id" in row else "", |
There was a problem hiding this comment.
"session_id" in row is checking values rather than column names for sqlite3.Row/aiosqlite rows, so this will usually evaluate false and drop session_id even when the column exists. Prefer accessing row["session_id"] directly (since the migration guarantees the column) or use a key-based check like "session_id" in row.keys() / try-except KeyError.
Suggested change
| session_id=row["session_id"] if "session_id" in row else "", | |
| session_id=row["session_id"] if "session_id" in row.keys() else "", |
Comment on lines
+186
to
+187
| runs = await repo.list_recent(limit=100) | ||
| session_runs = [r for r in runs if r.session_id == session_id] |
There was a problem hiding this comment.
This endpoint loads only the 100 most recent runs and then filters in Python, which can miss older runs for the requested session_id and is inefficient as run volume grows. Consider adding a repository query that filters by session_id in SQL (optionally with ordering/limit) and use that here instead of list_recent(limit=100).
Suggested change
| runs = await repo.list_recent(limit=100) | |
| session_runs = [r for r in runs if r.session_id == session_id] | |
| session_runs = await repo.list_by_session_id(session_id=session_id) |
Comment on lines
+74
to
+99
| try: | ||
| from prompture.infra.ledger import ModelUsageLedger | ||
|
|
||
| ledger = ModelUsageLedger() | ||
| all_stats = ledger.get_all_stats() | ||
| cost_by_model = { | ||
| s["model_name"]: round(s["total_cost"], 4) | ||
| for s in all_stats | ||
| if s.get("model_name") and s.get("total_cost", 0) > 0 | ||
| } | ||
| stats["cost_by_model"] = cost_by_model | ||
|
|
||
| # Compute cost_today and cost_this_month from agent_runs table | ||
| from datetime import datetime, timezone | ||
|
|
||
| now = datetime.now(timezone.utc) | ||
| today_start = now.replace(hour=0, minute=0, second=0, microsecond=0).isoformat() | ||
| month_start = now.replace(day=1, hour=0, minute=0, second=0, microsecond=0).isoformat() | ||
|
|
||
| today_stats = await repo.get_stats(since=today_start) | ||
| month_stats = await repo.get_stats(since=month_start) | ||
| stats["cost_today"] = today_stats.get("total_cost", 0.0) | ||
| stats["cost_this_month"] = month_stats.get("total_cost", 0.0) | ||
| except Exception: | ||
| pass | ||
| return stats |
There was a problem hiding this comment.
The broad except Exception: pass will silently swallow enrichment failures (including unexpected runtime errors), making it hard to debug missing analytics fields in production. Please at least log a warning (and ideally scope the try/except to the Prompture import vs. runtime errors) so failures are observable while still returning the base stats.
Comment on lines
+707
to
+713
| # Clean up tracker agent contexts if any were left open | ||
| for cm in self._tracker_agent_cms.values(): | ||
| try: | ||
| cm.__exit__(None, None, None) | ||
| except Exception: | ||
| pass | ||
| self._tracker_agent_cms.clear() |
There was a problem hiding this comment.
_tracker_agent_cms is never populated anywhere in this file (only initialized and cleared), so this cleanup block is currently dead code. Either remove _tracker_agent_cms/this finally cleanup, or implement the corresponding tracker context creation and ensure the correct exit method is used (__exit__ vs __aexit__) based on the context manager type.
Suggested change
| # Clean up tracker agent contexts if any were left open | |
| for cm in self._tracker_agent_cms.values(): | |
| try: | |
| cm.__exit__(None, None, None) | |
| except Exception: | |
| pass | |
| self._tracker_agent_cms.clear() | |
| # No tracker agent contexts are currently created in this method, | |
| # so there is nothing to clean up here. | |
| pass |
Comment on lines
+97
to
+98
| except Exception: | ||
| pass |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
Suggested change
| except Exception: | |
| pass | |
| except Exception as exc: | |
| # Best-effort enrichment: log and continue to return base stats if ledger is unavailable | |
| logger.warning("Failed to enrich agent stats with ledger data: %s", exc) |
| try: | ||
| cm.__exit__(None, None, None) | ||
| except Exception: | ||
| pass |
There was a problem hiding this comment.
'except' clause does nothing but pass and there is no explanatory comment.
Suggested change
| pass | |
| logger.warning("Failed to close tracker agent context", exc_info=True) |
Introduce a project "guides" knowledge base and per-agent override/configuration support. Adds guide tools (write_guide, read_guide, list_guides) and central ToolRegistry sets for designer, developer and reviewer so agents get role-appropriate tools. Persist guides on disk via ProjectManager and expose guide endpoints in the API; frontend client can fetch guides. Implement per-project and per-request agent_overrides (model, temperature, system_prompt_override) and apply them when creating agents. Pipe deps through pipelines, add richer streaming callbacks (thinking/steps/iteration/round events), enforce optional max_generation_cost, and auto-save designer StyleSpec and site-plan guides. Misc: move to Persona.extend for strict JSON prompts, initialize prompture tracker/cache on startup, add DB migration and repository changes for agent_overrides, remove obsolete gemini_patch, and update UI to surface retries/thinking state.
Introduce specialist agent support and a dynamic parallel pipeline: adds an AgentRegistry and descriptors, a specialists package (markup, style, style_scss, script, image, copywriter, seo, accessibility, animation), and a create_specialist_pipeline in the orchestrator that runs image generation, parallel specialist builders, post-processing, and optional review loops. Update personas to include specialist roles and PM logic for selecting monolithic vs specialist modes, increase developer agent max_tokens for larger outputs, expose registry and specialist personas from agents.__init__, add SCSS compiler and various API/frontend adjustments, and update README to document the new specialist mode and agent architecture. This enables faster, modular builds and centralizes agent discovery for UI and pipeline construction.
Allow projects to enable/disable individual agents and exclude disabled agents from generation. - agentsite/api/routes/generate.py: Merge project-level "enabled" into agent overrides so project settings can toggle agents. - agentsite/engine/pipeline.py: Filter out agents marked disabled in agent configs before running the pipeline; log removed agents and add a "developer" fallback if all specialists are disabled (unless developer is also disabled). - frontend/src/pages/ProjectSettingsPage.jsx: Update UI to list all catalog agents (not just enabled ones), add per-agent enable/disable toggle, expand/collapse configuration panel, reset button, and visual/UX updates to reflect disabled state. Cleans and persists enabled overrides along with model/temperature/prompt. This change enables projects to opt out of specific agents so the pipeline respects project-level preferences and provides a UI for managing those overrides.
jhd3197
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a modular specialist agent architecture, a project-level guides/knowledge base system with per-agent tool and configuration overrides, and deep Prompture usage tracking integration across the full stack. Together, these changes make generation faster and more modular, give users fine-grained control over each agent, and surface detailed cost and usage analytics in the UI.
57 files changed | +3,652 / -543 lines
Features
Specialist Agents & Parallel Pipeline
personas.pyagents.__init__Project Guides, Agent Tools & Overrides
write_guide,read_guide,list_guidestools let agents persist and retrieve project-level guides on diskPersona.extendfor strict JSON prompt compositionagent_overridesgemini_patchmodulePrompture Usage Tracking & Analytics
/api/agents/statsenriched withcost_by_modeland daily/monthly totals/stats/models,/stats/providers,/stats/today,/stats/session/{id}AgentRunmodel gainssession_id; DB migration adds the columnget_model_ratesfor accurate rate-based calculationFrontend & UI Improvements