Conversation
Introduces plain-text fallback variants for PM, Designer, and Developer agents to support models without structured output or tool calling. The pipeline now retries each agent phase with the appropriate fallback if structured output fails, improving robustness for a wider range of models. WebSocket events now include the resolved model name for each agent, and the frontend displays the short model name in the progress UI. Removes obsolete prompture fix documentation.
There was a problem hiding this comment.
Pull request overview
This PR enhances the AgentSite pipeline to support a wider range of LLM models by introducing plain-text fallback agents for PM, Designer, and Developer roles. When models don't support structured output or tool calling, the pipeline automatically retries with plain-text variants that use explicit JSON instructions or markdown code blocks instead. The frontend now displays the resolved model name for each agent in the progress UI.
Changes:
- Added plain-text fallback agent variants (
create_pm_agent_plain,create_designer_agent_plain,create_developer_agent_plain) that work without structured output or tool-calling features - Implemented automatic retry logic in the pipeline that falls back to plain-text agents when structured output fails
- Enhanced WebSocket events to include resolved model names for each agent, and updated frontend to display short model names in the progress UI
- Removed obsolete Prompture fix documentation files
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| frontend/src/components/builder/ProgressPipeline.jsx | Adds shortModelName utility and displays model name in agent progress badges |
| agentsite/engine/pipeline.py | Implements retry logic with plain-text fallbacks for PM, Designer, and Developer agents; adds model name tracking and emission in WebSocket events |
| agentsite/agents/pm.py | Adds create_pm_agent_plain variant that uses explicit JSON schema instructions instead of structured output |
| agentsite/agents/designer.py | Adds create_designer_agent_plain variant that uses explicit JSON schema instructions instead of structured output |
| agentsite/agents/developer.py | Adds create_developer_agent_plain variant that outputs markdown fenced code blocks instead of using file-writing tools |
| PROMPTURE_FIX_PLAN.md | Removes obsolete documentation about Prompture state collision fix |
| PROMPTURE_FIX.md | Removes obsolete documentation about Prompture state collision fix |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| from prompture import clean_json_text | ||
|
|
There was a problem hiding this comment.
This import of clean_json_text appears to be unused in the designer agent phase. The function is only used in the PM phase (line 303) for parsing the site plan. Consider removing this import to avoid confusion.
| from prompture import clean_json_text |
| if _dev_failed and not written_files: | ||
| dev_errors = [ | ||
| e for e in getattr(result, "errors", []) | ||
| if getattr(e, "agent_name", "") in ("developer", "agentsite_developer") | ||
| ] | ||
| logger.warning( | ||
| "Developer pipeline produced no output (success=%s, errors=%d, " | ||
| "written_files=%d, dev_output_len=%d, tool_calls=%d). " | ||
| "Retrying with plain text developer.", | ||
| result.success, | ||
| len(dev_errors), | ||
| len(written_files), | ||
| len(self._developer_output_text), | ||
| len(self._developer_tool_calls), | ||
| ) | ||
| from ..agents.developer import create_developer_agent_plain | ||
|
|
||
| dev_model = self._agent_models["developer"] | ||
| dev_agent_plain = create_developer_agent_plain(dev_model) | ||
|
|
||
| dev_prompt = ( | ||
| "Build the website page based on this plan:\n\n" | ||
| f"Site Plan: {initial_state['site_plan']}\n\n" | ||
| f"Style Spec: {initial_state.get('style_spec', '')}\n\n" | ||
| f"Logo URL: {initial_state.get('logo_url', '')}\n" | ||
| f"Icon URL: {initial_state.get('icon_url', '')}\n\n" | ||
| "Generate complete, self-contained HTML with inline or linked CSS/JS." | ||
| ) | ||
|
|
||
| dev_callbacks = GroupCallbacks( | ||
| on_agent_start=_on_agent_start, | ||
| on_agent_complete=_on_agent_complete, | ||
| on_agent_error=_on_agent_error, | ||
| ) | ||
| dev_pipeline_plain = SequentialGroup( | ||
| [(dev_agent_plain, "{dev_prompt}")], | ||
| callbacks=dev_callbacks, | ||
| state={"dev_prompt": dev_prompt}, | ||
| ) | ||
| _patch_pipeline_deps(dev_pipeline_plain, deps) | ||
| plain_result = dev_pipeline_plain.run(dev_prompt) | ||
|
|
||
| # Use the plain result's usage and state | ||
| if hasattr(plain_result, 'aggregate_usage'): | ||
| if not hasattr(result, 'aggregate_usage'): | ||
| result.aggregate_usage = {} | ||
| for k, v in plain_result.aggregate_usage.items(): | ||
| if isinstance(v, (int, float)): | ||
| result.aggregate_usage[k] = result.aggregate_usage.get(k, 0) + v | ||
| result = plain_result |
There was a problem hiding this comment.
The developer retry logic updates the result but doesn't reset tracking variables like written_files, self._developer_output_text, and self._developer_tool_calls. This could lead to incorrect fallback detection in the subsequent file extraction logic (lines 504-542), which checks written_files but will find files from the original failed attempt if they were written.
Additionally, methods like _write_files_from_output, _extract_fenced_blocks, and _try_extract_raw_html write files using _pm.write_version_file which doesn't call the _on_file_written callback, so these files won't be added to the written_files list or trigger WebSocket events. This creates an inconsistency where files written via tools are tracked but files written via fallback extraction are not.
Consider either:
- Resetting these tracking variables before the retry, or
- Ensuring all file writing paths (including fallback extraction) properly call the
_on_file_writtencallback and update thewritten_fileslist.
| system_prompt=( | ||
| DESIGNER_PERSONA.system_prompt | ||
| + "\n\nIMPORTANT: You MUST respond with ONLY a valid JSON object matching this schema:\n" | ||
| + f"```json\n{__import__('json').dumps(json_schema, indent=2)}\n```\n" |
There was a problem hiding this comment.
The inline use of __import__('json').dumps() is unconventional and harder to read than a regular import. Consider adding import json at the top of the file and using json.dumps(json_schema, indent=2) instead.
| def create_pm_agent_plain(model: str) -> Agent: | ||
| """Create a PM agent WITHOUT output_type for models that don't support structured output. | ||
|
|
||
| Uses explicit JSON instructions in the system prompt instead of schema enforcement. | ||
| The caller is responsible for parsing the JSON output manually. | ||
| """ | ||
| json_schema = SitePlan.model_json_schema() | ||
| return Agent( | ||
| model, | ||
| system_prompt=( | ||
| PM_PERSONA.system_prompt | ||
| + "\n\nIMPORTANT: You MUST respond with ONLY a valid JSON object matching this schema:\n" | ||
| + f"```json\n{__import__('json').dumps(json_schema, indent=2)}\n```\n" | ||
| "Do NOT include any text before or after the JSON. Return ONLY the JSON object." | ||
| ), | ||
| name="pm", | ||
| description="Plans website structure and pages (plain text mode)", | ||
| output_key="site_plan", | ||
| ) |
There was a problem hiding this comment.
The new plain-text agent variants (create_pm_agent_plain, create_designer_agent_plain, create_developer_agent_plain) lack test coverage. Given that the existing agent factory functions in this file have corresponding tests in tests/test_agents.py, these new functions should also have tests to ensure they are properly initialized and have the expected properties (e.g., correct name, output_key, and absence of output_type for plain variants).
| def create_designer_agent_plain(model: str) -> Agent: | ||
| """Create a Designer agent WITHOUT output_type for models that don't support structured output. | ||
|
|
||
| Uses explicit JSON instructions in the system prompt instead of schema enforcement. | ||
| The caller is responsible for parsing the JSON output manually. | ||
| """ | ||
| json_schema = StyleSpec.model_json_schema() | ||
| return Agent( | ||
| model, | ||
| system_prompt=( | ||
| DESIGNER_PERSONA.system_prompt | ||
| + "\n\nIMPORTANT: You MUST respond with ONLY a valid JSON object matching this schema:\n" | ||
| + f"```json\n{__import__('json').dumps(json_schema, indent=2)}\n```\n" | ||
| "Do NOT include any text before or after the JSON. Return ONLY the JSON object." | ||
| ), | ||
| name="designer", | ||
| description="Defines visual design system (plain text mode)", | ||
| output_key="style_spec", | ||
| ) |
There was a problem hiding this comment.
The new plain-text agent variant create_designer_agent_plain lacks test coverage. Given that the existing agent factory functions have corresponding tests in tests/test_agents.py, this new function should also have tests to ensure it is properly initialized and has the expected properties (e.g., correct name, output_key, and absence of output_type).
| def create_developer_agent_plain(model: str) -> Agent: | ||
| """Create a Developer agent WITHOUT tools for models that don't support tool calling. | ||
|
|
||
| Instead of using write_file/read_file tools, this agent outputs file contents | ||
| directly in its response using markdown fenced code blocks. The pipeline's | ||
| existing fallback extraction logic (_extract_fenced_blocks, _try_extract_raw_html) | ||
| handles parsing the output into files on disk. | ||
| """ | ||
| return Agent( | ||
| model, | ||
| system_prompt=( | ||
| "You are an expert frontend developer. You build complete, production-ready " | ||
| "web pages using semantic HTML5, modern CSS, and vanilla JavaScript.\n\n" | ||
| "WORKFLOW — you MUST follow this exact output format:\n" | ||
| "Output each file using markdown fenced code blocks with the language tag.\n" | ||
| "You MUST generate at least an index.html file.\n\n" | ||
| "Example output format:\n" | ||
| "```html\n" | ||
| "<!DOCTYPE html>\n" | ||
| "<html>...</html>\n" | ||
| "```\n\n" | ||
| "```css\n" | ||
| "/* styles.css */\n" | ||
| "body { ... }\n" | ||
| "```\n\n" | ||
| "```javascript\n" | ||
| "// script.js\n" | ||
| "document.addEventListener('DOMContentLoaded', ...);\n" | ||
| "```\n\n" | ||
| "Requirements:\n" | ||
| "- Write clean, semantic HTML with proper heading hierarchy\n" | ||
| "- Use CSS custom properties for theming (colors, fonts, spacing)\n" | ||
| "- Make pages fully responsive (mobile-first approach)\n" | ||
| "- Include smooth transitions and subtle animations\n" | ||
| "- Add proper meta tags, viewport settings, and favicon links\n" | ||
| "- Use Google Fonts via CDN link\n" | ||
| "- Write accessible markup (ARIA labels, alt text, focus styles)\n\n" | ||
| "Generate complete, self-contained files. Every HTML page should be fully functional " | ||
| "when opened directly in a browser.\n\n" | ||
| "IMPORTANT: Output ONLY the fenced code blocks with complete file contents. " | ||
| "Do not include any other text or explanation." | ||
| ), | ||
| name="developer", | ||
| description="Generates HTML/CSS/JS files for each page (plain text mode)", | ||
| output_key="page_output", | ||
| ) |
There was a problem hiding this comment.
The new plain-text agent variant create_developer_agent_plain lacks test coverage. Given that the existing agent factory functions have corresponding tests in tests/test_agents.py, this new function should also have tests to ensure it is properly initialized and has the expected properties (e.g., correct name, output_key, absence of output_type, and absence of tools).
| designer_result = designer_pipeline_plain.run(designer_prompt) | ||
| style_spec_text = designer_result.shared_state.get("style_spec", "") |
There was a problem hiding this comment.
The fallback retry logic for the Designer agent does not handle the case where the plain-text variant also fails. If designer_pipeline_plain.run() throws an exception, it will propagate up and terminate the entire pipeline. Consider wrapping this fallback call in a try-except block to provide more specific error handling and logging.
| plain_result = dev_pipeline_plain.run(dev_prompt) | ||
|
|
||
| # Use the plain result's usage and state | ||
| if hasattr(plain_result, 'aggregate_usage'): | ||
| if not hasattr(result, 'aggregate_usage'): | ||
| result.aggregate_usage = {} | ||
| for k, v in plain_result.aggregate_usage.items(): | ||
| if isinstance(v, (int, float)): | ||
| result.aggregate_usage[k] = result.aggregate_usage.get(k, 0) + v | ||
| result = plain_result | ||
|
|
There was a problem hiding this comment.
The fallback retry logic for the Developer agent does not handle the case where the plain-text variant also fails. If dev_pipeline_plain.run() throws an exception, it will propagate up and terminate the entire pipeline. Consider wrapping this fallback call in a try-except block to provide more specific error handling and logging.
| plain_result = dev_pipeline_plain.run(dev_prompt) | |
| # Use the plain result's usage and state | |
| if hasattr(plain_result, 'aggregate_usage'): | |
| if not hasattr(result, 'aggregate_usage'): | |
| result.aggregate_usage = {} | |
| for k, v in plain_result.aggregate_usage.items(): | |
| if isinstance(v, (int, float)): | |
| result.aggregate_usage[k] = result.aggregate_usage.get(k, 0) + v | |
| result = plain_result | |
| try: | |
| plain_result = dev_pipeline_plain.run(dev_prompt) | |
| except Exception: | |
| logger.exception( | |
| "Plain text developer pipeline retry failed; " | |
| "preserving original developer result." | |
| ) | |
| else: | |
| # Use the plain result's usage and state | |
| if hasattr(plain_result, 'aggregate_usage'): | |
| if not hasattr(result, 'aggregate_usage'): | |
| result.aggregate_usage = {} | |
| for k, v in plain_result.aggregate_usage.items(): | |
| if isinstance(v, (int, float)): | |
| result.aggregate_usage[k] = result.aggregate_usage.get(k, 0) + v | |
| result = plain_result |
| system_prompt=( | ||
| PM_PERSONA.system_prompt | ||
| + "\n\nIMPORTANT: You MUST respond with ONLY a valid JSON object matching this schema:\n" | ||
| + f"```json\n{__import__('json').dumps(json_schema, indent=2)}\n```\n" |
There was a problem hiding this comment.
The inline use of __import__('json').dumps() is unconventional and harder to read than a regular import. Since json is already imported at the top of the file (line 5 in agentsite/engine/pipeline.py), this pattern is unnecessary. Consider using the standard import instead: json.dumps(json_schema, indent=2).
| pm_result = pm_pipeline_plain.run(page_prompt) | ||
| site_plan_text = pm_result.shared_state.get("site_plan", "") |
There was a problem hiding this comment.
The fallback retry logic for PM, Designer, and Developer agents does not handle the case where the plain-text variants also fail. If pm_pipeline_plain.run() (line 296), designer_pipeline_plain.run() (line 370), or dev_pipeline_plain.run() (line 463) throw an exception, it will propagate up and terminate the entire pipeline, potentially without proper cleanup or error reporting.
Consider wrapping these fallback calls in try-except blocks to provide more specific error handling and logging, or at minimum document that these failures are intentionally unhandled and will terminate the pipeline.
| pm_result = pm_pipeline_plain.run(page_prompt) | |
| site_plan_text = pm_result.shared_state.get("site_plan", "") | |
| try: | |
| pm_result = pm_pipeline_plain.run(page_prompt) | |
| site_plan_text = pm_result.shared_state.get("site_plan", "") | |
| except Exception as pm_plain_exc: | |
| logger.error( | |
| "PM agent plain-text fallback failed; proceeding with default agent set: %s", | |
| pm_plain_exc, | |
| exc_info=True, | |
| ) | |
| site_plan_text = "" |
Introduces plain-text fallback variants for PM, Designer, and Developer agents to support models without structured output or tool calling. The pipeline now retries each agent phase with the appropriate fallback if structured output fails, improving robustness for a wider range of models. WebSocket events now include the resolved model name for each agent, and the frontend displays the short model name in the progress UI. Removes obsolete prompture fix documentation.