Dev server: survive rapid-save rebuild races and explain missing plugin state

docs/changelog.md

@@ -22,6 +22,8 @@ Release notes for Pyxle. While we're in beta (`0.x`), minor versions may include
 - **`Accept: text/markdown` negotiation now follows RFC 9110.** The check used to be a substring match on the raw `Accept` header, which ignored q-values entirely — `text/html, text/markdown;q=0` served Markdown even though `q=0` means *not acceptable*, and `text/html;q=0.9, text/markdown;q=0.8` served Markdown even though HTML was preferred. The header is now properly parsed: media types match by exact type/subtype token (no substrings — `text/markdownish` no longer counts), q-values are honoured (`q=0` excludes; default `1.0`, clamped to `0..1`), wildcards like `*/*` never select Markdown, ties go to Markdown, and malformed headers never raise — they fall back to HTML. Since negotiated responses carry `Vary: Accept`, shared caches now key on the correct variant. New public helper: `pyxle.devserver.llms.markdown_is_acceptable(accept)`. See [AI accessibility → Negotiation rules](guides/llms.md#negotiation-rules).
 - **The generated `/llms.txt` no longer advertises Markdown that isn't there.** The default index used to emit relative links and to link every concrete page's `.md` URL — including pages whose `.md` URL just `307`-redirects to HTML. It now emits **absolute URLs** derived from the request's scheme and host (proxy headers respected), links `.md` only for pages whose Markdown actually resolves (a co-located `.md`, a `to_markdown` handler, an ancestor `llms.py` — or every page when `autoConvert` is on), and lists the rest at their canonical HTML URL. Custom `llms_txt` hooks are unaffected; `ctx.render_default()` returns the improved index. See [AI accessibility → The /llms.txt index](guides/llms.md#the-llmstxt-index).
 - **Converted Markdown keeps agents on the Markdown channel** (breaking behavior change). `autoConvert` output — and the public `html_to_markdown()` helper — now rewrites internal page links to their `.md` renditions: `[About](/about)` becomes `[About](/about.md)`, preserving query strings and fragments (`/about?x=1#y` → `/about.md?x=1#y`). External URLs, `mailto:`/`tel:` links, `/api/` routes, asset paths with a file extension, and links already ending in `.md` are untouched, and `wrap_markdown` hooks receive the rewritten Markdown. **Breaking:** `html_to_markdown()` rewrites by default — pass `rewrite_links=False` for the old verbatim behavior. See [AI accessibility → autoConvert](guides/llms.md#autoconvert-the-lossy-fallback).
+- **Fix: a burst of rapid file saves can no longer kill the dev server.** Two dev-server robustness fixes land together. First, build passes are now serialized and `meta.json` is written atomically and only when its content changed — previously a debounce-timer race could run two rebuilds at once, and one pass reading the build metadata mid-write misdiagnosed it as a schema mismatch and wiped the whole `.pyxle-build` cache, deleting and recreating `vite.config.js`. Vite answers a config change with a full restart, and a restart racing the next rebuild could die (`[vite] process exited`), leaving nothing listening until a manual restart. Second, the Vite subprocess is now properly supervised: **any** exit the dev server didn't initiate — including a "clean" exit code 0 — logs a clear warning and relaunches Vite with a short exponential backoff (0.5s, doubling) and a bounded budget (3 consecutive attempts). If relaunching keeps failing, a structured `ViteSupervisionError` is surfaced loudly in the terminal instead of a silently dead asset server.
+- **Fix: touching `request.state.<name>` without the providing plugin now explains itself.** A loader or `@action` that reads a `request.state` attribute nothing populated (most commonly `request.state.db` without the pyxle-db plugin) used to 500 with a bare `AttributeError: 'State' object has no attribute 'db'` and a raw traceback that pointed nowhere. The framework now detects exactly this case and reports it as a structured error with guidance: state attributes are provided by plugins or middleware — e.g. `request.state.db` requires the pyxle-db plugin in `pyxle.config.json` (`"plugins": ["pyxle-db"]`). In dev the message appears in the error overlay, the action's JSON error, and the server log (with the original traceback chained); in production the response stays generic, as always. Any other `AttributeError` from your code flows through unchanged.
 
 ## 0.6.1 — 2026-07-01
 

pyxle/devserver/build.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import json
+import os
 import shutil
 from dataclasses import dataclass
 from pathlib import Path
@@ -127,11 +128,33 @@ def _load_metadata(path: Path) -> BuildMetadata | None:
 
 
 def _write_metadata(path: Path, metadata: BuildMetadata) -> None:
-    payload = metadata.to_dict()
+    """Persist ``metadata`` to ``path`` — write-if-changed and atomic.
+
+    Identical content is never rewritten, so a no-op build pass leaves the
+    file's mtime untouched (the same guard every other stable generated
+    artifact uses). Changed content lands via a temp file + ``os.replace`` so
+    a concurrent reader can never observe a truncated ``meta.json`` — a torn
+    read here used to be misdiagnosed as a schema mismatch, wiping the entire
+    build cache.
+    """
+    payload = json.dumps(metadata.to_dict(), indent=2, sort_keys=True) + "\n"
+
+    if path.exists():
+        try:
+            current = path.read_text(encoding="utf-8")
+        except OSError:  # pragma: no cover - unreadable cache file; rewrite it
+            current = None
+        if current == payload:
+            return
+
     path.parent.mkdir(parents=True, exist_ok=True)
-    with path.open("w", encoding="utf-8") as file:
-        json.dump(payload, file, indent=2, sort_keys=True)
-        file.write("\n")
+    temp_path = path.with_name(path.name + ".tmp")
+    try:
+        temp_path.write_text(payload, encoding="utf-8")
+        os.replace(temp_path, path)
+    finally:
+        if temp_path.exists():  # pragma: no cover - only on a failed write
+            temp_path.unlink(missing_ok=True)
 
 
 def load_build_metadata(build_root: Path, *, schema_version: str = BUILD_CACHE_SCHEMA_VERSION) -> BuildMetadata:

pyxle/devserver/builder.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import shutil
+import threading
 from dataclasses import dataclass, field
 from pathlib import Path
 from typing import Dict
@@ -47,9 +48,33 @@ def any_changes(self) -> bool:
         )
 
 
+#: Serializes build passes within the process. The watcher debounces
+#: filesystem events, but a debounce timer that fires while a previous build
+#: is still running starts a second pass on another thread
+#: (``threading.Timer.cancel()`` cannot stop a timer that has already fired).
+#: Two interleaved passes read and rewrite the same ``meta.json`` and
+#: generated artifacts; a torn read used to make
+#: :func:`ensure_fresh_build_cache` mistake a mid-write ``meta.json`` for a
+#: schema mismatch and wipe the whole build cache — deleting and recreating
+#: ``vite.config.js``, which Vite answers with a full (and racy) dev-server
+#: restart. Running one pass at a time keeps every pass' view of the cache
+#: consistent.
+_BUILD_PASS_LOCK = threading.Lock()
+
+
 def build_once(settings: DevServerSettings, *, force_rebuild: bool = False) -> BuildSummary:
-    """Run a single build pass for the project located at ``settings``."""
+    """Run a single build pass for the project located at ``settings``.
+
+    Passes are serialized process-wide (see :data:`_BUILD_PASS_LOCK`): a call
+    that arrives while another pass is running blocks until that pass
+    finishes, then rebuilds against the fresh metadata it left behind.
+    """
+
+    with _BUILD_PASS_LOCK:
+        return _build_once_locked(settings, force_rebuild=force_rebuild)
+
 
+def _build_once_locked(settings: DevServerSettings, *, force_rebuild: bool) -> BuildSummary:
     paths, previous_metadata = ensure_fresh_build_cache(settings)
     sources = scan_source_tree(settings)
     summary = BuildSummary()

pyxle/devserver/starlette_app.py

@@ -1390,6 +1390,26 @@ async def _dispatch_action(
             payload["fields"] = exc.fields
         return JSONResponse(payload, status_code=exc.status_code)
     except Exception as exc:
+        from pyxle.ssr.view import (  # noqa: PLC0415
+            MissingRequestStateError,
+            missing_state_attribute,
+        )
+
+        # A read of an unset ``request.state`` attribute means a plugin or
+        # middleware isn't configured — wrap the bare AttributeError with
+        # guidance (chained, so the original traceback stays in the log).
+        # Every other exception flows through unchanged.
+        attribute = missing_state_attribute(exc)
+        if attribute is not None:
+            import logging as _logging  # noqa: PLC0415
+
+            state_error = MissingRequestStateError(attribute)
+            state_error.__cause__ = exc
+            _logging.getLogger(__name__).error(
+                "Action '%s' failed: %s", action_name, state_error, exc_info=state_error
+            )
+            error_msg = str(state_error) if debug else "Internal server error"
+            return JSONResponse({"ok": False, "error": error_msg}, status_code=500)
         error_msg = str(exc) if debug else "Internal server error"
         return JSONResponse({"ok": False, "error": error_msg}, status_code=500)
 

pyxle/devserver/vite.py

@@ -17,9 +17,51 @@
 
 _ViteProbe = Callable[[str, int], Awaitable[bool]]
 
+#: Maximum consecutive relaunch attempts after an unexpected Vite exit.
+#: Three attempts (with the exponential backoff below) are enough to ride out
+#: transient causes — e.g. a rebuild burst rewriting generated files while
+#: Vite is mid config-restart — while a persistently broken setup (bad
+#: ``vite.config.js``, missing dependency) still fails fast and loudly instead
+#: of crash-looping forever.
+DEFAULT_RESTART_ATTEMPTS = 3
+
+#: Base delay in seconds before relaunching Vite after an unexpected exit.
+#: Doubles on every consecutive failure (0.5s → 1s → 2s) so the condition that
+#: killed Vite (an in-flight rebuild, a port not yet released) has time to
+#: clear, while the first relaunch is quick enough that HMR is back before the
+#: developer notices.
+DEFAULT_RESTART_DELAY = 0.5
+
+
+class ViteSupervisionError(RuntimeError):
+    """Raised when Vite exits unexpectedly and cannot be relaunched.
+
+    Produced after the supervisor exhausts its relaunch budget
+    (:data:`DEFAULT_RESTART_ATTEMPTS` by default). The dev server is still
+    running at that point but can no longer serve client assets, so the error
+    is surfaced prominently instead of leaving a silently dead asset server.
+    """
+
+    def __init__(self, attempts: int) -> None:
+        super().__init__(
+            "Vite dev server exited unexpectedly and could not be relaunched "
+            f"after {attempts} attempt(s). Client assets can no longer be "
+            "served — restart `pyxle dev`. Check the [vite] log output above "
+            "for the underlying failure."
+        )
+        self.attempts = attempts
+
 
 class ViteProcess:
-    """Launch and supervise the Vite dev server."""
+    """Launch and supervise the Vite dev server.
+
+    Supervision: any exit the supervisor did not initiate via :meth:`stop` —
+    including a "clean" exit code 0, which Vite produces when e.g. its config
+    file disappears mid config-reload — is treated as unexpected and answered
+    with a bounded, exponentially backed-off relaunch. Exhausting the relaunch
+    budget records a :class:`ViteSupervisionError` (see :attr:`fatal_error`)
+    and logs it as a fatal error.
+    """
 
     def __init__(
         self,
@@ -32,7 +74,8 @@ def __init__(
         readiness_timeout: float = 10.0,
         readiness_interval: float = 0.1,
         probe: _ViteProbe | None = None,
-        restart_delay: float = 0.5,
+        restart_delay: float = DEFAULT_RESTART_DELAY,
+        max_restart_attempts: int = DEFAULT_RESTART_ATTEMPTS,
     ) -> None:
         self._settings = settings
         self._logger = logger or ConsoleLogger()
@@ -48,6 +91,9 @@ def __init__(
         self._stopping: bool = False
         self._restart_task: asyncio.Task[None] | None = None
         self._restart_delay = restart_delay
+        self._max_restart_attempts = max_restart_attempts
+        self._restart_attempts = 0
+        self._fatal_error: ViteSupervisionError | None = None
         self._command_override: list[str] | None = None
         self._npm_install_attempted = False
 
@@ -56,6 +102,11 @@ def running(self) -> bool:
         process = self._process
         return process is not None and process.returncode is None
 
+    @property
+    def fatal_error(self) -> ViteSupervisionError | None:
+        """The supervision failure that ended relaunch attempts, if any."""
+        return self._fatal_error
+
     async def start(self) -> None:
         if self.running:
             return
@@ -68,6 +119,13 @@ async def start(self) -> None:
             with suppress(asyncio.CancelledError):
                 await restart_task
             self._restart_task = None
+        if current_task is not restart_task or restart_task is None:
+            # A manual (re)start expresses fresh intent: clear any previous
+            # supervision failure so the relaunch budget starts over. The
+            # supervisor's own relaunch loop must NOT reset the counter here,
+            # or the retry budget could never be exhausted.
+            self._fatal_error = None
+            self._restart_attempts = 0
 
         command = self._build_launch_command()
         self._logger.info("Launching Vite dev server: " + " ".join(command))
@@ -319,16 +377,25 @@ async def _monitor_process(self, process: asyncio.subprocess.Process) -> None:
                     await task
 
         returncode = await process.wait()
-        crashed = returncode not in (0, None) and not self._stopping
 
         if returncode not in (0, None):
             self._logger.error(f"[vite] process exited with code {returncode}")
         else:
             self._logger.info("[vite] process exited")
 
-        if crashed:
-            self._logger.warning("Vite process exited unexpectedly; attempting restart")
-            self._process = None
+        if self._stopping:
+            return
+
+        # Any exit the supervisor did not initiate is unexpected — including a
+        # "clean" exit code 0 (Vite exits 0 when, e.g., its config file
+        # vanishes during a config-reload). Without a relaunch the dev server
+        # would keep running with no asset server behind it, which looks like
+        # a dead page in the browser.
+        self._logger.warning("Vite process exited unexpectedly; attempting restart")
+        self._process = None
+        if self._fatal_error is None and (
+            self._restart_task is None or self._restart_task.done()
+        ):
             self._restart_task = asyncio.create_task(self._restart_after_exit())
 
     async def _pipe_stream(self, stream: asyncio.StreamReader, *, is_error: bool) -> None:
@@ -345,18 +412,73 @@ async def _pipe_stream(self, stream: asyncio.StreamReader, *, is_error: bool) ->
                 self._logger.info(f"[vite] {message}")
 
     async def _restart_after_exit(self) -> None:
+        """Relaunch Vite after an unexpected exit, with backoff and a budget.
+
+        Each consecutive failure doubles the delay; a relaunch that reaches
+        readiness resets the budget (the supervisor is healthy again).
+        Exhausting :attr:`_max_restart_attempts` records a
+        :class:`ViteSupervisionError` and logs it as fatal.
+        """
         try:
-            await asyncio.sleep(self._restart_delay)
-            if self._stopping:
+            while not self._stopping:
+                self._restart_attempts += 1
+                if self._restart_attempts > self._max_restart_attempts:
+                    self._fatal_error = ViteSupervisionError(self._max_restart_attempts)
+                    self._logger.error(str(self._fatal_error))
+                    # Never leave a hung child running unsupervised past the
+                    # budget — it would hold the port and confuse the next
+                    # manual start().
+                    await self._terminate_unready_child()
+                    return
+                delay = self._restart_delay * (2 ** (self._restart_attempts - 1))
+                self._logger.warning(
+                    f"Relaunching Vite dev server in {delay:.1f}s "
+                    f"(attempt {self._restart_attempts}/{self._max_restart_attempts})"
+                )
+                await asyncio.sleep(delay)
+                if self._stopping:
+                    return
+                try:
+                    await self.start()
+                    await self.wait_until_ready()
+                except Exception as exc:
+                    self._logger.warning(f"Vite relaunch attempt failed: {exc}")
+                    # A child that launched but never became ready (hung Vite)
+                    # must not survive the attempt: ``start()`` would no-op on
+                    # it next iteration and the budget would burn re-probing
+                    # the same dead-end process.
+                    await self._terminate_unready_child()
+                    continue
+                # The relaunched process accepts connections again; treat the
+                # supervisor as healthy and forget the failure streak.
+                self._restart_attempts = 0
                 return
-            await self.start()
-            await self.wait_until_ready()
-        except asyncio.CancelledError:  # pragma: no cover - cancellation path
-            raise
-        except Exception as exc:  # pragma: no cover - defensive
-            self._logger.error(f"Failed to restart Vite dev server: {exc}")
         finally:
-            self._restart_task = None
+            if self._restart_task is asyncio.current_task():
+                self._restart_task = None
+
+    async def _terminate_unready_child(self) -> None:
+        """Terminate a live child without entering shutdown state.
+
+        Used only by the supervisor when a relaunch left Vite running but not
+        accepting connections. ``_stopping`` stays False (this is not a
+        shutdown) — the monitor task skips rescheduling because the restart
+        task is still the current task.
+        """
+        process = self._process
+        if process is None or process.returncode is not None:
+            return
+        self._logger.warning(
+            "Vite is running but not accepting connections; terminating the hung process"
+        )
+        process.terminate()
+        try:
+            await asyncio.wait_for(process.wait(), timeout=self._stop_timeout)
+        except asyncio.TimeoutError:
+            self._logger.warning("Hung Vite process did not exit after SIGTERM; killing")
+            process.kill()
+            await process.wait()
+        self._process = None
 
     @staticmethod
     async def _default_probe(host: str, port: int) -> bool:
@@ -371,4 +493,9 @@ async def _default_probe(host: str, port: int) -> bool:
             return True
 
 
-__all__ = ["ViteProcess"]
+__all__ = [
+    "DEFAULT_RESTART_ATTEMPTS",
+    "DEFAULT_RESTART_DELAY",
+    "ViteProcess",
+    "ViteSupervisionError",
+]