feat: enhance server export support and update documentation

Author: raajnadar

.github/workflows/ci.yml

@@ -110,3 +110,13 @@ jobs:
       # gate that sees a hydration mismatch.
       - name: Static export check
         run: pnpm --filter example check:export
+
+      # The same app with `web.output: 'server'`. The pages move to
+      # `dist-server/server` and the files to `dist-server/client`, so this
+      # proves that the seo tools read that layout. Then `expo serve` must
+      # send 200 for a page and 404 for an unknown path: the server sends
+      # the status itself, where a static host sends what its rewrite rule
+      # says. The pages are the same HTML as above, so the browser pass is
+      # not repeated.
+      - name: Server export check
+        run: pnpm --filter example check:export:server

docs/docs/seo/api.md

@@ -70,13 +70,13 @@ Each returns a `JsonLd` with `@context: 'https://schema.org'` and `@type` set. E
 ## `@rootnative/seo/node` — Node only, no peer
 
 - `rootnative-seo <outDir> [--site-url <url>] [--sitemap <url>]... [--no-robots]` — the command for the deploy step, after the export. It runs `writeSeoFiles`, prints each page it leaves out with the reason, and exits with 1 on an error. `--sitemap` replaces the sitemap list in `robots.txt`; repeat it for more than one.
-- `writeSeoFiles({ outDir, siteUrl?, robots? })` — reads every `.html` file under `outDir` and writes `sitemap.xml` and `robots.txt`. A page goes in the sitemap with its canonical link, when the link is under the site and the page has no `noindex`. Left out, with a `reason`: a special route (a `+` or `_` segment, or a root `404.html`), a dynamic route template (`[id].html`), `noindex`, no canonical link, a canonical link outside the site, and a second page with the same link. A page with `hreflang` links gets the same links as `xhtml:link` entries. `siteUrl` defaults to the canonical link of `index.html`. `robots: false` writes no `robots.txt`; an object passes options to `writeRobots`. Resolves to `{ siteUrl, sitemap, robots?, urls, skipped: { file, reason }[] }`. Each sitemap entry holds `loc` only: Google ignores `changefreq` and `priority`, and an export does not know `lastmod`. Throws when no page can go in the sitemap, because an empty sitemap fails with no other signal.
+- `writeSeoFiles({ outDir, siteUrl?, robots? })` — reads every `.html` file under `outDir` and writes `sitemap.xml` and `robots.txt` there. A server export, which has `server/_expo/routes.json` and `client/`, is read from `<outDir>/server`, and the two files go to `<outDir>/client`, the folder the server sends as files; every `/node` function below reads the same layout. A page goes in the sitemap with its canonical link, when the link is under the site and the page has no `noindex`. Left out, with a `reason`: a special route (a `+` or `_` segment, or a root `404.html`), a dynamic route template (`[id].html`), `noindex`, no canonical link, a canonical link outside the site, and a second page with the same link. A page with `hreflang` links gets the same links as `xhtml:link` entries. `siteUrl` defaults to the canonical link of `index.html`. `robots: false` writes no `robots.txt`; an object passes options to `writeRobots`. Resolves to `{ siteUrl, sitemap, robots?, urls, skipped: { file, reason }[] }`. Each sitemap entry holds `loc` only: Google ignores `changefreq` and `priority`, and an export does not know `lastmod`. Throws when no page can go in the sitemap, because an empty sitemap fails with no other signal.
 - `readExport(outDir)` — `{ file, route, canonical?, noindex, alternates: { lang, url }[] }[]` for every exported HTML file, sorted by path. Use it to build a URL list of your own.
 - `rootnative-seo audit <outDir> [--site-url <url>] [--ignore <rule>]...` — the gate for CI, after the export and after `rootnative-seo <outDir>`. It runs `auditExport`, prints each problem with its file and rule, and exits with 1 when there is one. It writes nothing. `--ignore` skips one rule; repeat it for more.
 - `auditExport({ outDir, siteUrl?, ignore? })` — reads every `.html` file under `outDir` and returns `{ pages, problems: { file, rule, message }[] }`. The rules: `title` (one `<title>` with text), `canonical` (an absolute canonical link, or `noindex`), `h1` (exactly one `<h1>`), `main` (exactly one `<main>`), `lang` (a `lang` on `<html>`), `alt` (every `<img>` has an `alt` attribute), `hreflang` (on each indexable page with `hreflang` links: every code is a language code or `x-default`, not `-UK`; every link is absolute; one link per code; a link to the page itself; every linked page in the export has a file, no `noindex`, and the same set; and the `xhtml:link` entries of the sitemap, when it has them, are the links of the page), and `sitemap` (`sitemap.xml` exists, every `<loc>` is the canonical link of a page, and every page `writeSeoFiles` would list is in it). `siteUrl` defaults to the canonical link of `index.html`; with neither, the second half of the `sitemap` rule is skipped. An empty `problems` list is a pass. It throws only when the folder is missing or holds no HTML file.
 - `rootnative-seo browse <outDir> [--chrome <path>] [--base-path <path>] [--settle <ms>]` — the second gate for CI, after the export. It runs `browseExport`, prints each problem with its file and kind, and exits with 1 when there is one. It writes nothing. Chrome comes from `--chrome`, then `CHROME_PATH`, then the usual names on `PATH` and the usual install paths.
 - `browseExport({ outDir, basePath?, chrome?, settleMs?, timeoutMs? })` — serves `outDir` and loads every `.html` file in headless Chrome, driven over its debugging pipe with no package. Returns `{ pages, problems: { file, route, kind, message }[], chrome }`, where `kind` is `exception` (uncaught), `console` (`console.error` or `console.assert`), or `log` (a browser log entry, such as a request that failed). A hydration mismatch is a `console.error` in development and a reported error in production, so both reach the list. `settleMs` (default 750) is how long it listens after the load event, so hydration runs. Throws when no Chrome is found, the folder is missing, or it holds no HTML file.
-- `serveExport({ outDir, basePath?, port? })` — serves `outDir` on `127.0.0.1` as a static host serves an Expo Router export: `/about` is `about.html` or `about/index.html`, a route with no file gets `+not-found.html` or `404.html` with status 404, and a request outside `outDir` is refused. `basePath` defaults to the path of the canonical link of `index.html`. Resolves to `{ origin, basePath, urlOf(route), close() }`. Use it to run Lighthouse or a browser test on the export.
+- `serveExport({ outDir, basePath?, port? })` — serves `outDir` on `127.0.0.1` as a static host serves an Expo Router export (a server export: a file from `client/`, then a page from `server/`, never the route manifest; a dynamic route gets its own file only, where the Expo server sends the template for every slug): `/about` is `about.html` or `about/index.html`, a route with no file gets `+not-found.html` or `404.html` with status 404, and a request outside `outDir` is refused. `basePath` defaults to the path of the canonical link of `index.html`. Resolves to `{ origin, basePath, urlOf(route), close() }`. Use it to run Lighthouse or a browser test on the export.
 - `findChrome(path?)` — the Chrome or Chromium binary `browseExport` would use, or `undefined`.
 
 The three functions below write one file each. Each creates `outDir`, writes the file and resolves to its path. Each has a pure sibling that returns the text.

docs/docs/seo/checklist.md

@@ -73,6 +73,29 @@ height, a Stack screen that fills its parent fills nothing, and the page
 shows the top bar over an empty area while the DOM holds every node. Keep
 the reset in the shell, as the example in step 7 does.
 
+### Server output
+
+`web.output: 'server'` renders the same HTML at export time and adds a
+server for API routes. Every step in this checklist is the same. Three
+things change:
+
+- The export writes the pages to `dist/server` and the files the server
+  sends as they are to `dist/client`. Pass `dist` to `rootnative-seo`, as
+  for a static export: it reads the pages from `server/` and writes
+  `sitemap.xml` and `robots.txt` into `client/`. `audit` and `browse` read
+  the same layout.
+- The server sends the status itself. An unknown path gets
+  `+not-found.html` with a 404, on any host. A dynamic route is the other
+  way: the server sends the route template with a 200 for every slug, also
+  for a slug that has no page. So the `noindex` answer for an unknown entry,
+  as above, is the only signal a crawler gets there.
+- The server matches each route from the root of its origin and ignores
+  `experiments.baseUrl`, which moves only the URLs of the files. Host a
+  server output at the root of its origin, or remove the base path before
+  the request reaches the server.
+
+Check the status codes with `npx expo serve` before you deploy.
+
 ## 2. Links as `Link`
 
 Use `Link` from `expo-router` for every navigation. It renders an `<a href>`

docs/docs/seo/limits.md

@@ -46,6 +46,10 @@ No package changes that. Check the host:
   `index.html` with a 200 for an unknown path.
 - A dynamic route with real data needs `generateStaticParams` and a data
   source the export server can reach.
+- With `web.output: 'server'`, the server sends the 404 for an unknown path,
+  on any host. It sends a dynamic route template with a 200 for every slug,
+  so an unknown entry needs `noindex`. See
+  [server output](./checklist.md#server-output).
 
 ## The charset position
 

docs/static/llms-full.txt

@@ -2686,7 +2686,7 @@ pnpm add @rootnative/seo
 
 ## The checklist, in order
 
-1. **Static output.** `web.output: 'static'` in app.json, and `extra.router.sitemap: false` so `_sitemap.html` is not exported. A dynamic route needs `generateStaticParams`.
+1. **Static output.** `web.output: 'static'` in app.json, and `extra.router.sitemap: false` so `_sitemap.html` is not exported. A dynamic route needs `generateStaticParams`. `web.output: 'server'` writes the same HTML to `dist/server` and the files to `dist/client`; every `/node` function and command takes `dist` and reads that layout. The server sends a real 404 for an unknown path, sends a dynamic route template with a 200 for every slug (so an unknown entry needs `noindex`), and ignores `experiments.baseUrl`: host it at the origin root.
 2. **Links as `Link`.** `Link` from expo-router renders `<a href>`. A `Pressable` with `router.push` renders a `<div>`, and a `Button` with `Linking.openURL` renders a `<button>`; a crawler follows neither. Put a component inside `<Link href asChild>`. On the web `Link` opens an external URL in the same tab; for a new tab pass `hrefAttrs={{ target: '_blank', rel: 'noopener' }}` to the component, because under `asChild` `Link` gives `target` to the child as a plain prop, which react-native-web drops.
 3. **The head on every page.** `defineSite` once, `SeoProvider` at the root, `PageHead` on each screen, and `<RouteHead defaults={fn} />` after the navigator for the screen that has none: `defaults` maps the pathname to a `PageMeta`, and a `PageHead` on the route wins the whole page. A page with no `url` gets no canonical link and no share card, which is correct for a `noindex` overlay. A page in more than one language passes the same `alternates` list (`{ lang, url }[]`, every version, itself included) to the `PageHead` of each version. Every version keeps the `lang` of `app/+html.tsx`.
 4. **Robots and sitemap.** Run `rootnative-seo dist` in the deploy step, after the export. It lists every exported page that has a canonical link under the site and no `noindex`, writes `sitemap.xml` and `robots.txt`, and prints each page it leaves out with the reason. `robots.txt` is read at the origin root only.
@@ -2757,13 +2757,13 @@ Each returns a `JsonLd` with `@context: 'https://schema.org'` and `@type` set. E
 ## `@rootnative/seo/node` — Node only, no peer
 
 - `rootnative-seo <outDir> [--site-url <url>] [--sitemap <url>]... [--no-robots]` — the command for the deploy step, after the export. It runs `writeSeoFiles`, prints each page it leaves out with the reason, and exits with 1 on an error. `--sitemap` replaces the sitemap list in `robots.txt`; repeat it for more than one.
-- `writeSeoFiles({ outDir, siteUrl?, robots? })` — reads every `.html` file under `outDir` and writes `sitemap.xml` and `robots.txt`. A page goes in the sitemap with its canonical link, when the link is under the site and the page has no `noindex`. Left out, with a `reason`: a special route (a `+` or `_` segment, or a root `404.html`), a dynamic route template (`[id].html`), `noindex`, no canonical link, a canonical link outside the site, and a second page with the same link. A page with `hreflang` links gets the same links as `xhtml:link` entries. `siteUrl` defaults to the canonical link of `index.html`. `robots: false` writes no `robots.txt`; an object passes options to `writeRobots`. Resolves to `{ siteUrl, sitemap, robots?, urls, skipped: { file, reason }[] }`. Each sitemap entry holds `loc` only: Google ignores `changefreq` and `priority`, and an export does not know `lastmod`. Throws when no page can go in the sitemap, because an empty sitemap fails with no other signal.
+- `writeSeoFiles({ outDir, siteUrl?, robots? })` — reads every `.html` file under `outDir` and writes `sitemap.xml` and `robots.txt` there. A server export, which has `server/_expo/routes.json` and `client/`, is read from `<outDir>/server`, and the two files go to `<outDir>/client`, the folder the server sends as files; every `/node` function below reads the same layout. A page goes in the sitemap with its canonical link, when the link is under the site and the page has no `noindex`. Left out, with a `reason`: a special route (a `+` or `_` segment, or a root `404.html`), a dynamic route template (`[id].html`), `noindex`, no canonical link, a canonical link outside the site, and a second page with the same link. A page with `hreflang` links gets the same links as `xhtml:link` entries. `siteUrl` defaults to the canonical link of `index.html`. `robots: false` writes no `robots.txt`; an object passes options to `writeRobots`. Resolves to `{ siteUrl, sitemap, robots?, urls, skipped: { file, reason }[] }`. Each sitemap entry holds `loc` only: Google ignores `changefreq` and `priority`, and an export does not know `lastmod`. Throws when no page can go in the sitemap, because an empty sitemap fails with no other signal.
 - `readExport(outDir)` — `{ file, route, canonical?, noindex, alternates: { lang, url }[] }[]` for every exported HTML file, sorted by path. Use it to build a URL list of your own.
 - `rootnative-seo audit <outDir> [--site-url <url>] [--ignore <rule>]...` — the gate for CI, after the export and after `rootnative-seo <outDir>`. It runs `auditExport`, prints each problem with its file and rule, and exits with 1 when there is one. It writes nothing. `--ignore` skips one rule; repeat it for more.
 - `auditExport({ outDir, siteUrl?, ignore? })` — reads every `.html` file under `outDir` and returns `{ pages, problems: { file, rule, message }[] }`. The rules: `title` (one `<title>` with text), `canonical` (an absolute canonical link, or `noindex`), `h1` (exactly one `<h1>`), `main` (exactly one `<main>`), `lang` (a `lang` on `<html>`), `alt` (every `<img>` has an `alt` attribute), `hreflang` (on each indexable page with `hreflang` links: every code is a language code or `x-default`, not `-UK`; every link is absolute; one link per code; a link to the page itself; every linked page in the export has a file, no `noindex`, and the same set; and the `xhtml:link` entries of the sitemap, when it has them, are the links of the page), and `sitemap` (`sitemap.xml` exists, every `<loc>` is the canonical link of a page, and every page `writeSeoFiles` would list is in it). `siteUrl` defaults to the canonical link of `index.html`; with neither, the second half of the `sitemap` rule is skipped. An empty `problems` list is a pass. It throws only when the folder is missing or holds no HTML file.
 - `rootnative-seo browse <outDir> [--chrome <path>] [--base-path <path>] [--settle <ms>]` — the second gate for CI, after the export. It runs `browseExport`, prints each problem with its file and kind, and exits with 1 when there is one. It writes nothing. Chrome comes from `--chrome`, then `CHROME_PATH`, then the usual names on `PATH` and the usual install paths.
 - `browseExport({ outDir, basePath?, chrome?, settleMs?, timeoutMs? })` — serves `outDir` and loads every `.html` file in headless Chrome, driven over its debugging pipe with no package. Returns `{ pages, problems: { file, route, kind, message }[], chrome }`, where `kind` is `exception` (uncaught), `console` (`console.error` or `console.assert`), or `log` (a browser log entry, such as a request that failed). A hydration mismatch is a `console.error` in development and a reported error in production, so both reach the list. `settleMs` (default 750) is how long it listens after the load event, so hydration runs. Throws when no Chrome is found, the folder is missing, or it holds no HTML file.
-- `serveExport({ outDir, basePath?, port? })` — serves `outDir` on `127.0.0.1` as a static host serves an Expo Router export: `/about` is `about.html` or `about/index.html`, a route with no file gets `+not-found.html` or `404.html` with status 404, and a request outside `outDir` is refused. `basePath` defaults to the path of the canonical link of `index.html`. Resolves to `{ origin, basePath, urlOf(route), close() }`. Use it to run Lighthouse or a browser test on the export.
+- `serveExport({ outDir, basePath?, port? })` — serves `outDir` on `127.0.0.1` as a static host serves an Expo Router export (a server export: a file from `client/`, then a page from `server/`, never the route manifest; a dynamic route gets its own file only, where the Expo server sends the template for every slug): `/about` is `about.html` or `about/index.html`, a route with no file gets `+not-found.html` or `404.html` with status 404, and a request outside `outDir` is refused. `basePath` defaults to the path of the canonical link of `index.html`. Resolves to `{ origin, basePath, urlOf(route), close() }`. Use it to run Lighthouse or a browser test on the export.
 - `findChrome(path?)` — the Chrome or Chromium binary `browseExport` would use, or `undefined`.
 
 The three functions below write one file each. Each creates `outDir`, writes the file and resolves to its path. Each has a pure sibling that returns the text.

example/.gitignore

@@ -2,6 +2,7 @@ node_modules/
 .expo/
 .expo-shared/
 dist/
+dist-server/
 web-build/
 .DS_Store
 *.pem

example/app.config.ts

@@ -0,0 +1,13 @@
+import type { ConfigContext, ExpoConfig } from 'expo/config'
+
+/**
+ * `app.json` holds the config. `WEB_OUTPUT=server` exports the same app with
+ * `web.output: 'server'`, for the server export check in CI.
+ */
+export default ({ config }: ConfigContext): Partial<ExpoConfig> => ({
+  ...config,
+  web: {
+    ...config.web,
+    output: process.env.WEB_OUTPUT === 'server' ? 'server' : config.web?.output,
+  },
+})

example/package.json

@@ -11,7 +11,8 @@
     "typecheck": "tsc --noEmit",
     "export:web": "expo export --platform web && node scripts/write-seo.mjs",
     "lint": "eslint . --max-warnings 0",
-    "check:export": "expo export --platform web && node scripts/write-seo.mjs && node scripts/check-export.mjs"
+    "check:export": "expo export --platform web && node scripts/write-seo.mjs && node scripts/check-export.mjs",
+    "check:export:server": "WEB_OUTPUT=server expo export --platform web --output-dir dist-server && node scripts/write-seo.mjs dist-server && node scripts/check-export.mjs --dist dist-server --no-browser"
   },
   "dependencies": {
     "@react-native-vector-icons/material-design-icons": "^13.1.4",