docs: add version picker and manual release snapshots

.github/workflows/docs.yaml

@@ -71,6 +71,7 @@ jobs:
             -a \
             --delete \
             --exclude '/.git/' \
+            --exclude '/versions/' \
             ../docs/build/html/ \
             ./
           cp ../.asf.yaml .

dev/release/README.md

@@ -313,6 +313,68 @@ git tag 55.2.0
 git push apache 55.2.0
 ```
 
+#### Publish the versioned documentation
+
+**1. Build the documentation from the release tag using the a configuration
+overlay (`docs/scripts/release/conf.py`) from the main checkout. For example,
+for `55.0.0`, from the repository root:
+
+```shell
+export DATAFUSION_DOCS_OVERLAY="$PWD/docs/scripts/release"
+git fetch apache tag 55.0.0
+git worktree add --detach /tmp/datafusion-55.0.0 55.0.0
+cd /tmp/datafusion-55.0.0/docs
+DATAFUSION_DOCS_SOURCE="$PWD/source" DATAFUSION_DOCS_VERSION=55.0.0 \
+  SPHINXOPTS="-W -c $DATAFUSION_DOCS_OVERLAY" \
+  uv run --package datafusion-docs --with sphinx-sitemap ./build.sh
+cd -
+```
+
+Notes:
+1. You need ot use an absolute path to the overlay in your current checkout. 
+2. The `--with sphinx-sitemap` option supplies the sitemap extension for older tags.
+
+prepare a PR targeting `asf-site` that contains the
+documentation for this release. (For example
+[#25881](https://github.com/apache/datafusion/pull/25881)).
+
+```shell
+mkdir -p docs/build/html/versions/55.0.0
+rsync -a /tmp/datafusion-55.0.0/docs/build/html/ docs/build/html/versions/55.0.0/
+python3 -m http.server --directory docs/build/html 8000
+```
+
+**2. Add the built site to the `asf-site` branch.** Open a PR against
+`asf-site` that adds the release under `versions/<version>/`:
+
+```shell
+git fetch apache asf-site
+git worktree add -b docs/publish-55.0.0 /tmp/datafusion-asf-site apache/asf-site
+rsync -a --exclude '/.buildinfo' \
+  /tmp/datafusion-55.0.0/docs/build/html/ \
+  /tmp/datafusion-asf-site/versions/55.0.0/
+cd /tmp/datafusion-asf-site
+git add versions/55.0.0
+git commit -m "Publish documentation for 55.0.0"
+git push origin docs/publish-55.0.0
+```
+
+The release is live at `https://datafusion.apache.org/versions/55.0.0/` once
+the PR is merged.
+
+**3. Add the release to the version picker.** Open a PR against `main` that
+adds the release to `docs/source/_static/versions.json`. Put the new release
+first and move `"preferred": true` to it, so that it is the default shown.
+Only add releases that are already published, since the picker links to them
+directly.
+
+Finally, remove the temporary worktrees:
+
+```shell
+git worktree remove /tmp/datafusion-55.0.0
+git worktree remove /tmp/datafusion-asf-site
+```
+
 ### 10. Publish on Crates.io
 
 Only approved releases of the tarball should be published to

docs/README.md

@@ -20,8 +20,9 @@
 # DataFusion Documentation
 
 This folder contains the source content of the [User Guide](./source/user-guide)
-and [Contributor Guide](./source/contributor-guide). These are both published to
-https://datafusion.apache.org/ as part of the release process.
+and [Contributor Guide](./source/contributor-guide). The site root shows the
+development documentation built from `main`. Released versions of the complete
+site are available under `/versions/<version>/`.
 
 ## Dependencies
 
@@ -48,9 +49,7 @@ Run the provided script to build the HTML pages.
 ```
 
 The HTML will be generated into a `build` directory. Open `build/html/index.html`
-in your preferred browser, e.g.
-
-Preview the site on Linux by running this command.
+in your preferred browser, for example on Linux via
 
 ```bash
 # On macOS
@@ -59,6 +58,15 @@ open build/html/index.html
 firefox build/html/index.html
 ```
 
+Note that some features of the site, such as the "Version Picker" do not work
+when read from files. To preview such features, you can use the sphinx like this:
+
+```bash
+uv run --with sphinx-autobuild sphinx-autobuild source build/html --port 8000
+```
+
+And then open http://localhost:8000/ in your browser
+
 ## Making Changes
 
 To make changes to the docs, simply make a Pull Request with your
@@ -69,7 +77,7 @@ automatically updated.
 
 This documentation is hosted at https://datafusion.apache.org/
 
-When the PR is merged to the `main` branch of the DataFusion
+When a PR is merged to the `main` branch of the DataFusion
 repository, a [github workflow](https://github.com/apache/datafusion/blob/main/.github/workflows/docs.yaml) which:
 
 1. Builds the html content

docs/scripts/release/conf.py

@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Sphinx configuration overlay for building docs from a release tag.
+
+Loads ``conf.py`` from a``docs/source`` directory located at
+``DATAFUSION_DOCS_SOURCE`` and overrides the following settings:
+- the version string (``DATAFUSION_DOCS_VERSION``),
+- the site base URL used for the sitemap
+- the version switcher.
+
+Why?
+
+Documentation from older release tags do not have the version picker code and
+expect their base URL to be ``/`` rather than ```versions/<version>/``, so we can
+not publish docs built for the archived version without modification.
+
+Rather than modifying the documentation in the old release branch,
+the release process builds docs with the following command from a main
+checkout:
+
+sphinx-build -c docs/scripts/release
+"""
+
+import os
+from pathlib import Path
+import sys
+
+source = Path(os.environ["DATAFUSION_DOCS_SOURCE"]).resolve()
+sys.path.insert(0, str(source.parent))
+exec(compile((source / "conf.py").read_bytes(), str(source / "conf.py"), "exec"))
+
+# Resolve paths relative to the tag, not this configuration overlay.
+templates_path = [str(source / path) for path in templates_path]
+html_static_path = [str(source / path) for path in html_static_path]
+html_extra_path = [str(source / path) for path in html_extra_path]
+html_logo = str(source / html_logo)
+html_favicon = str(source / html_favicon)
+
+version = release = os.environ["DATAFUSION_DOCS_VERSION"]
+html_baseurl = f"https://datafusion.apache.org/versions/{version}/"
+sitemap_url_scheme = "{link}"
+if "sphinx_sitemap" not in extensions:
+    extensions.append("sphinx_sitemap")
+html_context = {**html_context, "github_repo": "datafusion", "github_version": version}
+html_theme_options = {
+    **html_theme_options,
+    "navbar_end": ["version-switcher", "theme-switcher"],
+    "check_switcher": False,
+    "switcher": {
+        "json_url": "/_static/versions.json",
+        "version_match": version,
+    },
+}
+# The old tag's absolute redirect would leave the release documentation.
+redirects = {**redirects, "library-user-guide/upgrading": "upgrading/index.html"}

docs/source/_static/versions.json

@@ -0,0 +1,17 @@
+[
+  {
+    "name": "Latest",
+    "url": "/",
+    "version": "main"
+  },
+  {
+    "name": "55.1.0",
+    "url": "/versions/55.1.0/",
+    "version": "55.1.0"
+  },
+  {
+    "name": "55.0.0",
+    "url": "/versions/55.0.0/",
+    "version": "55.0.0"
+  }
+]

docs/source/conf.py

@@ -37,6 +37,7 @@
 project = "Apache DataFusion"
 copyright = "2019-2025, Apache Software Foundation"
 author = "Apache Software Foundation"
+version = release = "main"
 
 
 # -- General configuration ---------------------------------------------------
@@ -95,7 +96,13 @@
     },
     "use_edit_page_button": True,
     "navbar_center": [],
-    "navbar_end": ["theme-switcher"],
+    "navbar_end": ["version-switcher", "theme-switcher"],
+    # Release docs are published separately; the manifest may not be live yet.
+    "check_switcher": False,
+    "switcher": {
+        "json_url": "/_static/versions.json",
+        "version_match": version,
+    },
 }
 
 html_context = {