Add Jupyter Book documentation infrastructure with auto-generated API reference via sphinx-autoapi by Mohit-Lakra · Pull Request #272 · mllam/neural-lam

Mohit-Lakra · 2026-02-26T16:09:40Z

Describe your changes

Adds a Jupyter Book-based documentation site with a fully automated API reference
generated by sphinx-autoapi. This means every module, class, and function in
neural_lam/ gets a structured reference page built directly from source code —
with zero manually maintained .rst files.

Files added:

docs/_config.yml — Jupyter Book config with sphinx-autoapi pointed at
../neural_lam/
docs/_toc.yml — site structure (Tutorials / How-To / API Reference /
Background / Contributing)
docs/intro.md + docs/installation.md — landing page and install guide
.github/workflows/deploy-docs.yml — builds docs on every PR, deploys to
GitHub Pages on merge to main
docs/scripts/autoapi_astroid_patch.py — patch to ensure sphinx-autoapi
works correctly with the project's dependency tree

Dependencies (docs group only, not required for model usage):
jupyter-book, sphinx-autoapi, myst-nb, sphinx-copybutton,
pydata-sphinx-theme

Install with:

pdm install --group docs

Build locally with:

jupyter-book build docs/

Live prototype: https://mohit-lakra.github.io/neural-lam/autoapi/index.html (intro page: https://mohit-lakra.github.io/neural-lam)

Issue Link

closes #61

Type of change

🐛 Bug fix (non-breaking change that fixes an issue)
✨ New feature (non-breaking change that adds functionality)
💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
📖 Documentation (Addition or improvements to documentation)

Checklist before requesting a review

My branch is up-to-date with the target branch
I have performed a self-review of my code
For any new/modified functions/classes I have added docstrings that clearly describe its purpose, expected inputs and returned values
I have placed in-line comments to clarify the intent of any hard-to-understand passages of my code
I have updated the README to cover introduced code changes
I have added tests that prove my fix is effective or that my feature works
I have given the PR a name that clearly describes the change, written in imperative form
I have requested a reviewer and an assignee

Checklist for reviewers

the code is readable
the code is well tested
the code is documented (including return types and parameters)
the code is easy to maintain

Author checklist after completed review

I have added a line to the CHANGELOG describing this change:
- added: Jupyter Book documentation site with sphinx-autoapi for
  auto-generated API reference and GitHub Actions deploy workflow (relates
  to Structured documentation for Neural-LAM #61)

Checklist for assignee

PR is up to date with the base branch
the tests pass
author has added an entry to the changelog (designated as added)
Once the PR is ready to be merged, squash commits and merge the PR.

Mohit-Lakra · 2026-02-26T16:12:31Z

@joeloskarsson @leifdenby
Could you please review it when you have a moment

sadamov · 2026-02-26T19:48:06Z

@Mohit-Lakra I just had a look at your autogenerated docs you are hosting. It looks great! This is certainly a solid technical framework to solve this issue. There is however another PR already working on this #252 with similar albeit not identical dependencies and methods.

Do you think you could coordinate with @varma1221 directly from that other PR and consolidate towards one common solution?

Mohit-Lakra · 2026-02-27T01:10:47Z

@sadamov, I've already looked at @varma1221's PR; in fact, his ReadTheDocs setup is what I'm missing. @leifdenby had enquired about per-PR preview URLs, and that configuration takes care of it
@varma1221 if you're up for it, easiest path is probably just porting that .readthedocs.yaml and the CI workflow into this branch — no point duplicating the whole thing. lmk how you want to do it 🙌

varma1221 · 2026-02-27T01:21:04Z

@Mohit-Lakra, I will open a PR against your branch with the .readthedocs.yaml and CI workflow adapted for the Jupyter Book setup. I will also fix the PEP 735 incompatibility in the RTD config while i am at it.

Mohit-Lakra · 2026-02-27T01:25:57Z

@varma1221 Sounds fantastic, appreciate it 👍

AmanChawan · 2026-03-18T07:27:46Z

I tried building this branch locally and found that train_model.py's main entry point has only a one-line docstring despite having 20+ CLI arguments. Would it make sense to add docstring coverage for this file as part of this PR, or handle it separately? I also noticed the models/ inheritance chain means gaps in ar_model.py propagate to all subclasses — happy to help audit these.

Mohit-Lakra · 2026-03-18T08:51:44Z

I tried building this branch locally and found that train_model.py's main entry point has only a one-line docstring despite having 20+ CLI arguments. Would it make sense to add docstring coverage for this file as part of this PR, or handle it separately? I also noticed the models/ inheritance chain means gaps in ar_model.py propagate to all subclasses — happy to help audit these.

Please check #252 for docstrings

Aditi2k5 · 2026-03-19T03:09:53Z

Hello @Mohit-Lakra and @varma1221, I have been looking into the documentation with this PR and found a small UI bug with the collapsible sidebar when scrolling down. The sidebar when scrolling down is cut in half and not extended till the end of the page. If okay, shall I work on solving this as an issue and open a PR for the same?

Mohit-Lakra · 2026-03-19T06:44:15Z

Hello @Mohit-Lakra and @varma1221, I have been looking into the documentation with this PR and found a small UI bug with the collapsible sidebar when scrolling down. The sidebar when scrolling down is cut in half and not extended till the end of the page. If okay, shall I work on solving this as an issue and open a PR for the same?

Thanks for looking,
https://github.com/Mohit-Lakra/neural-lam/tree/prototype-2
Check this branch you can raise a PR here

Current PR is only for API reference docs

alphaleporus · 2026-03-19T20:25:30Z

Hi @Mohit-Lakra, I tried merging your Auto-doc branch against current main locally to test the build and hit the conflict directly:

CONFLICT (content): Merge conflict in pyproject.toml

Should be straightforward to resolve, most likely a [tool.pdm.dev-dependencies] section clash from upstream dependency updates since your last rebase. Running git fetch upstream && git rebase upstream/main and fixing that section should clear it.

Also confirmed locally that ARModel has 8 Lightning lifecycle hooks (configure_optimizers, predict_step, training_step, validation_step, on_validation_epoch_end, test_step, on_test_epoch_end, on_load_checkpoint). Worth a quick sanity check against the live autoapi output to make sure all 8 show up in the generated reference once the conflict is resolved. Happy to verify once it's rebased.

Mohit-Lakra · 2026-06-21T16:25:23Z

@sadamov thanks for the feedback, I completely agree with your thinking regarding both the scope and the tooling.
I have updated the the PR to reflect your proposed changes.

I dropped all the outdated handwritten docs and strictly uses the autoapi refrence docs and a minimal intro only

I replaced Jupyter Book with sphinx + sphinx-autoapi + myst-nb + sphinx-book-theme

I also entirely removed the GitHub Pages deployment step from the CI workflow; we will exclusively stick to ReadTheDocs moving forward.

I removed unused steps from the CI like pyreverse UML generation, since its host page was dropped and cleaned up pyproject.toml accordingly

sadamov

Thanks @Mohit-Lakra, this settles all four points from last round.
Two blockers on our own rules, the rest are inline:

No CHANGELOG entry for this PR (the #252 line only mentions #272 in passing).
This deletes docs/notebooks/create_reduced_meps_dataset.ipynb, the only file in docs/notebooks/, but README.md:535 still points there and the notebook documents how meps_example_reduced was built. I would restore it (out of scope to drop).

What do you think?

sadamov · 2026-06-22T10:04:08Z

+[tool.setuptools]
+py-modules = ["neural_lam"]
+


Backend is hatchling (line 183), so [tool.setuptools] is never read, and py-modules is the wrong concept for a package. Drop it.

Suggested change

[tool.setuptools]

py-modules = ["neural_lam"]

sadamov · 2026-06-22T10:04:08Z

+}
+
+# OpenGraph settings
+ogp_site_url = "https://mllam.github.io/neural-lam/"


ogp_site_url still points at the Pages prototype. Update to the RTD URL (placeholder below, set once the slug is known).

Suggested change

ogp_site_url = "https://mllam.github.io/neural-lam/"

ogp_site_url = "https://neural-lam.readthedocs.io/en/latest/"

sadamov · 2026-06-22T10:04:08Z

+# HTML Theme
+html_theme = "sphinx_book_theme"
+html_logo = "_static/logo.png"
+html_favicon = "_static/favicon.ico"


favicon.ico is 0 bytes, so this ships a broken favicon. Commit a real icon or drop the line.

Suggested change

html_favicon = "_static/favicon.ico"

sadamov · 2026-06-22T10:04:08Z

+# Run a minimal test to trigger data download
+pytest tests/test_training.py -vv -s -k "test_training" --co


--co is collect-only, it never runs the fixture, so it won't download anything. Drop it.

Suggested change

# Run a minimal test to trigger data download

pytest tests/test_training.py -vv -s -k "test_training" --co

# Run a minimal test to trigger the data download

pytest tests/test_training.py -vv -s -k "test_training"

sadamov · 2026-06-22T10:04:08Z

@@ -0,0 +1,133 @@
+classDiagram


Nothing references these anymore (grep -rn _static/uml docs/ is empty) and they will drift from the code. I suggest to remove docs/_static/uml/.

sadamov · 2026-06-22T10:04:08Z

@@ -0,0 +1,260 @@
+/* Neural-LAM Documentation — High-Visibility Custom Styles */


260 lines of bespoke design-system overrides plus a remote Google Fonts @import (line 45) on a theme we agreed to track. I suggest to trim to what you need or drop it for the sphinx_book_theme defaults; remove the remote font @import regardless.

sadamov · 2026-06-22T10:04:08Z

+  tools:
+    python: "3.10"
+  commands:
+    - pip install uv
+    - uv pip install --system -e ".[cpu,docs]"


RTD can't be CI-tested yet, two things to verify: uv pip install (vs uv sync in CI) does not honour [tool.uv.sources], so RTD may pull the full CUDA torch and hit its time/disk limits; and python: "3.10" here vs 3.11 in docs.yml - align them.

- Update OpenGraph URL to ReadTheDocs and drop empty favicon - Remove --co flag from quickstart test command - Drop unused UML diagrams and custom CSS - Restore dataset notebook and add to toctree - Align RTD config with CI python version and uv sync - Add CHANGELOG entry for docs migration

Mohit-Lakra · 2026-06-22T17:14:37Z

Thanks @sadamov,
All feedback applied:

Removed [tool.setuptools] block from pyproject.toml.
Updated ogp_site_url to the placeholder ReadTheDocs URL.
Dropped the empty favicon.ico line.
Removed --co from the quickstart pytest command.
Removed outdated UML diagrams in docs/_static/uml/.
Dropped custom.css to rely on the default sphinx_book_theme styling.
Restored create_reduced_meps_dataset.ipynb and added it to the toctree.
Added the requested CHANGELOG.md entry.
Updated .readthedocs.yaml to use Python 3.11 and uv sync to correctly route PyTorch indices.

sadamov

Okay I did a full thorough review now, including inspection of the rendered docs end-to-end.

A few structural questions:

Installation duplicates the README, and installation.md has already drifted from it: its pip install block uses a .[dev] extra and a cu118 wheel that are both wrong here but correct in the README. Could we make it thin - the "build docs locally" block plus a MyST {include} of the README install section - so there is one source?
Tutorials: I would group all three example notebooks under one Tutorials section in index.md, kept in docs/notebooks/ with nb_execution_mode = "off" and committed outputs (they need data and GPU, so CI/RTD cannot run them): the existing create_reduced_meps_dataset.ipynb (please reset it to main first, the current diff is 234/234 lines of pure metadata churn, and it also sits oddly as a top-level peer today), plus the upcoming DANRA (#577) and COSMO (#392) notebooks. Agree the skeleton now and each is a one-line toctree add?

sadamov · 2026-06-23T09:28:40Z

@Mohit-Lakra I really like these docs! First of all the autodoc renders great, thanks to #252 🤩
for this PR #272 I would like to focus on getting a skeleton set up. This way we can make sure everything works well and reduce manual maintance to an absolute minimum. the content (mostly notebooks) will come later

…e docs correction

Mohit-Lakra · 2026-06-26T16:25:41Z

Hi @sadamov, thanks for the compliment ✨. I also realised the same these changes will help make the docs faster and better to manage.
All requested changes are applied.
You can review it now. I'd be happy to make any further changes if you want 😄 😄

sadamov

Thanks @Mohit-Lakra, this is basically ready. I rebuilt the site from the head with sphinx-build -W --keep-going (html and linkcheck) in a fresh env with no torch and both pass 👍 Two small things below, could you implement them? Then I would love for you to showcase the docs at the next dev meeting.

Co-authored-by: sadamov <45732287+sadamov@users.noreply.github.com>

Mohit-Lakra · 2026-06-29T08:54:22Z

Thanks @sadamov I have implemented the two requested changes. Please let me know if there's anything else that needs to be adjusted.

Regarding the dev meeting, I'd be happy to showcase the docs if my schedule allows, but I can't guarantee that I'll be able to attend.

sadamov · 2026-06-29T08:58:38Z

@Mohit-Lakra no worries if you're not there I can show the community

sadamov · 2026-07-12T20:55:35Z

@Mohit-Lakra for the showcase tomorrow I had to merge main, which added a new docs file about graph specs. I mvoed that one and the existing API into a new "References" Section.

sadamov · 2026-09-14T12:02:57Z

Heads-up: we discussed the notebook format in the dev meeting and ran a poll in Slack afterwards, marimo won over Jupyter. Going forward notebooks in this repo are marimo .py files under docs/notebooks/, no .ipynb in git anymore.

For this PR that means: convert the notebook with marimo convert <name>.ipynb -o <name>.py, drop the .ipynb, and check it runs top to bottom with python docs/notebooks/<name>.py (marimo notebooks are plain python scripts, so no kernel is needed and ruff lints them like the rest of the code). marimo edit <name>.py if you want to tidy up cells after the conversion.

Sorry for the churn, I know this lands mid-review. Let me know if smth is unclear or if you would rather I push the conversion to your branch.

This one needs a bit more thought since jupyter-book / myst-nb render .ipynb directly. marimo can export (marimo export html / marimo export md) so I would add an export step to the docs build and point the toc at the exported files, which would also let us drop myst-nb, jupyter-client and jupyter-core from the deps. What do you think?

Mohit-Lakra · 2026-10-01T05:23:01Z

Yeah, that makes sense to me. I’m happy to switch the notebook to marimo and drop the .ipynb.

For the docs, I agree that adding a marimo export step to the docs build and pointing the TOC at the generated files is a good approach. It also makes sense to remove the Jupyter-related dependencies if they’re no longer needed.

I’ll make those changes in this PR and make sure the notebook runs cleanly with python and the docs build still works.

Generated Autodocs using autoapi

6c7b113

Mohit-Lakra mentioned this pull request Feb 26, 2026

Structured documentation for Neural-LAM #61

Open

sadamov mentioned this pull request Feb 26, 2026

Initial Sphinx documentation setup with autodoc support #196

Closed

sadamov added the duplicate This issue or pull request already exists label Feb 26, 2026

sadamov mentioned this pull request Feb 26, 2026

Add structured Sphinx-based documentation scaffold (issue #61) #230

Closed

6 of 19 tasks

Mohit-Lakra mentioned this pull request Feb 27, 2026

docs: 100 percent coverage of docstrings #252

Merged

santhil-cyber mentioned this pull request Feb 27, 2026

ci: add GitHub Actions workflow for documentation build and deploy #286

Closed

6 of 12 tasks

Mohit-Lakra force-pushed the Auto-doc branch from d14e6d8 to 51f2fab Compare March 1, 2026 07:24

torch dependency removed

b16b3dd

Mohit-Lakra force-pushed the Auto-doc branch from 51f2fab to b16b3dd Compare March 1, 2026 07:39

sadamov added the documentation Improvements or additions to documentation label Mar 1, 2026

joeloskarsson mentioned this pull request Mar 6, 2026

docs: add initial Jupyter Book documentation scaffold #322

Closed

joeloskarsson requested a review from leifdenby March 15, 2026 14:10

sadamov removed the duplicate This issue or pull request already exists label Mar 15, 2026

sadamov mentioned this pull request Mar 23, 2026

My work: documentation system setup (issue #61) #501

Closed

17 of 22 tasks

kshirajahere mentioned this pull request Mar 25, 2026

Solving Issues#61 #517

Closed

17 of 22 tasks

sadamov mentioned this pull request Mar 27, 2026

Add initial Sphinx documentation setup with API reference #524

Closed

15 tasks

Mohit-Lakra mentioned this pull request Mar 29, 2026

Restore missing documentation source and fix build configurationFix documentation source Mohit-Lakra/neural-lam#4

Closed

17 of 22 tasks

This was referenced Mar 30, 2026

Add Sphinx-based documentation system and supporting files. #532

Closed

Initial documentation improvement and setup groundwork, Added initial documentation improvement #549

Closed

docs : Formatting Fix

b02bf79

Mohit-Lakra force-pushed the Auto-doc branch from 5540e05 to b02bf79 Compare June 21, 2026 16:29

sadamov reviewed Jun 22, 2026

View reviewed changes

sadamov reviewed Jun 23, 2026

View reviewed changes

Mohit-Lakra added 2 commits June 26, 2026 21:06

Removing mermaid dependencies, correcting slack channel link, and som…

2821398

…e docs correction

docs: address review feedback

517175b

Mohit-Lakra requested a review from sadamov June 28, 2026 16:48

Merge branch 'main' into Auto-doc

c419422

sadamov reviewed Jun 29, 2026

View reviewed changes

Comment thread docs/getting-started/quickstart.md Outdated

Comment thread docs/getting-started/installation.md Outdated

Mohit-Lakra and others added 2 commits June 29, 2026 14:18

changed hilam to hi_lam

5bc53ee

Co-authored-by: sadamov <45732287+sadamov@users.noreply.github.com>

dropping cpu --extra

589e52a

Co-authored-by: sadamov <45732287+sadamov@users.noreply.github.com>

sadamov and others added 3 commits July 12, 2026 21:50

Merge branch 'main' into Auto-doc

1bfa846

added graph_storage_spec to toctree

ad849a5

moved the References into their own section

9f5a79f

leifdenby modified the milestones: v0.8.0 (proposed), v0.8.0 Jul 13, 2026

sadamov mentioned this pull request Jul 15, 2026

Add DANRA tutorial notebook with pytest-nbmake #69

Open

sadamov added 2 commits August 17, 2026 12:14

Merge branch 'main' into Auto-doc

8c19c08

Merge branch 'main' into Auto-doc

75b7b81

sadamov mentioned this pull request Aug 23, 2026

Author notebooks as jupytext-paired py:percent sources instead of committing raw .ipynb #708

Open

	ogp_site_url = "https://mllam.github.io/neural-lam/"
	ogp_site_url = "https://neural-lam.readthedocs.io/en/latest/"

		# Run a minimal test to trigger data download
		pytest tests/test_training.py -vv -s -k "test_training" --co

		@@ -0,0 +1,260 @@
		/* Neural-LAM Documentation — High-Visibility Custom Styles */

Conversation

Mohit-Lakra commented Feb 26, 2026 • edited by leifdenby Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes

Issue Link

Type of change

Checklist before requesting a review

Checklist for reviewers

Author checklist after completed review

Checklist for assignee

Uh oh!

Mohit-Lakra commented Feb 26, 2026

Uh oh!

sadamov commented Feb 26, 2026

Uh oh!

Mohit-Lakra commented Feb 27, 2026

Uh oh!

varma1221 commented Feb 27, 2026

Uh oh!

Mohit-Lakra commented Feb 27, 2026

Uh oh!

AmanChawan commented Mar 18, 2026

Uh oh!

Mohit-Lakra commented Mar 18, 2026

Uh oh!

Aditi2k5 commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Mohit-Lakra commented Mar 19, 2026

Uh oh!

alphaleporus commented Mar 19, 2026

Uh oh!

Mohit-Lakra commented Jun 21, 2026

Uh oh!

sadamov left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

sadamov Jun 22, 2026

Choose a reason for hiding this comment

Uh oh!

Mohit-Lakra commented Jun 22, 2026

Uh oh!

sadamov left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sadamov commented Jun 23, 2026

Uh oh!

Mohit-Lakra commented Jun 26, 2026

Uh oh!

sadamov left a comment

Choose a reason for hiding this comment

Mohit-Lakra commented Feb 26, 2026 •

edited by leifdenby

Loading

Aditi2k5 commented Mar 19, 2026 •

edited

Loading

sadamov left a comment •

edited

Loading