Skip to content

Migrate datasources panel to the catalog children model#599

Draft
Light2Dark wants to merge 1 commit into
mainfrom
catalog-children-model
Draft

Migrate datasources panel to the catalog children model#599
Light2Dark wants to merge 1 commit into
mainfrom
catalog-children-model

Conversation

@Light2Dark

Copy link
Copy Markdown
Member

Summary

Adapts the extension to marimo-team/marimo#9874, which unifies catalog browsing around a recursive Database.children tree and one lazy-load endpoint. That PR:

  • replaces the flat Database.schemas list with a tagged children tree (schema / namespace / data_table nodes), using null buckets for deferred discovery; and
  • replaces the sql-table-list-preview + sql-schema-list-preview notifications with a single catalog-children-preview.

Since our notification/schema types are generated from @marimo-team/openapi, that is a compile break here once .marimo-version is bumped to the release containing #9874 (Database.schemas, SqlTableListPreviewNotification, and the old op tags all disappear). This migrates the consumers.

Changes

  • types.ts — drop SqlTableListPreviewNotification; add CatalogChildrenPreviewNotification and CatalogNode / SchemaNode / NamespaceNode / DataTableNode.
  • KernelManager.ts — route catalog-children-preview; the now-removed sql-schema-list-preview no longer needs to be ignored (the exhaustive unreachable(operation) switch enforces full coverage).
  • DatasourcesService.ts — databases now hold a recursive CatalogTreeNode tree instead of a flat schema map. normalizeCatalogNode walks schema.tables / namespace.children and flattens deferred (null) buckets to [] (the panel renders what the kernel has discovered; it does not drive lazy fetches). updateCatalogChildrenPreview skips error responses so a transient failure can't erase cached children.
  • DatasourcesView.ts — generalized the fixed Connection → Database → Schema → Table view into a recursive Connection → Database → (Schema | Namespace)* → Table tree using path-prefix matching. Adds a marimoNamespace contextValue (no menus key on it today). In-memory datasets render unchanged.
  • TestsDatasourcesService.test.ts (the panel's first): tree normalization (nested namespaces, root-level tables), deferred-null flattening, the error-skip guard, and clearNotebook.

Verification

Verified by checking out the #9874 commit in a sibling marimo checkout (so the linked @marimo-team/openapi + frontend are consistent):

  • tsc — clean
  • vp test — 5/5 pass
  • vp check (lint + format) — clean

⚠️ Merge ordering

This must land together with the .marimo-version bump to the release containing #9874. Against the currently pinned openapi package the new wire types don't exist yet, so CI here will not go green until that bump is in this branch.

🤖 Generated with Claude Code

marimo-team/marimo#9874 replaces the flat `Database.schemas` list and the
`sql-table-list-preview` / `sql-schema-list-preview` notifications with a
recursive `Database.children` tree (tagged `schema` / `namespace` /
`data_table` nodes) and a single `catalog-children-preview` notification.

Because the extension's notification types are generated from
`@marimo-team/openapi`, that change is a compile break here once the marimo
version is bumped. This migrates the consumers:

- types.ts: drop `SqlTableListPreviewNotification`; add
  `CatalogChildrenPreviewNotification` plus `CatalogNode` / `SchemaNode` /
  `NamespaceNode` / `DataTableNode`.
- KernelManager: route `catalog-children-preview`; the removed
  `sql-schema-list-preview` no longer needs ignoring (the exhaustive switch
  enforces this).
- DatasourcesService: store databases as a recursive `CatalogTreeNode` tree;
  `normalizeCatalogNode` walks `schema.tables` / `namespace.children` and
  flattens deferred (`null`) buckets to `[]`. `updateCatalogChildrenPreview`
  skips error responses so a transient failure can't erase cached children.
- DatasourcesView: render Connection -> Database -> (Schema | Namespace)* ->
  Table recursively via path-prefix matching; adds a `marimoNamespace`
  contextValue.
- Add DatasourcesService tests (the panel's first) covering tree
  normalization, deferred flattening, the error guard, and clearNotebook.

Lands with the `.marimo-version` bump to the release containing #9874 — the
new wire types don't exist in the currently pinned openapi package.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions

Copy link
Copy Markdown
Contributor

Coverage Report

Area Lines Branches Functions
Python 48.17% (503 / 988) 37.97% (101 / 266)
TypeScript 49.16% (2210 / 4495) 40.23% (865 / 2150) 46.81% (617 / 1318)

TypeScript statements: 48.95% (2296 / 4690)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant