fix: initialize the local database during setup

[RomneyDa]

What Problem This Solves

Fixes: a fresh local gitcrawl init reports success but leaves the configured database missing. Read-only commands fail with stat sqlite db: ... no such file or directory until a write command such as sync creates it.

User Impact

Local initialization now creates a database with the current schema before reporting success. Users still sync repositories to populate the archive. Cloud initialization and portable sources are preserved, including existing portable checkouts supplied through --db.

Why This Change Was Made

Use the existing storage initialization and migration owner, and the existing portable-store detector to avoid writable opens of source checkouts. The existing relative-path init test used a writable open that created the missing database and masked the bug; it now opens read-only and reads status immediately after init.

Evidence

• Reproduced on main at 15998d822daf41071c80ae947ec7388141c3e5c9: the changed regression fails at the read-only open with the missing-database error.
• Regression passes after the fix (0.16 seconds test time; 0.518 seconds package time). It extends an existing test; no new suite or process fixture.
• All TestInit cases pass, including portable and cloud setup (44.061 seconds package time).
• Built CLI: isolated init followed by doctor reports an existing, current schema version 16 database with zero threads; repeated init succeeds.
• go vet ./..., docs tests/build, formatting, and git diff --check pass.
• Independent Codex autoreview through P3: no actionable findings.
• Final focused run covers all init cases plus portable source preservation, read/doctor refresh, and creation below unrelated Git metadata: PASS (17.135 seconds package time).
• Full local go test ./... was not green: CLI hit its default 10-minute timeout in TestPortableRefreshRefusalsPreserveLastGood and reported analytics/portable failures. Other packages passed. That run exposed the initial --db portable-source regression, which is fixed and covered by the passing focused run above; the full suite was not rerun after the correction.
• Hosted CI on the first head: docs, Docker, Windows, security scanning, and CodeQL passed. Linux/macOS Go jobs stopped before tests at govulncheck: nine standard-library vulnerabilities in the unchanged Go 1.27.1 toolchain (fixed upstream in 1.27.2). No toolchain/dependency changes are included.

Published-release compatibility proof (October 10, 2026)

PASS against candidate 0e33a3cdda53e0593fe0e794c7fe24fe3737dd04 using the actual published v0.15.0 Darwin arm64 binary, not a reconstructed fixture. Verified the release tarball against published checksums and the GitHub asset digest: cf08a564146725993d286b65e436aa6f89fd708c1a17d9aa10ba84ae2c2e13a0.

1. Used v0.15.0 init --runtime-dir in isolated state, then a real sync openclaw/gitcrawl --numbers 97 --include-comments --with pr-details to populate the archive.
2. Compared sorted rows and hashes for all 66 SQLite tables before and after candidate init --runtime-dir on that populated database. Compared the complete SQLite schema separately. Snapshots were immutable/read-only with all commands closed and no WAL present.
3. All 65 archive-data tables were exactly preserved, including the repository, PR, comment and comment revision, thread revision, document/FTS rows, fingerprint, four PR files, one commit, ten checks, five workflow runs, observation reservations, and sync history.
4. Verified the only two differences against the existing storage owner: idx_threads_repo_embed_order was added by the already-merged embedding index fix, and observation_schema_convergence.checked_observation_sequence advanced from 0 to 1, matching the allocator's existing value. No archive data or schema objects were removed.
5. The release-generated config bytes remained identical when initialized with the same runtime-dir. Candidate threads ... --numbers 97 --include-closed --json returned exactly the release's result; local keyword search found the PR. Doctor reported current schema 16 and healthy SQLite; integrity and foreign-key checks passed.
6. A second candidate init preserved the entire post-upgrade database snapshot and config exactly.

This proves preservation for a populated published v0.15.0 archive through the actual init/read/search boundary. It does not claim coverage of every older schema or custom configuration; init retains its existing config-writing behavior. No additional production changes were needed for this proof. The previously recorded full-suite timeout and Go-toolchain vulnerability scan failures remain separate validation gaps.

Rebase after the toolchain repair

The separate Go 1.27.2 PR #237 passed all hosted checks and merged as cec2fc4e65fa159f620839fee436c12aeca90680. This PR is now rebased onto that main revision, at 79dae7ac98fe9422f769438fe99bbb4b2ad7ec0f.

git range-diff reports both initialization commits unchanged across the rebase. On the new Go 1.27.2 base, the focused init, portable source-preservation, read/doctor refresh, and unrelated-Git-metadata cases pass together (14.010 seconds package time). The published-release preservation proof above remains prior-head evidence for the identical implementation. The historical Go 1.27.1 scan blocker is resolved by the merged toolchain repair; hosted CI is running for the rebased head.

[clawsweeper]

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

[clawsweeper]

Codex review: needs changes before merge.

What this changes

Local setup initializes the SQLite database before reporting success while preserving cloud and portable-source handling.

Example: Run gitcrawl init --runtime-dir /tmp/gitcrawl-demo, then inspect the archive.

• Before: Setup succeeds without creating /tmp/gitcrawl-demo/gitcrawl.db, and subsequent read-only access fails.
• After: Setup creates an empty database with the current schema that read-only commands can open.

Review scores

Measure	Result	What it means
Overall readiness	🐚 platinum hermit (4/6)	Focused repair with meaningful regression coverage and detailed release-compatibility evidence; complete-suite validation remains limited.
Proof confidence	🐚 platinum hermit (4/6)	The MEMBER-authored PR is outside the external-contributor proof gate; its full body records built-CLI init/doctor behavior and candidate init/read/search against a populated published-release archive, without a separately inspectable terminal artifact.
Patch quality	🐚 platinum hermit (4/6)	No actionable review findings were identified.

Product

Kind: Bug fix · Worth it: Yes · Fix scope: Complete
User problem: Successful local setup leaves no database for subsequent read-only commands.
Reason: A narrow maintainer-authored repair makes successful setup produce a usable empty archive without adding configuration or schema definitions.

Merge readiness

⛔ Needs changes before merge - 1 item remains

Keep open: this maintainer-authored PR fixes a source-confirmed initialization bug still present on main, with no concrete introduced defect found.

Priority: P2
Reviewed head: 0e33a3cdda53e0593fe0e794c7fe24fe3737dd04

Before merge

• Complete next step - Establish the cause of the current-head vulnerability-scan failures with accessible diagnostics and a base comparison, then determine their required-check disposition.

Findings

None.

Tests

• Missing end-to-end proof: The author reports CLI fresh-install and published-release upgrade runs, but hosted Go tests were skipped after scanning failed and the full local suite was not rerun after the portable-source correction.

Agent review details

How this fits together

The init command resolves archive paths and writes configuration; local storage creates and migrates SQLite, while portable archives retain their existing source-preservation boundary.

flowchart TD
  A[Init command] --> B[Resolve paths and save config]
  B --> C[Select archive mode]
  C --> D[Local SQLite initialization]
  C --> E[Preserve portable source]
  C --> F[Cloud configuration]
  D --> G[Report setup success]
  E --> G
  F --> G

Loading

Technical review

Best possible solution:

Land the focused initialization repair after resolving the disposition of the failed required checks.

Do we have a high-confidence way to reproduce the issue?

Main visibly omits database creation, and the original test's writable open masks that omission; the author also records the revised test failing on pinned main. No target code was executed during this review.

Is this the best way to solve the issue?

Reusing store.Open fixes initialization at the existing schema owner without duplicating migration logic, and the portable detector preserves the established source boundary.

AGENTS.md: found but not applied because it conflicted with ClawSweeper's review contract.

Codex review notes: model internal, reasoning medium; reviewed against 15998d822daf.

Provenance checked

• Local initialization changes intended behavior with a stated reason (72243a7: The original command initialized configuration and directories; the later extraction explicitly preserved behavior.)
• Anchored-path regression coverage keeps the original intent (fix(init): anchor local filesystem paths #97: Ensure saved filesystem paths remain anchored and usable across working directories.)
• Initialization documentation keeps the original intent (feat: add isolated runtime initialization #80: Document isolated runtime paths while preserving the database-only meaning of --db.)

Testing

Proof path: shipped entry point.

Security

None.

Evidence

What I checked:

• Main still lacks initialization: Main saves configuration and creates runtime directories, then reports success without opening the database. (internal/cli/init.go:117, 15998d822daf)
• Verified introduced behavior: The pinned diff invokes the existing storage initializer only for local paths that the existing detector does not classify as portable. (internal/cli/init.go:126, 0e33a3cdda53)
• Regression test removes masking behavior: The existing anchored-path test now opens read-only and reads status instead of creating the missing database through a writable open. (internal/cli/app_test.go:87, 0e33a3cdda53)
• Published-release compatibility account: The full PR body records candidate initialization of a populated archive produced by the published v0.15.0 binary, preservation of all archive-data tables and configuration, successful reads and search, and idempotent repeated initialization. The two reported database differences match existing index and convergence code. (internal/store/store.go:277, 0e33a3cdda53)
• Feature history and original intent: The original initialization commit saved configuration and directories without creating SQLite. feat: add isolated runtime initialization #80 added isolated paths; fix(init): anchor local filesystem paths #97 anchored paths; refactor: organize CLI commands by responsibility #188 mechanically extracted the command. Their documented path contracts remain intact. Some historical blobs are unavailable locally; GitHub commit and PR reads supplied the relevant provenance. (internal/cli/app.go, 72243a7fdce9)
• Current-head CI stops before tests: Both Go jobs fail at vulnerability scanning; subsequent tests and builds are skipped. Linux annotations identify HTTP/2 calls through github.Client.doOnce. Full job logs redirect to a network-blocked artifact host, and no same-failure base comparison was established, so the failure is neither attributed to this diff nor cleared as unrelated. (.github/workflows/ci.yml:62, 0e33a3cdda53)

Likely related people:

• Peter Steinberger: Raw commit 777a404 adds internal/cli/init.go:116 relative to its recorded parents. This identifies author metadata, not feature responsibility or a PR merger. (role: source-line author; confidence: high; commits: 777a404d28a7; files: internal/cli/init.go)

Labels

Label changes:

• add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
• remove rating: 🦐 gold shrimp: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.

Label justifications:

• P2: Repairs a limited local CLI setup defect with an existing write-command workaround.
• rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
• status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only; repair and merge need explicit maintainer commands such as @clawsweeper autofix or @clawsweeper automerge.

History

Review history (1 earlier review cycle)

• reviewed 2026-10-10T05:02:21.051Z sha be72a4d :: blocked before merge. :: none

Reviewed October 10, 2026, 1:14 AM ET / 05:14 UTC (Revision 2).