Restructure sei data folder for Giga by yzang2019 · Pull Request #3155 · sei-protocol/sei-chain

yzang2019 · 2026-03-31T23:44:33Z

Describe your changes and provide context

The current data folder is not intuitive and not very clean for Giga storage. It currently looks like this:

The goal of this PR is to restructure the data folder for Giga into this new structure while also making it backward compatible:

Testing performed to validate your change

Added unit test to make sure it works for both new nodes and existing nodes which has the old path

github-actions · 2026-03-31T23:45:29Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	Apr 2, 2026, 9:05 AM

codecov · 2026-03-31T23:58:05Z

Codecov Report

❌ Patch coverage is 92.30769% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 58.60%. Comparing base (a9ffa16) to head (b4d61bb).

Files with missing lines	Patch %	Lines
sei-tendermint/cmd/tendermint/commands/reset.go	80.00%	3 Missing and 2 partials ⚠️
...endermint/cmd/tendermint/commands/reindex_event.go	75.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3155      +/-   ##
==========================================
+ Coverage   58.58%   58.60%   +0.01%     
==========================================
  Files        2055     2099      +44     
  Lines      168301   173447    +5146     
==========================================
+ Hits        98594   101642    +3048     
- Misses      60917    62662    +1745     
- Partials     8790     9143     +353

Flag	Coverage Δ
sei-chain-pr	`62.10% <92.30%> (?)`
sei-db	`70.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
app/receipt_store_config.go	`71.42% <100.00%> (ø)`
sei-db/common/utils/path.go	`100.00% <100.00%> (ø)`
sei-db/config/ss_config.go	`100.00% <ø> (ø)`
sei-db/state_db/sc/composite/store.go	`65.59% <100.00%> (ø)`
sei-db/state_db/sc/memiavl/store.go	`92.59% <100.00%> (ø)`
sei-db/state_db/ss/composite/store.go	`72.63% <100.00%> (+1.35%)`	⬆️
sei-tendermint/config/config.go	`70.62% <100.00%> (+0.46%)`	⬆️
sei-tendermint/config/db.go	`100.00% <100.00%> (ø)`
...endermint/cmd/tendermint/commands/reindex_event.go	`74.84% <75.00%> (+0.31%)`	⬆️
sei-tendermint/cmd/tendermint/commands/reset.go	`60.62% <80.00%> (+15.44%)`	⬆️

... and 118 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

blindchaser · 2026-04-01T14:57:38Z

sei-tendermint/config/db.go

+	return filepath.Join(baseDir, subDir)
+}
+
+func pathExists(path string) bool {


dup func pathExists

You mean this one and the one in sei-db? I think this is fine we don't wanna couple sei-tendemrint and sei-db. Tendermint manages their own db separately than sei-db

blindchaser · 2026-04-01T14:59:17Z

sei-tendermint/cmd/tendermint/commands/reindex_event.go

-	if !os.FileExists(filepath.Join(cfg.DBDir(), "blockstore.db")) {
+	blockstoreDir := tmcfg.ResolveDBDir("blockstore", cfg.DBDir())
+	if !os.FileExists(filepath.Join(blockstoreDir, "blockstore.db")) {
 		return nil, nil, fmt.Errorf("no blockstore found in %v", cfg.DBDir())


the error msg seems to should be against blockstoreDir instead of cfg.DBDir()

good catch!

blindchaser · 2026-04-01T14:59:56Z

sei-tendermint/cmd/tendermint/commands/reindex_event.go

-		return nil, nil, fmt.Errorf("no blockstore found in %v", cfg.DBDir())
+	stateDir := tmcfg.ResolveDBDir("state", cfg.DBDir())
+	if !os.FileExists(filepath.Join(stateDir, "state.db")) {
+		return nil, nil, fmt.Errorf("no state store found in %v", cfg.DBDir())


same as above, error msg against stateDir instead of cfg.DBDir()

good catch!

blindchaser · 2026-04-01T15:12:04Z

sei-tendermint/cmd/tendermint/commands/reset.go

-	}
+// ResetState removes all blocks, tendermint state, indexed transactions and evidence.
+// It handles both the legacy flat layout and the new subdirectory layout.
+func ResetState(dbDir string) error {


consider updating reset related unit testings

blindchaser · 2026-04-01T15:15:50Z

app/receipt_store_config.go

 	}
 	if receiptConfig.DBDirectory == "" {
-		receiptConfig.DBDirectory = filepath.Join(homePath, "data", "receipt.db")
+		receiptConfig.DBDirectory = utils.GetReceiptStorePath(homePath)


readReceiptStoreConfig() was changed to default to data/ledger/receipt.db, but the emitted config template still says /data/receipt.db

need to update the related toml.go

* main: plt-228 fixed static check on app and evmrpc package (#3154) flatkv cache (#3027) Make cryptosim state store backend configurable + No Op Wrapper + Read Disable Config (#3145) Add warning message for IAVL deprecation (#3159) Change default min valid per window to zero (#3157) support for starting autobahn from non-zero global block (#3136) Fix upgrade list comparison to respect semver (#3153)

masih

No blockers 🚀

masih · 2026-04-02T09:10:20Z

sei-db/common/utils/path.go

 func GetStateStorePath(homePath string, backend string) string {
-	return filepath.Join(homePath, "data", backend)
+	legacyPath := filepath.Join(homePath, "data", backend)
+	if PathExists(legacyPath) {


Minor: Is there ever a case where an abrupt node failure could cause the creation of an empty directory here?

If so, for robustness i recommend checking if this dir is empty and proceed with legacy path iff it is not empty.

masih · 2026-04-02T09:12:49Z

sei-db/common/utils/path.go

-	return filepath.Join(homePath, "data", "committer.db")
+// PathExists returns true if the given path exists on disk.
+func PathExists(path string) bool {
+	_, err := os.Stat(path)


Harden this by requiring the type of path? e.g. directory? file etc.

This is racy in that it doesn't atomically check and it repeatedly checks but hopefully the probability of race is low enough that we can accept the race condition as negligible.

cody-littley

Does this PR handle migration (i.e. moving memiavl data from original location to new location)?

Restructure sei data folder for Giga

7c6ac8f

yzang2019 requested review from Kbhat1, blindchaser, cody-littley and masih March 31, 2026 23:45

yzang2019 added the non-app-hash-breaking label Mar 31, 2026

yzang2019 requested a review from jewei1997 March 31, 2026 23:45

Fix unit test

88daf32

blindchaser reviewed Apr 1, 2026

View reviewed changes

Change method name

0df74b4

blindchaser reviewed Apr 1, 2026

View reviewed changes

blindchaser approved these changes Apr 1, 2026

View reviewed changes

Kbhat1 approved these changes Apr 1, 2026

View reviewed changes

yzang2019 and others added 3 commits April 1, 2026 13:07

Fix path

23d7a91

Merge branch 'main' into yzang/restruct-data-folder

b4d61bb

masih approved these changes Apr 2, 2026

View reviewed changes

cody-littley approved these changes Apr 2, 2026

View reviewed changes

jewei1997 approved these changes Apr 2, 2026

View reviewed changes

Conversation

yzang2019 commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

masih left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cody-littley left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

yzang2019 commented Mar 31, 2026 •

edited

Loading

github-actions bot commented Mar 31, 2026 •

edited

Loading

codecov bot commented Mar 31, 2026 •

edited

Loading