Skip to content

Wait for backend migrations before starting stats and the indexers - #3

Merged
Evangel90 merged 1 commit into
mainfrom
stats-startup-race
Sep 30, 2026
Merged

Evangel90 merged 1 commit into
mainfrom
stats-startup-race

Conversation

@Evangel90

@Evangel90 Evangel90 commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Summary

On a fresh stack the explorer home page showed NaN for daily transactions (and the 24h stats) for up to a day.

Cause: stats, user-ops-indexer and nft-media-handler depended on the backend with condition: service_started, which doesn't wait for its database migrations. stats ran its first update against an empty database (relation "blocks" does not exist), and after a failure it scheduled the daily groups about 23 h out (83514s). The frontend renders the resulting null as NaN. The other two services hit the same race as restart loops (4 and 1 restarts).

Fix:

  • The backend gets a healthcheck: curl -fsS http://localhost:4000/api/health/liveness. The API only answers after create_and_migrate finishes in command. Liveness, not readiness, because readiness also judges indexing lag and would flap on a busy chain. start_period: 5m gives a fresh database time to migrate.
  • The three dependants use condition: service_healthy.

This affects every network, devnet-1's first day included. Separate from #2 (devnet env); no conflict expected.

Test plan

  • Reproduced on the local stack (.env.local, dev chain 56670): stats logs full of relation "blocks" does not exist, daily groups rescheduled ~23 h, home page NaN
  • With the fix, down -v + up -d from a clean database: backend healthy, stats hit 0 database errors, 0 restarts across all containers, and /api/v1/pages/main / pages/transactions populated on the first run (yesterday_transactions 0, transactions_24h 26, 30-point daily chart)
  • docker compose --env-file .env.local config -q
  • CI green

🤖 Generated with Claude Code

stats started alongside the backend, ran its first update against an
empty database ("relation blocks does not exist") and rescheduled the
daily groups ~23 h out, so the home page's daily figures read NaN for a
day after every fresh start. user-ops-indexer and the media worker hit
the same race as restart loops. The backend now has a healthcheck on
/api/health/liveness, which answers only after create_and_migrate, and
the three dependants wait for service_healthy. Reproduced and verified on
a clean local stack 2026-09-30: no database errors, no restarts, stats
populated on first run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Evangel90
Evangel90 merged commit 3ba5344 into main Sep 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant