Skip to content

feat(ratelimit): in-memory token bucket for pure-Pebble path#592

Merged
osvaldoandrade merged 1 commit into
mainfrom
feat/inmemory-ratelimiter
May 18, 2026
Merged

feat(ratelimit): in-memory token bucket for pure-Pebble path#592
osvaldoandrade merged 1 commit into
mainfrom
feat/inmemory-ratelimiter

Conversation

@osvaldoandrade

Copy link
Copy Markdown
Owner

Summary

The pure-Pebble path used a `noopLimiter` that allowed every request even when rate limits were configured. Silent foot-gun: operators saw the YAML knobs (`producer.requestsPerMinute`, `burstSize`, …) and trusted them; runtime ignored them.

This PR ships an in-process token-bucket limiter for the pure-Pebble path. Configuration now actually takes effect.

Design

  • Per-node enforcement: in a cluster of N codeq nodes, the effective per-(scope, subject) ceiling is N × `requestsPerMinute`. No external coordinator; state is in-process memory.
  • Trade-off: dragging a Redis dep into the embedded path would defeat the purpose of "100% Pebble". Per-node enforcement is the right knob — legitimate clients don't get rebalanced across nodes mid-burst, and the alternative (cluster-wide via raft) would add a write per Allow which is much worse than the limit it's protecting.
  • Concurrency: top-level RWMutex protects the buckets map; each bucket has its own `sync.Mutex` for refill + decrement. Typical workloads with one bucket per JWT subject never contend.
  • GC: janitor reaps buckets idle > 10 min so the map stays bounded under per-token churn.

What landed

  • `internal/ratelimit/inmemory.go`: `InMemoryLimiter` with the standard `Limiter` interface.
  • `pkg/app/application.go`: when `cfg.PersistenceProvider=="pebble"` wire `NewInMemoryLimiter()` instead of the noop.
  • `docs/14-configuration.md` "Rate limiting" section corrected — the "no-op in pure-Pebble" claim was wrong post-merge.

Test plan

  • 8 unit tests for InMemoryLimiter (disabled bucket, burst → block → refill, isolation per subject + per scope, capacity reconfig, concurrent stress, janitor)
  • `go test ./pkg/app` passes — no regressions wiring the new limiter into the Application
  • Manual: set `rateLimit.producer.requestsPerMinute: 6` + `burstSize: 3` on a single-node pebble deploy; verify the 4th request in a burst returns 429 with `Retry-After`

🤖 Generated with Claude Code

The pure-Pebble path used a noopLimiter that allowed every request
even when rate limits were configured. Silent foot-gun: operators
saw the config knobs and trusted them; runtime ignored them.

This commit ships an in-process token-bucket limiter for the
pure-Pebble path. Each (scope, subject) pair gets its own bucket;
refill uses real time, so the math matches the existing Redis Lua
implementation. No external coordinator means per-NODE enforcement
(cluster ceiling = N × bucket.RequestsPerMinute) — accepting that
trade-off avoids dragging a Redis dep into the embedded path.

Implementation:
- internal/ratelimit/inmemory.go: InMemoryLimiter with sync.RWMutex
  over a buckets map; per-bucket sync.Mutex for the refill+decrement
  critical section. Janitor reaps buckets idle > 10 min so the map
  stays bounded under per-token churn.
- pkg/app/application.go: when cfg.PersistenceProvider=="pebble",
  wire NewInMemoryLimiter() instead of the noopLimiter placeholder.

Tests (8 passing):
- Disabled bucket allows everything
- BurstSize=3 + 4th call → blocked with RetryAfter ≥ 1s
- After advancing the clock, tokens refill and allow again
- Different subjects have independent buckets; different scopes
  with the same subject also independent
- Capacity reconfigure clamps existing token count downward
- 32 goroutines × 100 calls on one shared bucket: allowed ≤ burst+slop
- Janitor: bucket idle 15 min → GC'd

Docs: 14-configuration.md "Rate limiting" section updated; the
"no-op in pure-Pebble" claim was wrong post-commit.
@osvaldoandrade osvaldoandrade merged commit 7766159 into main May 18, 2026
@osvaldoandrade osvaldoandrade deleted the feat/inmemory-ratelimiter branch May 18, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant