Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 15 additions & 14 deletions .design_docs/optimizer-v1-implementation-plan.md
Original file line number Diff line number Diff line change
Expand Up @@ -270,24 +270,24 @@ Allow a config with labels ⊇ query labels to serve that AQE. This is what enab
- ✅ Plug real `AtomicCosts` values into cost model — done for CMS/HLL/KLL, see 2f above and
"Running with real sketch-bench costs" below.
- ✅ `CountMinSketchWithHeap` wrapper in sketch-bench (`cms-heap` family — sketch-bench PR #104,
closed sketch-bench#95) — but not wired to ASAPQuery's cost lookup yet, see status below.
closed sketch-bench#95) — wired to ASAPQuery's temporary reference cost model below.
- ~~Add cardinality to sweep grids in `sketch-bench`~~ — decided unnecessary: CPU/mem costs for
CMS/HLL/KLL are functions of structural params (depth×width, lg_k, K), not cardinality: only
`CountMinSketchWithHeap` is cardinality-dependent, and that's the analytic-bound case below, not
a sketch-bench sweep axis.

**Status as of 2026-08-28:**
**Status as of 2026-09-03:**
1. `export_atomic_costs.sh` doesn't sweep `cms-heap` yet.
2. Mapping gap: sketch-bench's `CmsHeapParams` is `{rows, cols}` only — `top_k` is a compile-time
constant, Frequency-vs-TopK is two registry variants, not a param. ASAPQuery's grid sweeps
`{depth, width, heapsize, count_events}` — `heapsize` has no home in sketch-bench's data at
all, confirming it must stay analytic. Unresolved: which of the 4 sketch-bench variants
(FastPath/RegularPath × Frequency/TopK) matches what ASAPQuery's `CountMinSketchWithHeap`
accumulator actually deploys, and what `count_events` maps to.
3. Once (2) is resolved: extend `sketch_bench_key()` in `atomic_costs.rs` to translate
`CountMinSketchWithHeap` → the right variant + `{rows,cols}`, for CPU costs only.
4. Analytic memory bound (`heap_size · avg_key_size`) still not implemented — needed regardless
of (1)–(3), memory was always meant to be analytic here, never a lookup.
all, confirming it must stay analytic. Resolved for the current runtime: use
`cms-heap-topk-regularpath-vector2d`; `count_events` is intentionally ignored because it
does not change the structural cost model.
3. ✅ `atomic_costs.rs` translates `CountMinSketchWithHeap` to the selected `{rows,cols}`
reference row and scales CPU costs linearly from the fixed top-k=32 benchmark.
4. Analytic memory bound: runtime CMS counters use 8-byte values, and heap memory uses
explicit average-key-size and per-entry-overhead assumptions.
5. sketch-bench#14 (HLL register-width memory bug) — still open, still un-fixed.
6. asap_sketchlib#69 (`subtract` unimplemented) — still open, `subtract_cpu_secs` absent by design.

Expand Down Expand Up @@ -323,7 +323,8 @@ Takes the same `ControllerConfig` YAML format as `asap-planner --input_config`
(with a `metrics:` hints block for label schema — no live Prometheus needed).
Prints deployed streaming configs and query configs to stdout. `--rho` is the
placeholder arrival rate (see TODOs below — not real yet). `--atomic-costs` is
optional; omit it and every candidate costs at the flat stub, same as before #549.
optional; omit it and ordinary unbenchmarked candidates use the flat stub, while
CMS-with-heap candidates warn and are dropped until a matching reference row is available.

### Running with real sketch-bench costs

Expand All @@ -350,9 +351,9 @@ cargo run -p asap_planner --bin asap-optimizer-cli -- \
--atomic-costs path/to/atomic_costs.json
```

`candidate-gen-dump`'s output labels each params row `[real]` (resolved from the table) or
`[stub]` (fell through to `AtomicCosts::default()` — either an unbenchmarked family, or a
benchmarked family's param point missing from the table).
`candidate-gen-dump`'s output labels each resolved params row `[real]` or `[stub]`; candidates
whose required cost row is missing are shown as `DROPPED`. CMS-with-heap uses its fixed-top-k
reference model when the `{rows,cols}` row is present and is dropped otherwise.

Wire-in decision (deferred): once Phase 3 (MIP + feasibility + label superset
matching) lands, swap `Controller::generate()` to call `run_mip_pipeline()`
Expand All @@ -372,7 +373,7 @@ instead of `generator::generate_plan()`, likely behind an opt-in flag first.
| `translator.rs` | `retention_count_for_assignment(Subtract)` | Returns hardcoded `1`; should be the actual checkpoint count needed to cover the full lookback |
| `promql/generator.rs` | `generate_plan()` doc comment | Flags that `Controller::generate()` still uses the hardcoded path, not the optimizer — see "Offline Testing" section above |
| — | Accuracy constraint | No `Error(a,g) ≤ ε_a` check exists anywhere; nothing stops picking an under-provisioned sketch (Phase 3d) |
| — | sketch-bench | No cardinality sweep, no `CountMinSketchWithHeap` wrapper; `AtomicCosts` are still stub numbers (Phase 3c) |
| — | sketch-bench | `export_atomic_costs.sh` does not yet sweep `cms-heap`; CMS-with-heap currently uses the temporary fixed-top-k reference model and must fail louder once swept coverage exists |

---

Expand Down
Loading
Loading