Skip to content

Improve session adjacent turn recall - #345

Draft
okJiang wants to merge 5 commits into
mainfrom
codex/locomo-20260603T0050sessenum-session-enum
Draft

Improve session adjacent turn recall#345
okJiang wants to merge 5 commits into
mainfrom
codex/locomo-20260603T0050sessenum-session-enum

Conversation

@okJiang

@okJiang okJiang commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Improve session adjacent-turn recall by looking up raw session turns in bounded session_id + seq windows around already retrieved session memories.
  • Preserve the existing fallback path for backends that do not support sequence-window lookup.
  • Add repository/service/handler tests for the new adjacent-turn expansion behavior.

LoCoMo evidence

  • Baseline run: 20260602T104638freshbase2
    • Result: /Users/jiangxianjie/code/okjiang/harness/mem9-locomo/20260602T104638freshbase2/benchmark-results.json
    • Overall LLM (micro): 0.6448051948
    • Product HEAD used by baseline: be522490716e2e16d961a89936a6c81e96a7a39f
  • Candidate run: 20260603T0155sessfull
    • Result: /Users/jiangxianjie/code/okjiang/harness/mem9-locomo/20260603T0155sessfull/benchmark-results.json
    • Overall LLM (micro): 0.6532467532
    • Absolute delta: +0.0084415584
    • Candidate commit: 9908dbb17c771efed6b4bd1493dbe612f8d9f00f
  • Benchmark revision: mem9-benchmark 69d0d9c8fe665259bd3234fe9d16a256af266d34
  • Dataset: /Users/jiangxianjie/code/mem9-locomo-workspace/mem9-benchmark/locomo/data/locomo10.json
  • Model/config: qwen3.6-plus, session ingest, retrieval limit 10, no sample/category filters, 1540/1540 LLM-judged rows, 1986 total rows.

Category LLM movement:

Category Baseline Candidate Delta
Cat1 0.3829787234 0.3794326241 -0.0035460993
Cat2 0.7663551402 0.7538940810 -0.0124610592
Cat3 0.3750000000 0.3958333333 +0.0208333333
Cat4 0.7170035672 0.7360285375 +0.0190249703

Cat1/Cat2 regressions were reviewed and accepted as non-blocking for this draft: Cat1 is a net -1 LLM hit, Cat2 is a net -4 LLM hits, while the primary metric improves by +13/1540 judged rows. Cat2 temporal/entity rows are recorded as the next follow-up risk.

Local checks

gofmt -l server/internal/domain/types.go server/internal/handler/memory_test.go server/internal/repository/factory.go server/internal/repository/repository.go server/internal/repository/tidb/sessions.go server/internal/repository/tidb/sessions_test.go server/internal/service/session.go server/internal/service/session_test.go
git diff --check
go test ./internal/service ./internal/repository/tidb ./internal/handler
go build ./cmd/mnemo-server
rm -f server/mnemo-server

All checks passed in the candidate worktree before opening this draft PR.

Benchmark integrity

The benchmark dataset, gold answers, judge prompts, judge model, scoring logic, category labels, and benchmark semantics were not changed. The candidate was evaluated with the checked-in harness runner ./locomo.sh --sample-concurrency 5 --evaluation-concurrency 8 --sample-retries 2 using the same benchmark configuration as the baseline.

@okJiang okJiang added the codex label Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant