Skip to content

fix(memory): bound extraction conversation prompts#3230

Closed
huangruiteng wants to merge 1 commit into
volcengine:mainfrom
huangruiteng:codex/fix-3226-extraction-prompt-budget
Closed

fix(memory): bound extraction conversation prompts#3230
huangruiteng wants to merge 1 commit into
volcengine:mainfrom
huangruiteng:codex/fix-3226-extraction-prompt-budget

Conversation

@huangruiteng

Copy link
Copy Markdown
Contributor

动机

修复 #3226。当前 session compression / memory extraction 会把累计会话直接拼进一次 VLM 请求,长度没有上限。线上已观测到 52k+ token 的单次请求;它既可能触发 provider 永久失败,也会挤占批处理资源,甚至在激进的 serving 配置下造成 OOM。

改动方案

  • 在共享的 SessionExtractContextProvider 边界增加 memory.extraction_prompt_max_chars,默认 60000 字符,覆盖 v2/v3 compressor 及复用该 provider 的 extraction 路径。
  • 未超限时保持原输出不变;超限时优先保留完整的最近消息,并显式插入 omission marker。
  • 单条最新消息自身超限时保留原始 range header、消息开头和结尾,避免把有效索引改写成新的局部编号。
  • 补充中英文配置文档与 ov.conf 示例。

关键逻辑可概括为:

conversation_budget = configured_limit - wrapper_overhead
if formatted_history <= conversation_budget:
    keep history unchanged
elif latest_message > conversation_budget:
    keep latest header + head + omission marker + tail
else:
    keep newest complete messages + omission marker

复现

基于 origin/main@a0e822a0 构造一条约 120k 字符的 session message:修复前生成的 conversation user message 为 120383 字符,随输入线性增长。

修复后使用默认配置运行相同输入:

prompt_chars_after=60000
configured_limit=60000
bounded=true

验证

python -m pytest -s -o addopts='' --confcutdir=tests/session/memory \
  tests/session/memory/test_memory_react_system_prompt.py -q
# 18 passed

ruff check openviking/session/memory/session_extract_context_provider.py \
  openviking_cli/utils/config/memory_config.py \
  tests/session/memory/test_memory_react_system_prompt.py

ruff format --check openviking/session/memory/session_extract_context_provider.py \
  openviking_cli/utils/config/memory_config.py \
  tests/session/memory/test_memory_react_system_prompt.py

另验证配置默认值、合法自定义值及 <1000 的校验拒绝。

风险与边界

  • 当前是确定性的字符预算与 recent-window 策略,不在本 PR 中引入 map-reduce;优点是改动集中、所有共享 extraction 路径立即获得硬上限。后续如需更高保真,可在同一 provider 边界增加分块策略。
  • 预算只覆盖 conversation-history user message;system/schema 与后续 tool follow-up 消息不计入该字段,文档中已明确。
  • 未超限历史的序列化保持不变;超限时 omission marker 会让模型明确知道上下文被裁剪。

Fixes #3226

@huangruiteng

Copy link
Copy Markdown
Contributor Author

@qin-ctx 请协助 review:本 PR 修复 #3226,改动是「fix(memory): bound extraction conversation prompts」。必要性、验证与风险已写在 PR 描述中,谢谢!

@qin-ctx

qin-ctx commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

@zhoujh01 支持session设置token阈值自动提交机制,本pr先关闭

@qin-ctx qin-ctx closed this Jul 14, 2026
@github-project-automation github-project-automation Bot moved this from Backlog to Done in OpenViking project Jul 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

compressor_v2 sends unbounded prompts to the VLM (52k+ tokens observed) — no prompt budget on the session-compression/memory-extraction path

2 participants