fix(memory): bound extraction conversation prompts#3230
Closed
huangruiteng wants to merge 1 commit into
Closed
Conversation
Contributor
Author
Collaborator
|
@zhoujh01 支持session设置token阈值自动提交机制,本pr先关闭 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
动机
修复 #3226。当前 session compression / memory extraction 会把累计会话直接拼进一次 VLM 请求,长度没有上限。线上已观测到 52k+ token 的单次请求;它既可能触发 provider 永久失败,也会挤占批处理资源,甚至在激进的 serving 配置下造成 OOM。
改动方案
SessionExtractContextProvider边界增加memory.extraction_prompt_max_chars,默认60000字符,覆盖 v2/v3 compressor 及复用该 provider 的 extraction 路径。ov.conf示例。关键逻辑可概括为:
复现
基于
origin/main@a0e822a0构造一条约 120k 字符的 session message:修复前生成的 conversation user message 为120383字符,随输入线性增长。修复后使用默认配置运行相同输入:
验证
另验证配置默认值、合法自定义值及
<1000的校验拒绝。风险与边界
Fixes #3226