Skip to content

Commit edf2faf

Browse files
lpalbouclaude
andcommitted
2.26.2: transformers prompt cache crops with the negative form (transformers 5.20)
transformers 5.20 removed Cache.crop(n) with a positive n (keep n tokens, deprecated since 5.x); the call failed, _transformers_crop_cache returned False, and every warm full-context call rebuilt the cache from scratch (test_full_context_warm_call_feeds_only_the_suffix: 249 then 258 tokens fed). Crop with keep - current (drop that many tokens), which every supported transformers version accepts; the test fake follows the real semantics. Passes on transformers 5.17.0, 5.19.0 and 5.20.1. The 2.26.1 release run failed on this test (nothing was published), so the Windows fixes of 2.26.1 ship in 2.26.2. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent e600f01 commit edf2faf

5 files changed

Lines changed: 30 additions & 4 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,18 @@ All notable changes to AbstractCore will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [2.26.2] - 2026-10-18
9+
10+
2.26.1 was not published on PyPI; its Windows fixes ship in this release.
11+
12+
### Fixed
13+
14+
- **Hugging Face (transformers) prompt cache with transformers 5.20 or newer**: a warm call
15+
feeds only the new part of the conversation again. transformers 5.20 removed `Cache.crop`
16+
with a positive length; the cache is now cropped with the negative form (drop that many
17+
tokens), which every supported transformers version accepts. With 5.20, every warm call
18+
re-read the whole conversation (correct output, no prefill savings).
19+
820
## [2.26.1] - 2026-10-17
921

1022
Windows fixes.

‎abstractcore/providers/huggingface_provider.py‎

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4531,8 +4531,19 @@ def _transformers_crop_cache(self, state: _TransformersPromptCacheValue, keep_to
45314531
)
45324532
return False
45334533

4534+
# `crop(n)` with a positive `n` (keep n tokens) was deprecated and is gone in
4535+
# transformers 5.20; a negative `n` (drop |n| tokens) works on every version.
4536+
keep = int(keep_tokens)
45344537
try:
4535-
crop(int(keep_tokens))
4538+
lengths = self._transformers_layer_seq_lengths(cache)
4539+
except Exception:
4540+
lengths = []
4541+
current = max(lengths) if lengths else None
4542+
try:
4543+
if current is None:
4544+
crop(keep) # length unknown: the legacy form is the only one available
4545+
elif current > keep:
4546+
crop(keep - current)
45364547
except Exception:
45374548
return False
45384549

‎abstractcore/utils/version.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,4 +10,4 @@
1010
"""
1111

1212
# Package version - update this when releasing new versions
13-
__version__ = "2.26.1"
13+
__version__ = "2.26.2"

‎llms-full.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
> Expanded companion to `llms.txt`: setup, API and provider guidance, followed by canonical documentation pages. Use the linked source pages for authoritative documentation and the source code for behavior.
44

55
Last updated: 2026-09-30
6-
Package version: 2.26.1
6+
Package version: 2.26.2
77
Supported Python: 3.9+; GitHub CI tests 3.9, 3.10, 3.11, 3.12, and 3.13.
88

99
## How to use this file (agents)

‎tests/huggingface/test_transformers_cached_full_context_delta.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -236,7 +236,10 @@ class _ExactCropCache:
236236
def __init__(self, length):
237237
self._len = length
238238
def crop(self, n):
239-
self._len = min(self._len, int(n))
239+
# transformers' semantics: a negative n drops |n| tokens (the only form
240+
# transformers >= 5.20 accepts); a positive n keeps n tokens (older ones).
241+
n = int(n)
242+
self._len = max(0, self._len + n) if n < 0 else min(self._len, n)
240243
def get_seq_length(self):
241244
return self._len
242245

0 commit comments

Comments
 (0)