Repository navigation
Replies: 1 comment
|
I really expected for the module to support different context management technics, it does not seem right for a consumer to implement that, because if one does this, there is like a 100 line code implementation of addition support for OpenAI compatible endpoint, and what is the use case for the module at this point? Had to use |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
It would also be useful to provide a ContextTooLargeError, with error translation for anthropic, google, and openai - and compatible.
Why
I am primarily interested in being able to continue conversations in Crush which, for whatever reason, accidentally overflow:
The above is what happens if you overflow + try and compact. To resolve this issue elegantly, we need to understand the error from the API within crush. IMO, the best way tot do that is through error translation. Then crush can recover as it sees fit (perhaps dropping last message which caused the overflow and summarizing).
This would require writing translators for at least anthropic (and compat), Gemini and OpenAI (and compat).
I would be happy to create these! Please let me know.
All reactions