Repository navigation
[Draft] Integrate ImpactIA methodology - #259
Open
benoit-cty wants to merge 1 commit into
Open
benoit-cty wants to merge 1 commit into
benoit-cty wants to merge 1 commit into
Conversation
Prompt for Fable 5.1: I want you to look at CODECARBON/ImpactIA and carefully improve EcoLogits to integrate the work of ImpactIA into EcoLogits.
Author
|
ccusage measure: |
Member
|
Hey @benoit-cty, thanks I think I'd rather meet them and discuss some parts of the methodology before reviewing this PR! |
|
Thanks for this detailed pull request and all the work behind. A first few things on what you explained :
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Claiude Fable 5.1 did
The ImpactIA work is now integrated on the
feat/impactiabranch (uncommitted, 27 files). Lint passes, the docs build, and the suite is at 211 passed. The 4 failures intests/test_openai.py(Responses API) fail identically on the untouched branch: the recorded cassette lackscache_write_tokens, which the current OpenAI SDK requires, so they are unrelated.Core inference model, on by default (llm.py):
New experimental
trainingphase (training.py): final run, R&D ×4, dataset storage, allocated per output token over the model's lifetime tokens. It requires a model release date (added to 146 models inmodels.json) and a provider compute capacity (Anthropic, Google, Mistral, OpenAI). Otherwise a newtraining-not-modeledwarning is emitted. It is reported separately and not added to the totals, so existing numbers only shift by the network and building additions.Also fixed a real bug on the way:
RangeValue.__rtruediv__computed the wrong interval.Docs: new llm_training.md, updated llm_inference.md, tutorial, warnings page, nav, and a blog post whose byline is a placeholder (
samuelrince).Something to report back to the ImpactIA team
Their spreadsheet has a unit inconsistency in cell
Model | Training share of GWP | Energy change, 100k input tokens -- | -- | -- gpt-4.1-mini | 0.19% | +5.9% claude-opus-4-6 | 0.04% | +15.2% mistral-large-latest | 7.4% | +4.0% gemini-2.5-flash | 0.01% | +10.3%Calcul!N115: provider compute capacity is stored in kW but multiplied by FLOPS per W without the ×1000. This divides the estimated lifetime token volume by 1000 and is what produces their headline "training = 13% median". I implemented the guide's method with consistent units, so the share EcoLogits reports is much smaller:The token-volume estimate remains the weak point of the method in both directions, which the methodology page states plainly. All parameters can be overridden, including
model_lifetime_output_token_count.Deliberately left out
Points needing your decision
claude-opus-4-7, some Gemini 3.1 previews, rolling-latestaliases), so they get the warning instead of a training estimate.