Skip to content

[Draft] Integrate ImpactIA methodology - #259

Open
benoit-cty wants to merge 1 commit into
mainfrom
feat/impactia
Open

benoit-cty wants to merge 1 commit into
mainfrom
feat/impactia

Conversation

@benoit-cty

@benoit-cty benoit-cty commented Sep 20, 2026 •

Copy link
Copy Markdown

What Claiude Fable 5.1 did

The ImpactIA work is now integrated on the feat/impactia branch (uncommitted, 27 files). Lint passes, the docs build, and the suite is at 211 passed. The 4 failures in tests/test_openai.py (Responses API) fail identically on the untouched branch: the recorded cassette lacks cache_write_tokens, which the current OpenAI SDK requires, so they are unrelated.

Core inference model, on by default (llm.py):

  • Input tokens now matter. Pre-fill latency is modeled from the input token count (Agrawal et al. coefficient, scaled by each model's OpenRouter TTFT, as ImpactIA proposes). All seven tracers pass the provider-reported input tokens.
  • Datacenter network equipment (firewall, router, switch): energy plus embodied GWP, allocated like the server.
  • Datacenter building embodied GWP factor per kWh of IT energy.
  • ADPe and PE are not extended for these two items because ImpactIA only provides GWP data; the docs say so.

New experimental training phase (training.py): final run, R&D ×4, dataset storage, allocated per output token over the model's lifetime tokens. It requires a model release date (added to 146 models in models.json) and a provider compute capacity (Anthropic, Google, Mistral, OpenAI). Otherwise a new training-not-modeled warning is emitted. It is reported separately and not added to the totals, so existing numbers only shift by the network and building additions.

Also fixed a real bug on the way: RangeValue.__rtruediv__ computed the wrong interval.

Docs: new llm_training.md, updated llm_inference.md, tutorial, warnings page, nav, and a blog post whose byline is a placeholder (samuelrince).

Something to report back to the ImpactIA team

Their spreadsheet has a unit inconsistency in cell Calcul!N115: provider compute capacity is stored in kW but multiplied by FLOPS per W without the ×1000. This divides the estimated lifetime token volume by 1000 and is what produces their headline "training = 13% median". I implemented the guide's method with consistent units, so the share EcoLogits reports is much smaller:

Model | Training share of GWP | Energy change, 100k input tokens -- | -- | -- gpt-4.1-mini | 0.19% | +5.9% claude-opus-4-6 | 0.04% | +15.2% mistral-large-latest | 7.4% | +4.0% gemini-2.5-flash | 0.01% | +10.3%

The token-volume estimate remains the weak point of the method in both directions, which the methodology page states plainly. All parameters can be overridden, including model_lifetime_output_token_count.

Deliberately left out

  • RAG embedding stage: worth doing as an embeddings methodology, but needs embedding models in the repository and tracer support for embedding endpoints.
  • "Front applicatif": SNCF-specific hosting assumption, out of scope for an API-level library.
  • AWARE-weighted embodied water: a different indicator from WCF in liters, so I did not merge it.
  • ResilioDB embodied factors: kept EcoLogits' sourced values.

Points needing your decision

  • License: ImpactIA's guide and spreadsheet are CC BY-NC-SA. I re-implemented the formulas and wrote original documentation text with attribution, but you may want a legal check before merging into an MPL-2.0 project.
  • Release dates are missing for a few models (claude-opus-4-7, some Gemini 3.1 previews, rolling -latest aliases), so they get the warning instead of a training estimate.
  • Nothing is committed; review the diff and commit when satisfied.

Prompt for Fable 5.1:
I want you to look at CODECARBON/ImpactIA and carefully improve EcoLogits to integrate the work of ImpactIA into EcoLogits.
@benoit-cty

benoit-cty commented Sep 20, 2026 •

Copy link
Copy Markdown
Author

ccusage measure:
| Agent │ Models │ Input │ Output │ Cache Create │ Cache Read │ Total Tokens │ Cost (USD) │
| - Claude │ - fable-5-1 │ 996 │ 144,004 │ 430,255 │ 8,806,508 │ 9,381,763 │ $18.02 │

@samuelrince

Copy link
Copy Markdown
Member

Hey @benoit-cty, thanks I think I'd rather meet them and discuss some parts of the methodology before reviewing this PR!

@benoit-cty benoit-cty changed the title Integrate ImpactIA methodology [Draft] Integrate ImpactIA methodology Sep 20, 2026
@XavierVerne

Copy link
Copy Markdown

Thanks for this detailed pull request and all the work behind. A first few things on what you explained :

  1. Please let us know if there is a blocking condition regarding our licence, so as for us all to find a way. The purpose was to facilitate external integration while preventing people to close further improvments.
  2. thanks for the bug, fixing in progress.
  3. I understood that Samuel and Rosendo and Thibaud are about to talk to each other. May be you could join in ? refer back to Samuel I don't have all infos. This could accelerate things.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants