Repository navigation
Capture measurements in CSV based on time intervals #467
Description
- CodeCarbon version: 2.2.2
- Python version: 3.11.5
- Operating System: Linux-6.2.0-35-generic-x86_64-with-glibc2.35
Description
I have implemented a method that enables the capture and storage of consumption logs (CSV format) in parallel every second during the execution of my algorithm. Nevertheless, I would be interested in exploring whether there are more efficient alternatives from a technical or academic standpoint to accomplish this process. Below, I am providing the code of my current solution for review and suggestions.
What I Did
from codecarbon import EmissionsTracker
import threading
pthread=1
def fthread():
while pthread:
tracker.flush()
time.sleep(1)
# Creates an instance of a thread and passes the function is executed.
eng_measure = threading.Thread(target=fthread)
# Training
tracker = EmissionsTracker(project_name="Test",tracking_mode="machine")
tracker.start()
eng_measure.start()
try:
# code
finally:
pthread = 0
eng_measure.join()
emissions = tracker.stop()
Activity
benoit-cty commentedon Nov 5, 2023
Hello,
Doing it every second could be quite disk intensive, and useless as CodeCarbon only update measure every 15 seconds.
For a better solution we could update CodeCarbon to write to a csv file in the same place we call the API and Prometheus : https://github.com/mlco2/codecarbon/blob/master/codecarbon/emissions_tracker.py#L703
So you will just need to do:
tracker = EmissionsTracker(project_name="Test",tracking_mode="machine", csv_run_name="emissions_test54.csv")And CodeCarbon will update the CSV every 8 x 15 seconds (api_call_interval * measure_power_secs).
What do you think of this ?
Maybe CodeCarbon could generate a run_id to prevent the need of giving the file name.
seriab commentedon Nov 11, 2023
Thank you very much for your response. The proposed update would be very useful. I will be looking forward to your reply.
NerfedAce commentedon May 19, 2026
Hi! I noticed the previous PRs related to this issue were closed.
I’d be interested in working on this feature if it’s still needed. I’ll first review the previous implementations and maintainer feedback before opening a PR.
harsith1410 commentedon Jun 1, 2026
Hi @benoit-cty,
I'd love to take a shot at implementing this feature if it's still open!
I've reviewed your feedback regarding the disk-intensive nature of 1-second flushes and your blueprint for linking the CSV write to the existing API/Prometheus data loop inside
codecarbon/emissions_tracker.py(running everyapi_call_interval * measure_power_secs). I also plan to look into incorporating the auto-generatedrun_idfor file naming as you suggested.Since the previous PRs were closed, I'm happy to dive in, set it up locally, and work on a clean implementation. Could you please assign this to me? Thanks!
sankalpsthakur commentedon Jul 22, 2026
I'd like to work on this — planning a minimal change so CSV measurement export can capture rows on a configurable time interval, with tests.
sankalpsthakur commentedon Jul 22, 2026
Opened PR with the maintainer-suggested approach (interval CSV write on the API/Prometheus live cadence via opt-in
csv_run_name): #1270Add opt-in interval CSV measurement export
Add opt-in interval CSV measurement export
Add opt-in interval CSV measurement export
Add opt-in interval CSV measurement export
Snehavijayakumara commentedon Sep 25, 2026
Hi! I would like to work on this issue as my first contribution to codecarbon.
I am interested in implementing a cleaner way to capture and store measurements in CSV at configurable time intervals.
Before I start working on it, could you please confirm whether this issue is still relevant and whether there is a preferred implementation approach?