Understanding power measurement better

[Jones-S]

Your question

Hi
I am currently trying to get a more or less accurate power consumption value from browsertime.
Following this example: #3944 (comment)
I get the initial info in my terminal:
INFO: CPU / Power usage: 7334 which I guess is in µWh.

If I open the browsertime.json I see this:
"powerConsumption":[0.003353606209]
and "powerConsumption":{"median":0.0034, ...

Now the numbers are pretty close to each other. But what is the difference? I also had the case that they differed quite a bit.
The first [0].info.powerConsumption and the second is [0].info.statistics.powerConsumption.

Also I don't understand the key cpu which is the number from before 7334.
So is this just the Power used by the CPU? What about GPU for example?
Is GPU taken into account in one of the powerConsumption keys?

If I then open the saved profile (attached) I see in the parent process → process power: 2.1mWh in the full range.
How is that value linked to the powerConsumption in the browsertime summary?

In the process github.com itself I see 967 µWh for the full range.

I also wonder: Which number would go up if I had power intensive tasks running in the background on my machine?
Would all be affected?

Maybe somebody could help me out to understand which of the values would be most accurate number to measure if I want to find out about the power consumption of one page.

🙏

geckoProfile-1.json.gz

browsertime.json

[soulgalore]

Hi @Jones-S !

For the actual number and what Firefox exactly measures I think Nazim from Mozilla has better knowledge or could point you in the right direction. I think the goal is to measure the consumption used when accessing the page, but I'm not 100% sure what's more included and how precise it is.

For the metrics in the JSON file: The info.powerConsumption should be an array of each run. So for example if you run browsertime with 5 iterations, you should have five numbers there. The statistics use https://www.npmjs.com/package/fast-stats to calculate median, different percentiles etc (useful when you do many runs).

[Jones-S]

@soulgalore
Thanks for your quick reply!
I did get that with the powerConsumption Array, but do you know the difference between the two?
It would also help me which numbers you take from the profile json to mix them into the browsertime.json calculation.
I was not able to find the numbers or even keys in the geckoProfile-1.json.
That would be of great help if I could match the numbers I see in my profiler interface with the summarized numbers of the browsertime.json.

🙏

[fqueze]

I don't know how the numbers stored in browsertime.json are computed. It's likely a subset of the profile, based on a start time and stop time for an interesting range (maybe between the page load was requested, and finished?).

So is this just the Power used by the CPU? What about GPU for example?

On Apple Silicon hardware we don't have the CPU vs GPU breakdown. We have it on Intel CPUs.

If I then open the saved profile (attached) I see in the parent process → process power: 2.1mWh in the full range. How is that value linked to the powerConsumption in the browsertime summary?

In the process github.com itself I see 967 µWh for the full range.

I also see these 2 numbers in the profile.

I also wonder: Which number would go up if I had power intensive tasks running in the background on my machine? Would all be affected?

Hopefully neither, because you are power profiling on an Apple Silicon machine where we have power information per process.

Maybe somebody could help me out to understand which of the values would be most accurate number to measure if I want to find out about the power consumption of one page.

Parent process + content process is a reasonable thing to look at on Mac. One thing you need to pay attention to is if there is unrelated CPU activity in the profile. In your profile in the parent process I see about 750ms of activity in the parent process that might not be related to your page load (might be the browser finishing font loading from the OS a few seconds after startup is over).

geckoProfile-1.json.gz

It's easier to look at your profile if you upload it from the profiler UI and paste a share.firefox.dev link.

[Jones-S]

thanks for the hints. this already helps a lot.
Here is the profiler link: https://share.firefox.dev/4pqMrsY

[canova]

The powerConsumption values in browsertime json is coming from here:
https://github.com/sitespeedio/browsertime/blob/88501b651908459fdc7f7a3490bc929f7d67bd6d/lib/firefox/geckoProfiler.js#L267
and
https://github.com/sitespeedio/browsertime/blob/88501b651908459fdc7f7a3490bc929f7d67bd6d/lib/firefox/geckoProfiler.js#L44-L72

So it sums up all the power values in the whole "profile". If you look at it in the Firefox Profiler UI, you'll see the power consumption of that process only. So that's the main difference between the values coming from the profiler UI vs browsertime.

If you sum up all these values and if they don't match, then it's probably a bug and we should fix it. You can write a function that sums up all the power values in the DevTools console.

When I was working on this, I wanted to report the whole Firefox instance since when we use browsertime, the only that that's open is the tab that we created. And reporting the whole Firefox made sense at that time. I'm not too attached to it, but I assume that there is going to be some idle content processes that increase that number.

---

About the INFO: CPU / Power usage: 7334 thing,
Apparently it gets the total CPU time from Glean, and it's CPU time in ms:
https://github.com/sitespeedio/browsertime/blob/88501b651908459fdc7f7a3490bc929f7d67bd6d/lib/firefox/webdriver/firefox.js#L265-L274

Which is not a power usage value at all. I think it's very misleading, and we should either remove it completely or rename it better so it's not confusing to people. (I don't know why that was added initially, maybe for perf testing inside mozilla?) @fqueze might know more.