sitespeedio reporting LCP element incorrectly in the docker version

[CharlsChacko]

Have you read the documentation?

• Yes, I've read the how to make a reproducable bug guide
• Yes, I've read the how to debug my script guide

URL

https://uk.puma.com/uk/en

What are you trying to accomplish

• Noticed that LCP element is tracked incorrectly while running tests against this page with a video element on viewport (https://uk.puma.com/uk/en) - but only while using the docker version.

• While using the sitespeedio native app, can see the LCP element is identified correctly - which is the hero video on the view port. Attaching results from this test: sitespeedio-test-native-results.zip

• Performance tab on Chrome browser also reports the same as LCP element, as expected.
  

• Whereas the sitespeedio docker version apparently tracks LCP element incorrectly! It seems to be not loading the video element correctly and probably causing the LCP element to be tracked incorrectly.
  Attaching results from this test: sitespeedio-test-docker-results.zip.

• Same config file was used to run the tests on both cases: config_mobile.json

What browser did you use?

Chrome

How to reproduce

- Please run test against URL `https://uk.puma.com/uk/en` using the config file attached above.

- Commands used:
-- native app: 
sitespeed.io --config config_mobile.json https://uk.puma.com/uk/en

-- docker: 
docker run --shm-size=2g --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:37.4.1 --config config_mobile.json https://uk.puma.com/uk/en

Log output

[soulgalore]

Hi @CharlsChacko thank you for creating the issue. So there's no magic in how the LCP element is chosen in sitespeed.io, that happens directly in Chrome/Firefox using the LCP API in the browser. I'll check your result this weekend and if I can see what's going on.

[soulgalore]

Hi, I'm having a hard time debug it because I don't get the video when I run the tests from Sweden. Do you know if there's a way to hack that?

1. I was thinking maybe the Docker container miss out on video codec but when I checked it looks ok (I checked the container). It seems that the container do not show the cover image or the start of the video?
2. Looking at the test without Docker, there's no video/filmstrip so I don't see exactly whats going on (I cannot compare the two).
3. One thing I noticed was that there's one version of difference in Chrome 135 vs 136 between the tests.

I looked at the Crux data for that page (--crux.key YOUR_KEY) to see what elements that's reported back to Google for the users that agreed to send back data. It seems that the image tag is the most common:

[CharlsChacko]

Hello @soulgalore

So the content on the homepage (https://uk.puma.com/uk/en) has moved on unfortunately by the time you tested and the video hero on the page is not there anymore. I will try to find if there is a similar page or another way for you to reproduce this.

[CharlsChacko]

Hello @soulgalore

I can see the a hero video is back on as the LCP element on https://uk.puma.com/uk/en now. As you can see in the screenshot below from performance tab on Chrome, the hero video is the LCP element

So if you try running the test on the docker version of sitespeedio using the config file shared, you will see that it does not detect the LCP element correctly. Below is the results of this test I have run using docker version of sitespeedio.
UK-Homepage-Test-LCP-Incorrect.zip

Please note: I attached the native app test results to show that it detects the LCP element correctly, may be something you can try from your side as well. Sorry, I understand the version etc. are different and not comparable to docker version tests. But wanted to share that as an observation that the issue seems to be on the docker version alone.

[soulgalore]

I'm thinking there's something in your configuration? When I run a test like this:
docker run --rm -v "$(pwd):/sitespeed.io" sitespeedio/sitespeed.io:37.6.0 https://uk.puma.com/uk/en -n 1 --mobile
I get the video tag as LCP:

I see you have kiosk as an argument for Chrome, maybe its that.

[CharlsChacko]

Hello @soulgalore
Thank you for sharing the details. Have analysed the different configurations we have on the config file and was able to find out the config that was causing the issue.

It looks like the video element is not loading when the emulated device is Apple devices - tried out different options on mobileEmulation.deviceName config like 'iPhone 12 Pro', 'iPhone X' and 'iPad Pro'. All these resulted in not loading the video in the test and also detected the LCP element incorrectly. Whereas when I have the config to use devices like 'Samsung Galaxy S8+' or 'Moto G4', it seems to work as expected. And looks like the reason why the command you shared worked as it will use 'Moto G4' by default.

At the moment, the content has moved on again and there are no hero videos on the prod site to replicate the issue. Have sent you a mail with details to replicate the issue. Can you please give it a try when you have some time. Thank you.

[CharlsChacko]

Hello @soulgalore
Did you get a chance to check on this issue? Please let me know.
Thank you.

[soulgalore]

Sorry, let me do that first thing tomorrow, I promise.

[soulgalore]

I tried this morning and I couldn't reproduce? I used the scripts you supplied.

Running with iPhone X setting I get: