headless scraping, 3rd installment

[bartveneman]

summary

Setup an endpoint that does headless scraping of a url and give a detailed, actionable overview of what should be improved about your css.

Perf

• skip downloading images and fonts
• use sensible timeouts

Features

• scrape css
• analyze coverage
• • trigger hover/focus states
• • trigger dark/light modes
• • trigger reduced motion/transparency
• • etc etc
• show high level results, link to deeper pages for deeper analysis
• allow scraping directly from homepage
• • use view transition to result page
• • make easy to share url with coworkers

implementation

• Connect to a 3rd party that allows to run playwright fully headless.
• Cover the variants listed above to increase coverage
• Run a tiny, fast crawler to scrape more and relevant pages
• combine coverage reports in the end
• can we do calculated result of css values that consist of one or more variables? Example: syntax website colors use a system of vars but they can be computed at runtime
• Maybe look at visbug to see if we can use it to extract information
• check if all referenced files (font face urls, background imgs) are used and download successfully