Conversation
There was a problem hiding this comment.
Pull request overview
Adds a “Websites I Like” feature backed by Keystatic content entries and a Postgres table that stores per-website OG images/favicons (including scripts to fetch/migrate and a new page to render the list).
Changes:
- Add
websitesKeystatic collection and seed initial website entries incontent/websites/. - Add
website_imagestable + Drizzle schema/repo utilities and a cached DTO for retrieving images as data URIs. - Add scripts to fetch OG images/favicons and migrate existing local images into the DB, plus a new
/websites-i-likepage.
Reviewed changes
Copilot reviewed 40 out of 42 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/migrate-images-to-db.ts | New script to migrate local images into website_images table (with resize/convert via sharp). |
| scripts/fetch-og-images.ts | New script to scrape OG images / fall back to favicon, optimize, and upsert into DB. |
| pnpm-lock.yaml | Locks new dependencies (sharp, js-yaml, @types/js-yaml) and related updates. |
| package.json | Adds npm scripts for the new image fetch/migration workflows; adds deps. |
| keystatic.config.ts | Adds websites collection definition for Keystatic. |
| drizzle/meta/_journal.json | Records a new Drizzle migration entry. |
| drizzle/meta/0003_snapshot.json | New Drizzle snapshot including website_images table. |
| drizzle/0003_marvelous_captain_flint.sql | New migration creating website_images table + unique index. |
| docker-compose.yml | Renames Postgres service and updates DB name/volume path. |
| data/websites.dto.ts | Adds DTOs for listing websites and retrieving cached image map from DB. |
| data/website-images.repo.ts | Adds repository for upserting and reading website images from DB. |
| data/db/schema.ts | Adds website_images table schema including bytea image data. |
| data/cms.ts | Exposes cms.websites.all() for reading website entries. |
| content/websites/timo-mamecke.yaml | Adds website entry. |
| content/websites/stefan-judis.yaml | Adds website entry. |
| content/websites/paul-scanlon.yaml | Adds website entry. |
| content/websites/not-a-number.yaml | Adds website entry. |
| content/websites/nico-espeon.yaml | Adds website entry. |
| content/websites/maxime-heckel.yaml | Adds website entry. |
| content/websites/maggie-appleton.yaml | Adds website entry. |
| content/websites/josh-w-comeau.yaml | Adds website entry. |
| content/websites/jordi-enric.yaml | Adds website entry. |
| content/websites/jahir-fiquitiva.yaml | Adds website entry. |
| content/websites/ineza-bonte.yaml | Adds website entry. |
| content/websites/henry-codes.yaml | Adds website entry. |
| content/websites/hardeeps-iphone-notes.yaml | Adds website entry. |
| content/websites/gwern.yaml | Adds website entry. |
| content/websites/emma-goto.yaml | Adds website entry. |
| content/websites/drew-devault.yaml | Adds website entry. |
| content/websites/dimitrios-lytras.yaml | Adds website entry. |
| content/websites/daniel-sun.yaml | Adds website entry. |
| content/websites/daniel-miessler.yaml | Adds website entry. |
| content/websites/cold-takes.yaml | Adds website entry. |
| content/websites/brittany-chiang.yaml | Adds website entry. |
| content/websites/brian-lovin.yaml | Adds website entry. |
| content/websites/alistair-shepherd.yaml | Adds website entry. |
| content/websites/alexey-guzey.yaml | Adds website entry. |
| content/websites/100-rabbits.yaml | Adds website entry. |
| content/websites/0xdf4.yaml | Adds website entry. |
| app/websites-i-like/page.tsx | New page rendering the websites list and preview image (OG/favicons). |
| app/about/page.mdx | Adds a link pointing to the new “Websites I Like” page. |
| .gitignore | Ignores new Postgres volume directory and the local OG image staging folder. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| import type { WebsiteImagesRepository as Repo } from '../data/website-images.repo' | ||
|
|
||
| loadEnvConfig(process.cwd(), true) | ||
|
|
||
| const CONTENT_DIR = join(process.cwd(), 'content/websites') | ||
|
|
||
| async function getRepo(): Promise<typeof Repo> { | ||
| // Dynamic import after env is loaded so DATABASE_URL is available | ||
| const mod = await import('../data/website-images.repo') | ||
| return mod.WebsiteImagesRepository | ||
| } |
There was a problem hiding this comment.
getRepo is typed as Promise<typeof Repo>, but Repo is imported with import type and therefore has no runtime value; using typeof here will fail TypeScript compilation. Use a value import for WebsiteImagesRepository, or type getRepo as Promise<typeof import('../data/website-images.repo').WebsiteImagesRepository> (or similar) instead.
| import type { WebsiteImagesRepository as Repo } from '../data/website-images.repo' | ||
|
|
||
| loadEnvConfig(process.cwd(), true) | ||
|
|
||
| const IMAGE_DIR = join(process.cwd(), 'public/website-og-images') | ||
|
|
||
| async function getRepo(): Promise<typeof Repo> { | ||
| const mod = await import('../data/website-images.repo') | ||
| return mod.WebsiteImagesRepository | ||
| } |
There was a problem hiding this comment.
getRepo is typed as Promise<typeof Repo>, but Repo is imported with import type so it cannot be used with a typeof type query. This will break pnpm typecheck. Consider typing it via typeof import('../data/website-images.repo').WebsiteImagesRepository or switch to a value import.
| console.log(`Processing: ${data.title} (${data.url})`) | ||
|
|
||
| if (!forceRefetch && (await repo.exists(slug))) { | ||
| console.log(` Skipped (already in DB)`) | ||
| skipped++ | ||
| continue | ||
| } |
There was a problem hiding this comment.
The skip condition checks await repo.exists(slug) without specifying imageType. If a previous run only stored a favicon for a site, subsequent runs will incorrectly skip fetching the OG image forever (unless --force). Consider checking exists(slug, 'og') (or checking both types separately) so the script can upgrade favicon-only records to OG images when available.
| for (const file of yamlFiles) { | ||
| const slug = file.replace(/\.ya?ml$/, '') | ||
| const content = await readFile(join(CONTENT_DIR, file), 'utf-8') | ||
| const data = yaml.load(content) as WebsiteEntry | ||
|
|
||
| console.log(`Processing: ${data.title} (${data.url})`) |
There was a problem hiding this comment.
yaml.load(content) as WebsiteEntry is an unchecked cast; if the YAML is empty/invalid (or missing title/url), data can be null/non-object and data.title will throw at runtime. Add a small runtime validation (e.g., ensure data is an object with string title and url) and count/report the entry as failed when invalid.
| for (const file of imageFiles) { | ||
| const dotIndex = file.indexOf('.') | ||
| const name = file.slice(0, dotIndex) | ||
|
|
There was a problem hiding this comment.
Filename parsing uses file.indexOf('.'), which will truncate slugs when filenames contain multiple dots (e.g. foo.bar.png becomes foo) and behaves oddly if a filename starts with a dot. Prefer lastIndexOf('.') (and guard against -1) so the slug is derived from the full basename reliably.
| "fetch-og-images": "npx tsx scripts/fetch-og-images.ts", | ||
| "migrate-images": "npx tsx scripts/migrate-images-to-db.ts", |
There was a problem hiding this comment.
The new scripts are executed via npx tsx ..., but tsx is not listed in dependencies/devDependencies (and doesn’t appear to be in the lockfile). This makes runs non-reproducible because npx will download whatever the latest tsx is. Add tsx as a devDependency and invoke it via the local binary (e.g. tsx scripts/...).
No description provided.