Spike: run Ghost on Postgres

[jonatansberg]

Spike exploring what a Postgres-only Ghost would take, prompted by Ghost-CLI being deprecated in favour of a Docker-based setup (which removes the "self-hosters need the zero-dependency SQLite install" constraint).

Not for merging. This branch keeps MySQL and SQLite working and adds pg as a third client so the two can be compared. The point is to enumerate what actually breaks and how much of it is structural.

Running it

pnpm dev:postgres

That swaps the MySQL container for Postgres 16 via compose.dev.postgres.yaml. Or, against any Postgres, from ghost/core:

NODE_ENV=development \
database__client=pg \
database__connection__host=127.0.0.1 \
database__connection__port=5432 \
database__connection__user=root \
database__connection__password=root \
database__connection__database=ghost_dev \
node --conditions=source --import=tsx index.js

knex-migrator init runs on first boot: 94 tables, the view, and fixtures, in a few seconds.

What's in here

Driver and plumbing (commit 1)

• pg and pg-query-stream (needed for knex .stream()) added to Ghost core
• Config sanitizer accepts pg/postgres/postgresql and no longer strips connection keys for non-MySQL clients
• connection.js gets a pg branch: SET TIME ZONE 'UTC' per connection, and type parsers so bigint/numeric come back as numbers and naive timestamps parse as UTC (mirrors mysql2's timezone: 'Z' + decimalNumbers)
• schema/commands.js gains Postgres introspection for tables/indexes/columns, creates datetime columns as timestamp (not knex's default timestamptz), and accepts Postgres duplicate/undefined-object codes in the index guards
• A local DatabaseInfo subclass with isPostgres (needs to go upstream to @tryghost/database-info)
• pnpm patch on knex-migrator: create/drop database for pg, pg error codes for "database/relation does not exist", and the internal lock-table migration only swallowed MySQL's multiple-primary-key error
• pnpm patch on @tryghost/bookshelf-pagination: clear('order') on the count query, which Postgres rejects under count(*)

Dialect fixes (commit 2)

A new data/db/sql-helpers.js with isDuplicateEntryError, isForeignKeyError, isUnknownColumnError, rawRows, quoteIdentifier, and groupConcat, used at the 14 sites that previously checked ER_DUP_ENTRY/SQLITE_CONSTRAINT by hand. Then the behavioural differences, each handled only on Postgres unless the portable form is identical:

Postgres behaviour	Where it bit	Fix
FOR UPDATE rejected with DISTINCT (bookshelf emits DISTINCT for eager-loaded relations, and passes options.lock down to them)	Post publish	crud.js locks the target row explicitly; the fetching hook skips forUpdate() on distinct queries
NULLs sort first on DESC	Drafts sorted first under published_at desc; pinned comments	events.js sets knex nulls: first/last on non-raw orderBy; comment ordering made NULL-aware
int / int truncates	Open/click rates in stats, ARR	* 1.0, / 100.0
CAST(x AS CHAR) is char(1)	MRR date	CHAR(10)
Real booleans	subscribed = 1, track_opens = 1, email_disabled = 1	= TRUE
Untyped bound values in CASE id WHEN ? THEN ? END resolve to text	Batched email analytics / event storage updates	ELSE <column> branch gives the CASE the column type
Failed statement aborts the transaction	Bulk insert chunk-then-retry-rows pattern	Each attempt runs in a savepoint
Unquoted identifiers fold to lowercase	PostClicks CTE, maxTimestamp alias	Lowercased / bound with ??
ORDER BY inside a WHERE fragment reused by the count query	Member click events	Moved to autoOrder
LIKE is case-sensitive	Member search	ILIKE on pg (NQL ~ was already case-insensitive via lower())
Backticks are not identifier quotes	Links, automations, slug ordering, post sentiment ordering	Portable quoting via ?? / wrapIdentifier
No GROUP_CONCAT	Members CSV export	string_agg on pg
No CONVERT_TZ / two-arg DATE()	Members-by-date stats	Interval arithmetic branch

Note DATE(col) itself works on Postgres, so the 15 bare DATE() sites are fine.

Verified against Postgres

Seeded members, a paid subscription with MRR events, an email with recipients, link clicks, comments, and a draft post, then checked by hand:

• Newsletter stats: open rate 0.25, click rate 0.5 (both would have been 0 with integer division)
• MRR history 1500 → 1234; ARR 148.08
• Link click counts, member click events, member counts over time
• Member search PAID matches Paid.Member@Example.com
• Drafts last under published_at desc, first under asc; pinned comment first
• Bulk insert with a duplicate inside a transaction: 2 committed, 1 rejected with 23505, transaction still usable
• Post create → publish with authors/tags → edit → frontend render; members CSV export; JSON db export; ~50 admin endpoints return 200

Test status

• Unit suite: green except 7 pre-existing failures (cron day-of-week, gift-preview PNG, email-renderer) that fail identically on clean main
• MySQL integration and legacy suites: fully green
• MySQL e2e-api: admin/members.test.js and members/webhooks.test.js are flaky locally on clean main too (36 failures clean vs 19 on this branch, overlapping sets), so not regressions
• No Postgres-backed test harness yet; test/utils/db-utils.js and db-template.js are MySQL-specific (SET FOREIGN_KEY_CHECKS, SHOW CREATE TABLE, GET_LOCK)

Not done / open questions

• The three external packages (knex-migrator, @tryghost/database-info, @tryghost/bookshelf-pagination) need upstream PRs; the pnpm patches and local subclass are stand-ins
• @tryghost/mongo-knex should get a Postgres test suite; it appears to work but is only exercised via the API here
• Migration path for existing installs: fresh installs only run init/ from schema.js, so the 340 versioned migrations don't need to be dual-dialect if MySQL → Postgres is a data-level export/import
• Test infra, CI, Dockerfile.production, e2e mysql-manager.ts, the Admin "unsupported database" warning, and the dev seeder (LOAD DATA INFILE, SET FOREIGN_KEY_CHECKS) are untouched
• newsletter-email-event-storage.js still string-interpolates ids/timestamps into its CASE statement (pre-existing)

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

It looks like this PR contains a migration 👀
Here's the checklist for reviewing migrations:

General requirements

• ⚠️ Tested performance on staging database servers, as performance on local machines is not comparable to a production environment
• Satisfies idempotency requirement (both up() and down())
• Does not reference models
• Filename is in the correct format (and correctly ordered)
• Targets the next minor version
• All code paths have appropriate log messages
• Uses the correct utils
• Contains a minimal changeset
• Does not mix DDL/DML operations

Schema changes

• Both schema change and related migration have been implemented
• For index changes: has been performance tested for large tables
• For new tables/columns: fields use the appropriate predefined field lengths
• For new tables/columns: field names follow the appropriate conventions
• Does not drop a non-alpha table outside of a major version

Data changes

• Mass updates/inserts are batched appropriately
• Does not loop over large tables/datasets
• Defends against missing or invalid data
• For settings updates: follows the appropriate guidelines