xapi-lrs

A production-ready, xAPI 1.0.3 and 2.0 conformant Learning Record Store built on Hono + PostgreSQL (or PGlite for zero-dependency local use).

Features

• Full xAPI 1.0.3 and 2.0 compliance (statements, documents, agents, activities), verified in CI against the official ADL conformance suite. Version is negotiated per request via the X-Experience-API-Version header.
• Statement validation per xAPI Data spec sections 2.2-2.6 and 4.0
• Multipart/mixed attachment support
• Server-Sent Events (SSE) for real-time statement streaming
• JWT and Basic Auth (credential-based) authentication
• Admin UI with dashboard, credential management, and statement browser
• OpenTelemetry metrics (Prometheus exporter)
• PostgreSQL with pg_notify for event-driven architecture
• PGlite mode: run with an embedded in-process database — no PostgreSQL required
• lrsql-compatible schema (v0.9.5): the bundled schema is catalog-parity with yetanalytics/lrsql v0.9.5's Postgres shape (CI-enforced — see Taking over an lrsql database), so xapi-lrs can take over a live lrsql database in place

Quick Start

With PostgreSQL

# Start PostgreSQL
docker compose up -d postgres

# Install dependencies
pnpm install

# Apply database schema
pnpm db:migrate

# Start in development mode
pnpm dev

Full stack via Docker Compose

pnpm docker-compose:up    # docker compose up -d (postgres + xapi-lrs)

The postgres service starts with an empty database — it no longer bundles the schema via docker-entrypoint-initdb.d. The xapi-lrs service instead runs with AUTO_MIGRATE=true, so it applies the schema itself on first boot (and is a no-op on subsequent restarts). This is also how you'd point the compose stack at a pre-existing (e.g. lrsql-provisioned) database: swap the postgres service's connection details for the target database's and the same AUTO_MIGRATE boot path performs the takeover — see Taking over an lrsql database below.

With PGlite (no PostgreSQL required)

PGlite embeds a full PostgreSQL engine in-process via WASM. No external database or Docker needed.

pnpm install

# In-memory database (data lost on restart):
DATABASE_DRIVER=pglite pnpm dev

# Persistent database (data survives restarts):
DATABASE_DRIVER=pglite PGLITE_DATA_DIR=./data/pglite pnpm dev

The schema is applied automatically on first start. The admin account is bootstrapped as described in Configuration below.

Limitations of PGlite mode:

• Single connection — concurrent transactions are serialized. Suitable for local development and low-concurrency workloads; not recommended for production.
• SSE uses in-process delivery (db.listen) instead of cross-process LISTEN/NOTIFY — works correctly within a single Node.js process.
• AUTO_MIGRATE and pnpm db:migrate are ignored in PGlite mode (migrations are applied directly from committed SQL files).

The LRS will be available at http://localhost:8081 and the admin server at http://localhost:8091.

Taking over an lrsql database

Because xapi-lrs's bundled schema is catalog-parity with yetanalytics/lrsql v0.9.5's Postgres shape, xapi-lrs can be pointed at a live lrsql database and take it over in place — statements, actors, documents, and credentials carry over unmodified.

1. Point xapi-lrs at the same database. Set PGHOST/PGPORT/PGDATABASE/PGUSER/PGPASSWORD (or DATABASE_URL) to the existing lrsql Postgres instance — no dump/restore needed.
2. Run migrations against it. node dist/migrate.js (or pnpm db:migrate, or boot with AUTO_MIGRATE=true). Against an already-lrsql-shaped database this is a no-op except for adding xapi-lrs's SSE NOTIFY trigger (trg_xapi_statement_stored) — the rest of the schema is already identical.
3. Bootstrap an admin account via env vars. lrsql admin accounts do not port: lrsql hashes passwords with a buddy bcrypt+sha512$... format that xapi-lrs's bcrypt-based passhash check does not (and cannot securely) verify. Existing lrsql admin logins will fail cleanly (401, not a 500) after takeover. Set XAPI_LRS_ADMIN_USER / XAPI_LRS_ADMIN_PASSWORD to bootstrap a fresh xapi-lrs admin account on startup (see Configuration).
4. API credentials DO port. Existing lrsql api_key/secret_key pairs and their scopes are read as-is (lrs_credential / credential_to_scope) — statement traffic authenticated with pre-existing lrsql credentials keeps working immediately after the migration runs, with no re-issuing of keys required.

A startup schema probe runs before the server accepts traffic and fails fast if the connected database's shape doesn't match what this release expects (empty database, legacy pre-0.6 xapi-lrs schema, or anything else unrecognized), rather than surfacing later as an opaque runtime 500 (see src/db-probe.ts).

Breaking change: pre-0.6 xapi-lrs databases are not upgradable. v0.6.0 rewrote the bundled schema to match lrsql v0.9.5's shape byte-for-byte (composite-key credential scopes, Sub*/positional group-member usages, explicit timestamp/stored/registration columns, a new scope vocabulary). Databases created by xapi-lrs pre-0.6 — PGlite data directories or Postgres databases provisioned by the old 000001 migration — use an incompatible shape and cannot be migrated forward; the startup probe detects this and refuses to boot rather than serve against a mismatched schema. Drop and re-provision: for Postgres, drop and recreate the database (or DROP SCHEMA public CASCADE; CREATE SCHEMA public;) and re-run migrations; for PGlite, delete the PGLITE_DATA_DIR directory.

Configuration

All configuration is via environment variables. See .env.test for defaults.

Variable	Default	Description
XAPI_LRS_PORT / PORT	8081	xAPI HTTP port
XAPI_LRS_ADMIN_PORT / ADMIN_PORT	8091	Admin/health/metrics port
DATABASE_DRIVER	pg	Database driver: pg (PostgreSQL) or pglite
PGLITE_DATA_DIR	(none)	PGlite data directory; omit for in-memory
PGHOST.	localhost	PostgreSQL host
PGPORT	5432	PostgreSQL port
PGDATABASE	xapi_lrs	PostgreSQL database
PGUSER	xapi_lrs	PostgreSQL user
PGPASSWORD	(empty)	PostgreSQL password
DATABASE_URL	(none)	Full connection string (overrides PG* vars)
XAPI_LRS_JWT_ISSUER / JWT_ISSUER	(none)	JWT issuer for token validation
XAPI_LRS_JWT_AUDIENCE / JWT_AUDIENCE	(none)	JWT audience for token validation
XAPI_LRS_JWKS_URI / JWKS_URI	(none)	JWKS endpoint URI
XAPI_LRS_OIDC_DISCOVERY_URL / OIDC_DISCOVERY_URL	(none)	OIDC discovery URL (auto-discovers JWKS)
XAPI_LRS_ADMIN_USER	(none)	Bootstrap admin username
XAPI_LRS_ADMIN_PASSWORD	(none)	Bootstrap admin password
XAPI_LRS_ADMIN_SESSION_SECRET / ADMIN_SESSION_SECRET	(random)	Session secret (required in production)
XAPI_LRS_LOG_LEVEL / LOG_LEVEL	info	Log level (silent/fatal/error/warn/info/debug/trace)
XAPI_LRS_CORS_ORIGIN / CORS_ORIGIN	*	CORS allowed origin
XAPI_LRS_STMT_GET_DEFAULT	50	Default GET /statements page size when no limit
XAPI_LRS_STMT_GET_MAX	50	Hard cap on GET /statements limit (silent clamp)
SHUTDOWN_TIMEOUT_MS	30000	Hard deadline for graceful shutdown before exit
PG_STATEMENT_TIMEOUT_MS	30000	Per-statement DB query timeout (0 disables)
PG_IDLE_IN_TRANSACTION_TIMEOUT_MS	60000	Idle-in-transaction connection timeout (0 disables)

Deprecated aliases. Two earlier prefixes are still accepted and log a startup warning: the LRS_* names shipped in 0.6.0 (e.g. LRS_ADMIN_USER, LRS_PORT) and lrsql's own LRSQL_* names (e.g. LRSQL_ADMIN_USER_DEFAULT, LRSQL_STMT_GET_MAX, LRSQL_ALLOW_ALL_ORIGINS, LRSQL_LOG_LEVEL). Each maps to its XAPI_LRS_* or standard equivalent above, which takes precedence when both are set. Prefer the canonical names; the aliases will be removed in a future release.

Health checks

On the admin port (XAPI_LRS_ADMIN_PORT, default 8091):

Path	Purpose	Returns 503 when
/healthz	Liveness probe	(never, unless the process is deadlocked)
/readyz	Readiness probe	shutting down, DB unreachable, or pg_notify listener disconnected
/ready	Deprecated alias for /readyz

On SIGTERM/SIGINT the server flips /readyz to 503, aborts long-lived SSE streams, waits for in-flight HTTP requests, stops the pg_notify listener, drains the DB pool, and exits — with a hard SHUTDOWN_TIMEOUT_MS deadline as a safety net.

Consistent-Through and incremental ingestion

Every GET /xapi/statements response carries X-Experience-API-Consistent-Through. xapi-lrs treats it as a conservative visibility bound, with this guarantee:

Every statement whose stored is at or before the header value is committed and queryable now. No statement at or before it can appear later.

That makes the header safe as an ingestion watermark: read statements up to it, record it, and start the next window there without risking a silently dropped statement.

The guarantee needs more than reporting the current time. A statement's stored is stamped when its INSERT is issued, but the row is invisible until that transaction commits — so a header naively set to now() would vouch for statements no query could yet return, and a consumer advancing its watermark past them would skip them permanently. (The more cursor does not cover this: it guarantees a complete walk of one query's snapshot, not that a not-yet-visible statement ever enters some page.) xapi-lrs therefore stamps stored from the database clock and bounds the header by the oldest open write transaction, so it never advances past a write still in flight.

Two operational caveats:

• Role coverage. The bound reads pg_stat_activity, where PostgreSQL hides other roles' transaction times unless the reader holds pg_read_all_stats. The guarantee therefore covers statements written through the LRS's own database role. If another role also writes to xapi_statement directly, grant the LRS role pg_read_all_stats so those writes are covered too.
• Idle transactions hold it back. A session left idle inside a transaction pins the header at its start time. The header lagging is safe by design — it only delays consumers, never skips statements — but keep PG_IDLE_IN_TRANSACTION_TIMEOUT_MS (default 60s) set so a stuck session cannot stall ingestion indefinitely.

Tracing

xapi-lrs emits OpenTelemetry traces for the xAPI data plane (request + DB query spans) over OTLP. Tracing is off unless an OTLP endpoint is configured — set the standard OTEL_* variables:

Variable	Purpose
OTEL_EXPORTER_OTLP_ENDPOINT / OTEL_EXPORTER_OTLP_TRACES_ENDPOINT	Enable + target (a collector, or a managed backend)
OTEL_EXPORTER_OTLP_HEADERS	Auth headers for a managed backend
OTEL_SERVICE_NAME	Service name (default xapi-lrs)
OTEL_TRACES_SAMPLER / OTEL_TRACES_SAMPLER_ARG	Sampling (default parentbased_always_on)

Point the endpoint at a local OpenTelemetry Collector in small environments, or directly at a managed backend (with OTEL_EXPORTER_OTLP_HEADERS) in production. The default samples every request; for production ingest volume set OTEL_TRACES_SAMPLER=parentbased_traceidratio with OTEL_TRACES_SAMPLER_ARG=0.1 (or 0.01 / lower).

Scripts

Script	Description
pnpm dev	Start with hot reload (node --watch)
pnpm build	Compile TypeScript to dist/
pnpm start	Run compiled output
pnpm test	Run unit tests
pnpm test:integration	Run integration tests (requires PostgreSQL)
pnpm test:conformance	Run ADL conformance suite
pnpm typecheck	Type-check without emitting
pnpm lint	Lint with oxlint
pnpm fmt	Format with oxfmt
pnpm db:migrate	Run database migrations
pnpm docker:build	Build Docker image
pnpm docker-compose:up	Start full stack (postgres + lrs)
pnpm docker-compose:down	Stop the stack

Architecture

src/
  admin/          # Admin UI (htmx + Pico CSS)
  auth/           # JWT verification, credential auth
  helpers/        # Enrichment, ETag, SQUUID utilities
  middleware/     # Authentication & authorization middleware
  repositories/   # PostgreSQL data access (statements, documents, agents)
  routes/         # Hono route handlers (xAPI endpoints)
  sse/            # Server-Sent Events (pg_notify → SSE)
  xapi/           # Statement validator, multipart parser, signature verification
  xapi-types/     # xAPI type definitions
  app.ts          # Hono app factory
  config.ts       # Environment-driven config with Zod validation
  db.ts           # PostgreSQL pool management
  server.ts       # Process entrypoint

Supply Chain

FIPS base image

Published images build on Minimus hardened Node (reg.mini.dev/node-fips), a CVE-reduced base whose FIPS variant runs OpenSSL's FIPS provider. FIPS mode is active at runtime with no flag — crypto.getFips() returns 1 — so the cryptography this LRS performs (SHA-256 attachment digests, SHA-1 ETags, HMAC-SHA-256 admin sessions) runs through a validated module. Images run as uid 1000 and are roughly 200 MB larger than the previous node:slim base.

Requirement: the database must use scram-sha-256 password authentication. This is the PostgreSQL 14+ default, so most deployments need no change. The FIPS provider refuses MD5, and node-postgres computes an MD5 digest to answer an AuthenticationMD5Password challenge — so a server configured for md5 auth fails to connect with Unrecognized algorithm name. md5 authentication is not FIPS-compliant in the first place; if you hit this, migrate the role (ALTER ROLE … PASSWORD … with password_encryption = 'scram-sha-256') rather than downgrading the image.

Two smaller consequences of FIPS mode, neither of which this application triggers today: MD5 throws wherever it is used, so a future dependency that hashes with MD5 will fail at runtime rather than silently degrade; and TLS is restricted to the NIST curves, which can affect outbound connections to endpoints offering only x25519.

Published image tags:

Tag	Points at
latest	the most recent release
<version>	that exact release, e.g. 0.9.5 (immutable)
edge	the current tip of main — unreleased, moves often

Container images published to ghcr.io/pelotech/xapi-lrs are signed with Sigstore cosign (keyless / OIDC) and carry SLSA build provenance attestations. Release images additionally have SPDX and CycloneDX SBOMs attached as Sigstore attestations and as downloadable release artifacts.

Verify an image (substitute the tag):

# Signature
cosign verify ghcr.io/pelotech/xapi-lrs:0.4.0 \
  --certificate-identity-regexp 'https://github.com/pelotech/xapi-lrs/.+' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

# Build provenance
cosign verify-attestation ghcr.io/pelotech/xapi-lrs:0.4.0 \
  --type slsaprovenance \
  --certificate-identity-regexp 'https://github.com/pelotech/xapi-lrs/.+' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

# SBOM (releases only)
cosign verify-attestation ghcr.io/pelotech/xapi-lrs:0.4.0 \
  --type spdxjson \
  --certificate-identity-regexp 'https://github.com/pelotech/xapi-lrs/.+' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

SBOM files are also attached to each GitHub Release as xapi-lrs-<version>-sbom.spdx.json and xapi-lrs-<version>-sbom.cdx.json.

License

Apache 2.0