Skip to content

feat(queue): support per-queue pg-boss work options - #5

Merged
niranjan94 merged 1 commit into
mainfrom
feat/queue-per-queue-work-options
Jul 14, 2026
Merged

niranjan94 merged 1 commit into
mainfrom
feat/queue-per-queue-work-options

Conversation

@niranjan94

Copy link
Copy Markdown
Owner

Summary

Adds an optional workOptions map to withQueues() so a consumer can tune individual queues (concurrency, batch size, polling) instead of every queue sharing pg-boss defaults. Entries are keyed by queue name and forwarded to boss.work(name, options, handler). Queues omitted from the map keep the current behavior: a single worker fetching one job at a time.

createApp()
  .withQueues(registry, {
    workOptions: {
      concierge: { localConcurrency: 5 }, // 5 independent workers for this queue
    },
  })
  .buildConsumer();

Motivation

Today registerQueueWorkers calls boss.work(name, handler) with no options for every queue, so each queue runs a single serial worker (batchSize: 1). A reactive, high-volume queue whose handler is long-running (e.g. one that launches and polls an external task) can be head-of-line-blocked by a single slow or stuck job: everything behind it waits until that handler returns. Meanwhile low-traffic queues are fine as single workers.

localConcurrency (pg-boss spawns N independent workers per node for that queue) is exactly the right lever, but there was no seam to pass it. This adds one.

Changes

File Change
src/queue/consumer.ts registerQueueWorkers takes an optional workOptions map and forwards a queue's entry to the three-arg boss.work() overload; falls back to the two-arg overload when absent. Adds exported QueueWorkOptions type.
src/builder.ts WithQueueOptions is now generic over the registry and gains a workOptions field (keys checked against queue names); withQueues() accepts it; buildConsumer() forwards it.
src/queue/__tests__/register-queue-workers.test.ts New unit tests: creates each queue, uses the two-arg overload without options, forwards options via the three-arg overload when present, and still dispatches jobs to the handler.
src/queue/__tests__/consumer-builder.test.ts Adds a builder-level test that withQueues(registry, { workOptions }) builds a consumer.

Backward compatibility

Additive and non-breaking. QueueRegistry, QueueJobHandler, and JobDataFor are untouched, so existing withQueues(registry) calls compile and behave exactly as before. The new WithQueueOptions type parameter defaults to QueueRegistry, so existing references to the type keep working.

Testing

  • pnpm build:check (tsc --noEmit): clean
  • pnpm test:unit: 273 passed (21 files)
  • pnpm lint: clean (2 pre-existing warnings in tests/e2e/error-handling.test.ts, unrelated)

Follow-up

A minor release (e.g. 1.16.0) is needed to consume workOptions downstream. Left out of this PR to match the repo's convention of separate chore: bump version commits.

Add an optional `workOptions` map to `withQueues()` so consumers can tune
individual queues (concurrency, batch size, polling) instead of every queue
sharing pg-boss defaults. Entries are keyed by queue name and forwarded to
`boss.work(name, options, handler)`; queues omitted from the map keep the
previous behavior (a single worker fetching one job at a time).

This lets a reactive, high-volume queue run several jobs concurrently (via
`localConcurrency`) so one long-running handler no longer head-of-line-blocks
the rest of that queue, while low-traffic queues stay single-worker.

The change is additive: `QueueRegistry`, `QueueJobHandler`, and `JobDataFor`
are unchanged, so existing `withQueues(registry)` calls compile and behave
exactly as before. `WithQueueOptions` is now generic over the registry so the
`workOptions` keys are checked against the registered queue names.
@vercel

vercel Bot commented Jul 14, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
telaio-docs Ready Ready Preview, Comment Jul 14, 2026 7:07am

@niranjan94
niranjan94 merged commit 8c35cba into main Jul 14, 2026
3 of 4 checks passed
@niranjan94
niranjan94 deleted the feat/queue-per-queue-work-options branch July 14, 2026 07:17

This branch was successfully deployed

1 active deployment
Preview — 996f6174 Deployed Jul 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant