include the pages activeSdk in algolia search results

[NWylynko]

🔎 Previews:

• https://clerk-git-nick-boost-active-sdk-search-results.clerkstage.dev/docs/pr/nick-include-guides-sdk-in-search

What does this solve? What changed?

• Adds the guides active sdk to the algolia search results, allowing search to boost search results that match the users active sdk

Deadline

Other resources

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
clerk-docs	Ready	Preview	Jun 2, 2026 7:55pm

[manovotny]

Beyond the per-variant sdk field, this branch now also codifies the docs index's faceting, ranking, and synonyms in scripts/update-algolia-records.ts, so they're enforced on every index run instead of living as mutable dashboard state.

What's in this PR specifically:

• sdk added to attributesForFaceting (filterOnly) — what makes the optionalFilters: sdk:<active> boost on the client side work at all.
• Ranking reorder enforced in the indexer — attribute/exact above proximity, so title/heading matches beat body-content matches.
• Hybrid synonyms — acronyms auto-derived from the _tooltips glossary + a curated phrasing list (fixes magic link, i18n, DKIM, login, …).
• AGENTS.md: don't tune ranking/synonyms/faceting in the Algolia dashboard — they're codified here and revert on every reindex.

Paired with the client-side active-SDK boost in clerk/clerk#2661. The full data-backed writeup — why boost sdk over availableSDKs, the ranking reorder, the synonym set, and every rejected alternative with measurements — lives there: https://github.com/clerk/clerk/pull/2661#issuecomment-4585030268

[NWylynko]

Suggested change

	await algolia.setSettings({
	indexName: ALGOLIA_INDEX_NAME,
	indexSettings: {
	attributesForFaceting: [
	...attributesForFaceting,
	...missingFacets.map((attribute) => `filterOnly(${attribute})`),
	],
	ranking,
	},
	})
	await algolia.setSettings({
	indexName: ALGOLIA_INDEX_NAME,
	indexSettings: {
	attributesForFaceting: [
	...attributesForFaceting,
	...missingFacets.map((attribute) => `filterOnly(${attribute})`),
	],
	ranking,
	},
	forwardToReplicas: true,
	})

@manovotny should the setSettings also forward the change to replicas. Granted I don't think we are using replicas, but should be consistent I think.

[manovotny]

Good Q — went back and forth on this, landed on: no, setSettings shouldn't forward.

We're codifying these settings (declare + overwrite every run), which is itself the mechanism for keeping indexes consistent. forwardToReplicas is the other model — "tune the primary, propagate it" — so under codification it's redundant: the script configures whatever index it targets directly.

It's also the riskier default here specifically because this setSettings bundles ranking/customRanking. A standard replica usually exists to hold a different sort, and forwardToReplicas: true would overwrite that on every index run. So if we ever add replicas, the right move is to declare them in the script (which can express per-replica settings), not blanket-forward.

The synonyms call does forward, and that's deliberate — synonyms should always be identical across replicas, so it's a safe no-op today and the correct behavior if a replica ever shows up. Added a comment in efe7c94 spelling out the asymmetry so it doesn't read like an oversight.

Two notes: there are no replicas on dev_docs/prod_docs today, so this is all forward-looking; and heads up the suggestion is anchored to the pre-refactor block (the read/merge/missingFacets code is gone as of d7d3955 / 9d61ccc), so it can't be committed directly.

[manovotny]

Follow-up: the indexer now codifies the docs index's relevance settings — declares them and overwrites on every run. d7d3955, 9d61ccc

Acting on review feedback: instead of read current settings → diff → additively merge each run, the indexer now declares the settings it owns and overwrites them every run, so any dashboard edit reverts on the next reindex. The additive facet merge actually worked against the goal — it preserved whatever was set in the dashboard rather than overwriting it, so indexes could (and did) drift apart.

const searchableAttributes = ['unordered(hierarchy.lvl0)', /* …lvl1–6 */ 'content', 'unordered(keywords)']
const attributesForFaceting = ['branch', 'record_batch', 'sdk'].map((a) => `filterOnly(${a})`)
const ranking = ['typo', 'geo', 'words', 'filters', 'attribute', 'exact', 'proximity', 'custom']
const customRanking = ['desc(weight.pageRank)', 'desc(weight.level)', 'asc(weight.position)']
await algolia.setSettings({ indexName: ALGOLIA_INDEX_NAME, indexSettings: { searchableAttributes, attributesForFaceting, ranking, customRanking } })

Why these four and not the whole settings object (i.e. we don't codify defaults — correct): setSettings is a top-level partial merge, so we declare only the relevance levers we deliberately own and leave everything else (typo tolerance, pagination, highlighting, and Algolia's server-managed defaults) untouched. Snapshotting the full object would freeze those defaults at snapshot time and silently clobber any future default change or intentional tweak — diff noise and maintenance for zero benefit. Synonyms stay generated from _tooltips (already a full replace).

The two added in 9d61ccc8f:

• searchableAttributes — the corpus + attribute priority the attribute ranking criterion rides on (it's what makes a heading match beat a body match, so the reorder depends on it). Identical across dev/prod today, so this locks in the value the ranking work was tested against rather than freezing an unknown state.
• customRanking — the final tiebreaker, normalized to the weights the indexer actually writes (pageRank/level/position). Drops the dead desc(weight.popularity) entry lingering on dev_docs — it's never written to records, so it ranked nothing.

Why availableSDKs is intentionally not faceted — it powers the per-result SDK icons, but the client only retrieves it (Search.tsx → SDKsIcon); it never filters or counts on it, and retrieval is independent of faceting. So the availableSDKs facet that existed on dev_docs was inert — dropping it doesn't touch the icons. branch/record_batch are filterOnly because the stale-record cleanup filters via facetFilters, never facet counts.

Effect on the next index run: adds sdk faceting everywhere, normalizes branch/record_batch to filterOnly, drops the stray availableSDKs facet and the dead customRanking entry on dev_docs, and locks searchableAttributes/ranking to the tested values — leaving prod/dev/test with an identical, code-defined config.

A "warn if the dashboard was edited" guard was considered and deferred: since the indexer rewrites these every run, drift self-heals on the next index, so an explicit alert is more than this needs right now.

[NWylynko]

@manovotny

With adding the settings update to the script it creates a bit of split understanding of the script. Before the settings updates was added, the script would entirely scope itself to the runners current branch, including the branch name in the records that where being added, allowing the search client to scope itself to that branch and know that its only getting that set of results.

But when updating the index settings its across all records in that index, not just the records that are scoped to the branch that is being updated. IMO this introduces a split in how you'd expect the script to run, either not caring about the git branch and just having a single 'dev_docs' index, or that changes made only effect the search items being updated and would expect settings would only change for that branch too.

This stems from a limitation of algolia ultimately and a desire on my part to not spin up a whole new index on every search development branch. So if this is a trade-off we are happy with then I'm happy to move ahead, but I haven't seen this addresses so just want to get your opinion.

[manovotny]

Totally fair catch — it's the sharpest point on the thread, and you're right that it was undocumented.

The way I've come to think about it: records and settings are genuinely different kinds of thing, so the two scopes are each correct rather than inconsistent. Records are data — rightly per-branch (the branch: tag + client filter is what lets us share one index). Settings/synonyms are the index's relevance contract — a property of the index, not of a branch's content. Algolia has no per-branch settings, and conceptually there shouldn't be: "how this index ranks" isn't branch-specific.

Codifying also shrinks the downside you're pointing at. Before, a run could push arbitrary hand-tuned settings index-wide; now every run converges the index to the same declared values, and any change to them is a reviewed diff. So a content branch's indexer run just re-asserts canonical settings — nothing leaks. The seam only bites a search-relevance branch (rare — like this one) that's actively changing settings, and the escape hatch already exists and is exactly what I used here: point ALGOLIA_INDEX_NAME at a personal throwaway index. You pay for an extra index only while experimenting with settings, not per branch.

Full per-branch isolation is the "pure" answer — if usage/cost weren't a factor I'd spin up an index per branch and call it done. But that's a lot of indexes (and money) for a rare need, so shared dev_docs + branch-scoped records + a personal index for settings experiments is the middle ground I'd rather steward toward.

So — accepted trade-off, agreed. And to your "haven't seen this addressed": it is now, in f85cd12 — a scope note in the settings block, plus a new "Search index (Algolia)" section in AGENTS.md spelling out the split, the codified settings, and the personal-index escape hatch.