Sync CLI skills to the catalog + init revamp (catalog installs, workflow picker, drop build skills)

.github/workflows/sync-catalog.yml

@@ -0,0 +1,65 @@
+name: Sync CLI skills to catalog
+
+# Mirrors skills/ into firecrawl/skills under skills/cli/.
+# The catalog copy is a read-only mirror; PRs against it are redirected here.
+
+on:
+  push:
+    branches: [main]
+    paths:
+      - "skills/**"
+      - ".github/workflows/sync-catalog.yml"
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: catalog-sync
+  cancel-in-progress: false
+
+jobs:
+  sync:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout CLI repo
+        uses: actions/checkout@v6
+        with:
+          path: cli
+
+      - name: Checkout catalog
+        uses: actions/checkout@v6
+        with:
+          repository: firecrawl/skills
+          ssh-key: ${{ secrets.CATALOG_DEPLOY_KEY }}
+          path: catalog
+
+      - name: Sync skills/cli
+        run: |
+          cd catalog
+          git config user.name "github-actions[bot]"
+          git config user.email "github-actions[bot]@users.noreply.github.com"
+
+          rm -rf skills/cli
+          mkdir -p skills/cli
+          cp -R ../cli/skills/. skills/cli/
+
+          cat > skills/cli/README.md <<'EOF'
+          # CLI skills (mirror)
+
+          Mirror of [`firecrawl/cli`](https://github.com/firecrawl/cli/tree/main/skills) — do not edit here.
+          PR changes against `firecrawl/cli`; CI overwrites this directory on every sync.
+          EOF
+
+          git add -A
+          if git diff --cached --quiet; then
+            echo "No changes to sync"
+            exit 0
+          fi
+
+          SHORT_SHA="${GITHUB_SHA:0:7}"
+          git commit -m "Sync from firecrawl/cli@${SHORT_SHA}" \
+            -m "Source: https://github.com/${GITHUB_REPOSITORY}/commit/${GITHUB_SHA}"
+
+          # Retry once if another sync landed first; never force-push.
+          git push origin main || { git pull --rebase origin main && git push origin main; }

README.md

@@ -62,11 +62,18 @@ detected harnesses (all selected by default) so you can pick a subset.
 
 ### Agent skills
 
-The init command installs all Firecrawl agent skill segments into AI coding agents (Cursor, Claude Code, Windsurf, etc.):
+The init command installs the **CLI skills** by default and offers the **workflow skills** as optional extras, into AI coding agents (Cursor, Claude Code, Windsurf, etc.):
 
-- **CLI skills** — teach agents how to use the Firecrawl CLI for live web work (search, scrape, interact, map, crawl, agent)
-- **Build skills** — teach agents how to integrate Firecrawl into application code (choose endpoints, wire SDKs, set up API keys)
-- **Workflow skills** — teach agents how to produce Firecrawl-powered deliverables such as research briefs, SEO audits, QA reports, lead lists, knowledge bases, and design-system extraction
+- **CLI skills** — teach agents how to use the Firecrawl CLI for live web work (search, scrape, interact, map, crawl, agent). Installed by default.
+- **Workflow skills** — teach agents how to produce Firecrawl-powered deliverables such as research briefs, SEO audits, QA reports, lead lists, knowledge bases, and design-system extraction. Interactive multi-select during init.
+
+All skill families live in the [`firecrawl/skills`](https://github.com/firecrawl/skills) catalog — including the **build skills** for integrating Firecrawl into application code:
+
+```bash
+npx skills add firecrawl/skills
+```
+
+> Contributing skills? CLI skills (including the research/developer index skills) → PR this repo (`skills/`). Build/SDK skills → PR the [`firecrawl`](https://github.com/firecrawl/firecrawl) monorepo (`skills/`). Workflow skills → PR [`firecrawl/firecrawl-workflows`](https://github.com/firecrawl/firecrawl-workflows). The catalog ([`firecrawl/skills`](https://github.com/firecrawl/skills)) is read-only — never PR it directly.
 
 To reinstall skills manually:
 
@@ -961,9 +968,9 @@ firecrawl x download https://docs.firecrawl.dev --include-paths "/features,/sdks
 
 ### Workflow Skills
 
-The old experimental AI workflow commands have moved to the NPX-installable
-[`firecrawl/firecrawl-workflows`](https://github.com/firecrawl/firecrawl-workflows)
-skills package. Workflow skills infer from the user's request first and only ask
+The old experimental AI workflow commands have moved to the
+[`firecrawl/skills`](https://github.com/firecrawl/skills) catalog
+(`skills/workflows/`). Workflow skills infer from the user's request first and only ask
 short clarifying questions when required inputs are missing. Install them with:
 
 ```bash

skills/firecrawl-developer-index/SKILL.md

@@ -1,41 +1,59 @@
 ---
 name: firecrawl-developer-index
-description: |
-  Search issues, merged pull requests, READMEs, and documentation. Use when the question is how a library or API behaves, what an error means, or whether a bug was fixed; prefer this over a general web page.
-allowed-tools:
-  - Bash(firecrawl *)
-  - Bash(npx firecrawl-cli *)
+description: Search issues, merged pull requests, READMEs, and documentation. Use when the question is how a library or API behaves, what an error means, or whether a bug was fixed; prefer this over a general web page.
 ---
 
-# firecrawl developer
+# Firecrawl Developer Index
 
-Answer a developer question from the primary source: the issue, the merged pull request that fixed it, or the README/docs passage that states the contract.
+Answer a developer question from the primary source: the issue where the bug was reported, the merged pull request that fixed it, the README or documentation page that states the contract. A blog post that describes a behaviour is a weaker answer than the passage that defines it, so reach for the index first and the open web second.
 
-## Quick start
+There is **no fixed recipe**. Read the question, decide what kind it is, and choose the approach below. A literal error string wants a different move than "how do I do X". Don't run machinery a question doesn't call for.
 
-```bash
-mkdir -p .firecrawl
-firecrawl developer "how do I configure retries" --limit 10 -o .firecrawl/developer.json --json
-jq -r '.results[] | .id, .url, .passages[].text' .firecrawl/developer.json
-```
+## The tools, and what each is uniquely good at
 
-Run `firecrawl developer --help` for the full option list.
+- HTTP: **`GET|POST https://api.firecrawl.dev/v2/search/developer`**
+  MCP: **`firecrawl_developer_search(query, k?, skills?)`**
+  CLI: **`firecrawl developer <query> [--limit <n>]`**
+  Ranked results over the whole index. Each carries `id` (`issue:owner/repo#123`), `url`, and the **matched passages in markdown**, so tables and code blocks survive. The artifact kind is the `id` prefix: `doc:`, `issue:`, `pull_request:`, or `readme:`.
+  The default first move for a developer question. It is the only surface that returns the passages, which is what lets you answer instead of pointing at a page.
+  `k` / `--limit` is 1–100 and defaults to 10. `skills="only"` (HTTP/MCP only) restricts the search to agent-skill files.
+  Keyless; send `Authorization: Bearer $FIRECRAWL_API_KEY` for higher rate limits.
 
-HTTP: `GET|POST https://api.firecrawl.dev/v2/search/developer`. MCP: `firecrawl_developer_search`. Each hit carries `id`, `url`, `passages`. Kind is the `id` prefix (`doc:`, `issue:`, `pull_request:`, `readme:`). Hits do not carry a `type` field.
+- MCP: **`firecrawl_search(query, categories: ["developer"])`**
+  CLI: **`firecrawl search <query> --categories developer`**
+  Developer hits in a `developer` group beside `web`, each with `url`, `title`, `description` (the matched passage), `position`, and `category: "developer"` — web results carry no `category`, so that is the field to key on when merging.
+  Use this when you are **already** running a web search and want developer sources weighed in the same call. It exposes none of the filters and no passage control.
 
-**Done when:** the answer quotes a matched passage and cites its `url` (fall back to `url` when `title` is absent), or you have moved to the open web because the index had nothing to say.
+- MCP: **`firecrawl_scrape(url)` / `firecrawl_search(query)`**
+  CLI: **`firecrawl scrape <url>` / `firecrawl search <query>`**
+  General web fetch and search, for what no primary source states: a comparison between two libraries, an outage, a migration write-up, a project with no public repository or indexed docs.
+  Also the follow-through when a hit is the right page but you need all of it — `scrape` the result's `url`.
 
-## Tips
+## Filters, and what each one costs you
 
-- Default first move is `firecrawl developer`. Use `search --categories developer` only when you are already running a web search and want developer hits in the same call (no passage control, no index filters).
-- Literal error or stack trace: search the string plus the library name. On HTTP, `types=["issue","pull_request"]`. Strip paths, line numbers, and ids, then retry.
-- API contract: `readme` and `doc` are authoritative. A merged PR supersedes an issue report. Never answer from an opening report alone.
-- Scope last: search the whole index, then narrow with HTTP `types`, `repos`, or `sources`. If a scoped search is empty, read the echoed `indexed` flag before concluding the repo is missing.
-- Repository filters (`language`, `topic`, `license`, `min_stars`, …) drop `doc` results unless you also pass `sources`. `types`, `repos`, `sources`, `passages`, and those repository filters are HTTP-only.
-- Comparison, opinion, news, or an unindexed project: `firecrawl search`, then `firecrawl scrape`.
+Only the HTTP surface takes these. On `GET`, pass `types=issue,pull_request` or repeat the parameter; on `POST`, pass arrays. All are optional.
 
-## See also
+- `types` — which of `doc`, `issue`, `pull_request`, `readme` to search. Defaults to all four. Narrowing here is the cheapest way to sharpen a query.
+- `repos` (`owner/name`) scopes the repository half, meaning `issue`, `pull_request`, and `readme`; `sources` (documentation source ids, at most 20) scopes the documentation half, meaning `doc`. Passing both **unions** the halves rather than intersecting them. Both echo back in the response with `indexed: true|false` — that is how you tell "not in the index" from "found nothing".
+- A filter that cannot match any requested `type` is a `400`, not an empty list: `repos` with no repository type in `types`, or `sources` without `doc`.
+- `passages` (1–5, default 1) is the _maximum_ passages per result, not a guarantee. Raise it when one page is clearly the right page but the first passage is the wrong part of it.
+- `language`, `topic`, `license`, `min_stars`, `max_stars`, `archived`, `fork` describe a **repository**. Most documentation pages in the index have no repository behind them, so no repository fact can admit or exclude one. Send any of these without a `sources` scope and the response holds repository evidence only — `issue`, `pull_request`, `readme`. That is the design, not an index fault: do not retry it and do not report the index broken. To keep documentation, drop the repository filters, or scope the documentation half with `sources` and read the `sources` echo to confirm the id is indexed.
 
-- [firecrawl-search](../firecrawl-search/SKILL.md) — open web, or `search --categories developer` in the same call
-- [firecrawl-scrape](../firecrawl-scrape/SKILL.md) — full page when a hit is right but you need all of it
-- [firecrawl-research-index](../firecrawl-research-index/SKILL.md) — papers, not this index
+## Match the approach to the question
+
+- **Literal error message or stack-trace string** → search the string itself plus the library name, with `types=["issue","pull_request"]`. Whoever hit it filed it. If nothing matches, strip the volatile parts (paths, line numbers, ids, addresses) and retry — the invariant middle of the message is what is indexed.
+- **Conceptual "how do I do X"** → the full question in natural language, all four types. The answer is usually a `doc` or a `readme`; raise `passages` before raising `k`.
+- **Known bug** → the issue reports it, the merged pull request _fixes_ it, and the fix is what you want. Search `types=["issue","pull_request"]`, then re-query the issue's own terms scoped to its repo with `types=["pull_request"]`. A merged PR's passages tell you what changed and in which direction.
+- **API contract** ("what does X return", "is Y required", "what is the default") → `readme` and `doc` are authoritative and a blog post is not. Use `types=["readme","doc"]`. If the contract looks like it moved, follow up with `pull_request` for the change that moved it.
+- **Version-specific behaviour** → an issue's opening report describes the broken version; its resolution supersedes it. Raise `passages` to see further into the thread, and read the resolution and the linked pull request before answering. Never answer from an opening report alone.
+- **Scoped to one library** → `repos=["owner/name"]` when you know the slug, plus `sources` if you want its docs in the same call. If a scoped search comes back empty, read the echoed `indexed` flag first: `false` means nothing from that repo or source can ever match and no rephrasing will help — drop the scope and search the whole index, or go to the web.
+- **Ecosystem-wide** ("which libraries do X", "who else hit this") → no scope. Use `language` / `topic` / `min_stars` to keep to maintained repositories, accepting that this gives up all `doc` results.
+- **Agent skills and tooling conventions** → `skills="only"` (HTTP/MCP only).
+- **Comparison, opinion, news, or an unindexed project** → the open web. `firecrawl_search`, then `firecrawl_scrape` whatever deserves a full read. Combining is often right: take the contract from the index and the trade-off from the web.
+
+## Principles
+
+- **Quote the passage, cite the `url`.** The passages are the evidence; hand them over rather than paraphrasing them into a claim the reader can't check. `title` is frequently absent on `doc` results — fall back to `url`.
+- **A merge supersedes a report.** When an issue and a pull request disagree, the merged pull request is the current behaviour. Say which one you read.
+- **Scope last, not first.** Search the whole index, then narrow with `types`, `repos`, or `sources` once you know what the hits look like. Scoping first hides the result that would have told you where to look.
+- **Go to the web when the index has nothing to say.** Trade-offs, ecosystem opinion, and anything about an unindexed project are web questions. Don't force them through the index, and don't dress a general web page up as a primary source.