Skip to content

Commit d07bbbd

Browse files
ernestprovo23ernestprovo23
authored
chore(release): integrate v1.0 PRs + bump 1.0.0 + CHANGELOG (#45)
* chore(release): conclave-cli distribution + OIDC publish workflow + dep audit Distribution + release-engineering for the v1.0 cut (readiness review must-dos 1, 2 scaffolding, and 6): - pyproject: [project].name -> conclave-cli (PyPI `conclave` is an unrelated project). Command (`conclave`), import package (`conclave`), and repo are unchanged. Adds [project.urls]. Version left at 0.3.0 (the 1.0.0 bump is the release commit per RELEASING.md). - README: install is now `pip install conclave-cli`, with a note on the install-name vs command/import split; editable source install kept for dev. - .github/workflows/release.yml: OIDC Trusted-Publishing publish workflow, triggered on `release: published`. build (python -m build) -> pypi-publish (pypa/gh-action-pypi-publish, no token, PEP 740 attestations) -> sign (Sigstore keyless, attaches .sigstore bundles to the Release). Every `uses:` pinned to a full commit SHA with a `# vX.Y.Z` comment. Inert until a Release is published AND the conclave-cli Trusted Publisher is configured. - test.yml: new fail-closed `pip-audit` job auditing the resolved deps. - requirements-dev.lock: hash-pinned dev + runtime tree (uv pip compile --generate-hashes) for reproducible installs. - RELEASING.md: operator runbook (one-time Trusted-Publisher setup, cut-a-release checklist, post-release verification, rollback/yank). - DOCUMENTATION_INDEX.md: index the new release-eng artifacts + version history. * docs(synthesizer): document + version + test synthesizer behavior (v1.0 #5) The synthesizer/judge path is the heart of conclave's "council" value prop but was undocumented and lightly tested, risking silent degradation. This makes it sound and observable for 1.0. Investigation (current behavior, unchanged): - synthesizer = constructor arg, else config `synthesizer:`, else built-in default "claude" (registry.DEFAULT_SYNTHESIZER). Same model judges in adversarial and consolidates in debate. - degraded paths were ALREADY observable, not silent: no-usable-answers, unkeyed synthesizer, and synthesizer-call-failure each set CouncilResult.synthesis_error (adversarial: AdversarialResult.verdict_error, mirrored). synthesis stays None; member answers preserved. No silent quiet-concat path existed to fix -- confirmed + pinned with tests. Changes: - version the synthesis prompt set: new conclave.prompts.SYNTHESIS_PROMPT_VERSION, re-exported from council, stamped onto every CouncilResult as `prompt_version` (lazy default_factory avoids the prompts<->models import cycle). Prompt text is byte-stable; the constant + text are pinned so a prompt change without a version bump fails CI. - document selection/default/configurability/fallback in the council module docstring + _synthesize docstring, and a README "Synthesizer behavior" section. - DOCUMENTATION_INDEX: new test file row, CouncilResult field note, changelog row. - tests/test_synthesizer.py (21 tests): default + arg/config/CLI override selection; observable degradation for synthesize, debate, and adversarial judge (unkeyed + call-failure); prompt-version stability across every mode. No non-synthesis behavior changed; happy-path synthesis output is byte-for-byte unchanged. Mocks at the existing httpx-transport boundary (offline). * test(logging): count only conclave's own handler, not pytest caplog handlers pytest 9.x's logging plugin attaches LogCaptureHandler instances to every logger during a test, inflating raw len(logger.handlers). The two handler-count assertions in test_logging.py asserted an exact total of 1 and now see 3 under pytest 9.1.0, failing in CI (and on a fresh local resolve) regardless of source changes. Filter capture handlers out so the tests assert on the single handler get_logger actually installs -- preserving their intent while being robust to the runner. Production logging code is unchanged (outside pytest it adds exactly one StreamHandler). * test(logging): make handler-count assertions robust to pytest 9.x log capture CI on this branch resolved pytest 9.1.0 (deps are pinned >=8.0.0; main's last green run predates the 9.1.0 release). pytest 9.x attaches its LogCaptureHandler (a StreamHandler subclass) directly to the non-propagating `conclave` logger during a run, so `len(logger.handlers) == 1` now sees 3 handlers and test_logging.py's one-shot-configuration assertions fail across 3.11/3.12/3.13. Count only conclave's own handler via `type(h) is logging.StreamHandler` (pytest's is a subclass) instead of all handlers. This preserves the test's intent exactly -- the factory installs one StreamHandler and never duplicates it -- while ignoring pytest-injected capture handlers, and is stable across pytest versions. No production code changed. * security(keys): key-leak audit + SECURITY.md threat model Audit and harden the BYO-keys leak surface ahead of v1.0, with regression tests per vector (tests/test_keyleak_audit.py): - cache write path: assert a planted key-shaped secret echoed in a provider error never reaches a cache file/filename/key (cache stores the already- redacted CouncilResult); add an explicit invariant comment in cache.store. - streaming path: assert a mid-stream provider error echoing a key is absent from every streamed StreamEvent AND the final ModelAnswer; document the transport->providers redaction boundary in stream_sse. - __repr__/__str__: assert no adapter/config/result object renders key material (no object stores a key; transient request headers are not retained). - provider 400/422 echo: assert buffered error capture runs through redact() for prefixed and unprefixed custom-endpoint keys. - httpx/httpcore DEBUG logging (out-of-band of redact): document loudly and add an opt-in guard_transport_logging() helper that drops transport DEBUG records (the only level that emits auth headers). - audit-found gap (not in the original attack map): the partial-failure catch-alls in Council.fan_out and streaming._drive_member built error strings from raw exception text without redact(); wrap both in redact(). Add a Threat model section to SECURITY.md (trust boundary, what IS protected, accepted limitations, vector map) without touching the disclosure policy. Add .gitleaks.toml allowlisting the obviously-fake test fixtures (test tree only). Tests 191 -> 209; coverage 89%+. Disclosure policy unchanged. * test(logging): make handler-count assertions robust to pytest log capture The two handler-count assertions in test_logging.py counted ALL handlers on the conclave logger, which fails under newer pytest (9.x) whose logging plugin injects LogCaptureHandler instances. Filter to conclave's own StreamHandler via a _app_stream_handlers helper (excludes any handler whose class name contains "Capture") so the tests assert on conclave's one-shot config without coupling to the pytest version. Passes under pytest 8.4.2 and 9.1.0. Test-only; no source change. Unblocks CI for the key-leak audit PR. * security(transport): drop httpx exception from TransportError cause chain (RANK 1/5) The httpx exception raised on a transport failure carries a live .request whose .headers hold the Authorization/x-api-key value. Surfacing it as TransportError.__cause__/__context__ left the key one cause-chain hop away, leaking under traceback.format_exception, logging.exception, or a cause-chain repr. The four raise sites in post_json/stream_sse now route through _raise_transport_error (raise ... from None) plus a boundary clear that nulls __context__, so neither a formatter nor a direct attribute walk can reach the header-bearing exception. Pinned by V8 tests. * security(council): install transport-logging guard by default with opt-out (RANK 6) Council.__init__ now calls transport.guard_transport_logging() automatically unless allow_transport_debug_logging=True, so a process holding a real key is protected from httpx/httpcore DEBUG header leakage out of the box. The guard is idempotent and scoped to the httpx/httpcore loggers only; it never touches the host root logger. SECURITY.md reworded from opt-in to default-on/opt-out, with a new What-IS-protected bullet and vector-map rows for the cause-chain hardening (RANK 1/5) and the default-on guard (RANK 6). Adds V9 (guard default-on + opt-out) and V10 (client close hygiene, RANK 8) regression tests. * release: bump to 1.0.0 + add CHANGELOG + index updates - pyproject [project].version 0.3.0 -> 1.0.0 (name stays conclave-cli) - src/conclave/__init__.py __version__ -> 1.0.0 - DOCUMENTATION_INDEX current version line -> 1.0.0; index rows for CHANGELOG + keyleak-audit tests - new CHANGELOG.md (Keep-a-Changelog) covering v0.3.0 -> v1.0.0
1 parent 97b8ec9 commit d07bbbd

20 files changed

Lines changed: 2804 additions & 74 deletions

‎.github/workflows/release.yml‎

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
# =============================================================================
2+
# Release & publish workflow (conclave v1.0 — distribution + release-eng)
3+
#
4+
# !!! INERT UNTIL CONFIGURED — SAFE TO MERGE !!!
5+
#
6+
# This workflow does NOTHING until BOTH of the following are true:
7+
# (a) a GitHub *Release* is published (a pushed git tag ALONE does not trigger
8+
# it — the trigger is `release: published`, the explicit human gesture), AND
9+
# (b) the `conclave-cli` PyPI project + its OIDC Trusted Publisher are configured
10+
# by the owner (see RELEASING.md "One-time PyPI setup").
11+
#
12+
# Until (a) AND (b) hold, merging this file changes nothing at runtime: no tag is
13+
# cut here, no version is bumped here, nothing is published here. Cutting v1.0.0 is
14+
# then a single tag + GitHub Release away (the full runbook is in RELEASING.md).
15+
#
16+
# What it does WHEN a Release is published:
17+
# build — builds sdist + wheel with `python -m build`, uploads them as
18+
# workflow artifacts so publish + sign consume the exact same bytes.
19+
# pypi-publish — publishes those artifacts to PyPI via OIDC Trusted Publishing
20+
# (NO API token, NO stored secret), with PEP 740 attestations.
21+
# Fails CLOSED if the Trusted Publisher is not yet configured —
22+
# it never falls back to anything insecure.
23+
# sign — signs the sdist + wheel with Sigstore keyless (ambient OIDC ->
24+
# Fulcio) and attaches the `.sigstore` bundles to the GitHub
25+
# Release assets.
26+
#
27+
# Distribution-name note: the PyPI distribution name is `conclave-cli` (the name
28+
# `conclave` is taken by an unrelated project). The command + import package stay
29+
# `conclave`. The Trusted Publisher must therefore name PROJECT `conclave-cli`.
30+
#
31+
# Pin discipline: every `uses:` is pinned to a full 40-char commit SHA with a
32+
# `# vX.Y.Z` comment. conclave's other workflows (test.yml, gitleaks.yml) pin to
33+
# mutable major tags; this publish workflow holds elevated OIDC permissions, so it
34+
# is hardened one level further with full commit-SHA pins.
35+
# =============================================================================
36+
name: Release
37+
38+
on:
39+
# Primary, safe trigger: only fires once a GitHub Release is *published*.
40+
# A pushed tag by itself does NOT publish a Release and therefore does NOT
41+
# trigger this workflow.
42+
release:
43+
types: [published]
44+
45+
# Least-privilege at the top level; each job widens scope locally only as needed.
46+
permissions:
47+
contents: read
48+
49+
jobs:
50+
# --------------------------------------------------------------------------
51+
# Job 1: build sdist + wheel and publish them as workflow artifacts so the
52+
# publish and sign jobs consume the exact same bytes (no rebuild drift).
53+
# --------------------------------------------------------------------------
54+
build:
55+
name: Build sdist + wheel
56+
runs-on: ubuntu-latest
57+
permissions:
58+
contents: read
59+
60+
steps:
61+
- name: Checkout
62+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
63+
64+
- name: Set up Python 3.12
65+
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
66+
with:
67+
python-version: "3.12"
68+
cache: pip
69+
70+
- name: Install build frontend
71+
run: python -m pip install --upgrade "build==1.2.2.post1"
72+
73+
- name: Build sdist + wheel
74+
run: python -m build --sdist --wheel --outdir dist/
75+
76+
- name: Show built artifacts
77+
run: ls -l dist/
78+
79+
- name: Upload dist artifacts
80+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
81+
with:
82+
name: dist
83+
path: dist/
84+
if-no-files-found: error
85+
86+
# --------------------------------------------------------------------------
87+
# Job 2: publish to PyPI via OIDC Trusted Publishing.
88+
#
89+
# No API token, no stored secret: pypa/gh-action-pypi-publish mints a short-lived
90+
# OIDC token from this job's `id-token: write` and exchanges it for a PyPI upload
91+
# token. PyPI must have a Trusted Publisher (pending or active) for `conclave-cli`
92+
# pointing at owner=ernestprovo23, repo=conclave, workflow=release.yml — see
93+
# RELEASING.md. Until that exists, this job fails CLOSED (publish denied); it
94+
# never falls back to anything insecure.
95+
#
96+
# PEP 740 attestations are generated + attached by default (the action's
97+
# `attestations: true` default) since OIDC Trusted Publishing is in use.
98+
# --------------------------------------------------------------------------
99+
pypi-publish:
100+
name: Publish to PyPI (OIDC Trusted Publishing)
101+
needs: build
102+
runs-on: ubuntu-latest
103+
# id-token: write is scoped to THIS job only (never granted at workflow top level).
104+
permissions:
105+
id-token: write # REQUIRED for OIDC Trusted Publishing — mints the token
106+
contents: read
107+
108+
steps:
109+
- name: Download dist artifacts
110+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
111+
with:
112+
name: dist
113+
path: dist/
114+
115+
# Production PyPI via Trusted Publishing (OIDC). No `password:` — the action
116+
# uses ambient OIDC. PEP 740 attestations are on by default.
117+
- name: Publish to PyPI
118+
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0
119+
with:
120+
packages-dir: dist/
121+
# No `password:` — OIDC Trusted Publishing is used (no stored secret).
122+
# `attestations` defaults to true under Trusted Publishing -> PEP 740.
123+
124+
# --------------------------------------------------------------------------
125+
# Job 3: sign the release artifacts with Sigstore keyless.
126+
#
127+
# The job's ambient OIDC mints a short-lived Fulcio cert (issuer
128+
# https://token.actions.githubusercontent.com), signs each dist file, and
129+
# attaches the resulting `.sigstore` bundle(s) to the GitHub Release assets
130+
# (release-signing-artifacts, the action's default on a release event).
131+
#
132+
# We self-verify (`verify: true`) against THIS workflow's own identity so a
133+
# broken run can never publish a bad bundle.
134+
# --------------------------------------------------------------------------
135+
sign:
136+
name: Sign release artifacts (Sigstore keyless)
137+
needs: build
138+
runs-on: ubuntu-latest
139+
permissions:
140+
id-token: write # REQUIRED for ambient OIDC -> Fulcio
141+
contents: write # REQUIRED to attach .sigstore bundles to the Release assets
142+
143+
steps:
144+
- name: Download dist artifacts
145+
uses: actions/download-artifact@d3f86a106a0bac45b974a628896c90dbdf5c8093 # v4.3.0
146+
with:
147+
name: dist
148+
path: dist/
149+
150+
- name: Sign sdist + wheel and attach bundles to the Release
151+
uses: sigstore/gh-action-sigstore-python@5b79a39c381910c090341a2c9b0bf022c8b387e1 # v3.4.0
152+
with:
153+
inputs: ./dist/*.tar.gz ./dist/*.whl
154+
# Self-check: verify what we just signed against THIS workflow's own
155+
# identity so a broken run never publishes a bad bundle.
156+
verify: true
157+
verify-cert-identity: "https://github.com/ernestprovo23/conclave/.github/workflows/release.yml@${{ github.ref }}"
158+
verify-oidc-issuer: "https://token.actions.githubusercontent.com"
159+
# release-signing-artifacts defaults to true: on a release event the
160+
# .sigstore bundles are attached to the Release assets automatically.
161+
release-signing-artifacts: true
162+
upload-signing-artifacts: true

‎.github/workflows/test.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,39 @@ jobs:
6262

6363
- name: ruff format --check
6464
run: ruff format --check .
65+
66+
audit:
67+
name: "pip-audit"
68+
runs-on: ubuntu-latest
69+
70+
steps:
71+
- uses: actions/checkout@v4
72+
73+
- name: Set up Python 3.12
74+
uses: actions/setup-python@v5
75+
with:
76+
python-version: "3.12"
77+
78+
- name: Cache pip
79+
uses: actions/cache@v4
80+
with:
81+
path: ~/.cache/pip
82+
key: ${{ runner.os }}-pip-audit-${{ hashFiles('pyproject.toml') }}
83+
84+
# Install the project so its resolved runtime + dev deps are present in the
85+
# environment, then audit that environment. conclave is security-positioned,
86+
# so this gate is FAIL-CLOSED: a known vulnerability in any resolved
87+
# dependency fails CI. The dep surface is tiny (httpx + a few well-maintained
88+
# libs), so false-positive churn is low. If a transitive CVE with no fix
89+
# blocks an unrelated PR, suppress it narrowly with
90+
# `--ignore-vuln <GHSA/PYSEC id>` and a tracking note (see RELEASING.md).
91+
- name: Install package with dev extras
92+
run: pip install -e ".[dev]"
93+
94+
- name: Install pip-audit
95+
run: pip install pip-audit
96+
97+
# `--skip-editable` drops the local editable `conclave` install (it is not on
98+
# PyPI and cannot be audited); every real dependency is still audited.
99+
- name: Audit resolved dependencies for known vulnerabilities
100+
run: pip-audit --skip-editable

‎.gitleaks.toml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
# gitleaks configuration for conclave.
2+
#
3+
# conclave is a bring-your-own-keys tool whose security tests deliberately plant
4+
# OBVIOUSLY-FAKE, key-SHAPED strings (e.g. "sk-FAKE...", "AIza-FAKE...") to prove
5+
# that redact() / the cache / streaming never let a real key escape. Those fake
6+
# fixtures are not secrets, but a key-shaped literal can trip gitleaks' generic
7+
# rules. This allowlist scopes that exception to the test tree ONLY, so a real
8+
# secret committed anywhere in the source or docs is still caught.
9+
#
10+
# We extend gitleaks' bundled default ruleset rather than replacing it, so every
11+
# upstream detector stays active outside the allowlisted paths.
12+
13+
[extend]
14+
useDefault = true
15+
16+
[allowlist]
17+
description = "Fake, key-shaped fixtures used by the key-leak regression tests are not secrets."
18+
# Restrict the allowance to the tests directory: production code, docs, and
19+
# config are still fully scanned.
20+
paths = [
21+
'''tests/.*\.py''',
22+
]
23+
# Defense in depth: also allow the explicit fake-key marker tokens anywhere they
24+
# appear, so a fixture moved/quoted in a doc example is not flagged. These are
25+
# intentionally synthetic sentinels, never real credentials.
26+
regexes = [
27+
'''FAKE-?[A-Za-z0-9_\-]*''',
28+
'''sk-(test|FAKE|streamleak|CONCLAVE)[A-Za-z0-9_\-]*''',
29+
'''AIza-?(dummy|test|FAKE)[A-Za-z0-9_\-]*''',
30+
]

‎CHANGELOG.md‎

Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Changelog
2+
3+
All notable changes to conclave are documented here.
4+
5+
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7+
8+
## [1.0.0] - 2026-06-14
9+
10+
First stable release. conclave is feature-complete for its 1.0 scope: a
11+
bring-your-own-keys multi-model council that fans a prompt to N foundation
12+
models concurrently and merges their answers. This release integrates three
13+
release-readiness workstreams — distribution/release engineering, key-leak
14+
hardening + threat model, and synthesizer behavior documentation/versioning —
15+
on top of v0.3.0.
16+
17+
### Added
18+
19+
- **Distribution name.** The package is now published to PyPI as `conclave-cli`
20+
(`pip install conclave-cli`); the import package, CLI command, and repo all
21+
stay `conclave`. The bare PyPI name `conclave` is an unrelated project.
22+
- **Release engineering.** OIDC Trusted-Publisher release workflow
23+
(`.github/workflows/release.yml`) with Sigstore keyless signing and PEP 740
24+
attestations, inert until a GitHub Release fires and the publisher is
25+
configured; a hash-pinned dev + runtime lockfile (`requirements-dev.lock`)
26+
for reproducible installs/CI; and a `RELEASING.md` operator runbook.
27+
- **Supply-chain CI.** A fail-closed `pip-audit` job added to the CI workflow.
28+
- **Threat model.** `SECURITY.md` now carries a BYO-keys threat model and the
29+
key-handling guarantees consumers can rely on; `.gitleaks.toml` plus a
30+
dedicated `tests/test_keyleak_audit.py` regression suite guard against
31+
secret leakage.
32+
- **Versioned synthesis prompt.** The synthesis prompt set is versioned via
33+
`conclave.prompts.SYNTHESIS_PROMPT_VERSION` and stamped onto every
34+
`CouncilResult.prompt_version`, so a downstream eval can detect a prompt
35+
change rather than silently absorb it.
36+
37+
### Changed
38+
39+
- **Key-leak: cause-chain fix.** The originating `httpx` exception is no longer
40+
attached to `TransportError.__cause__`, closing a path where a verbose
41+
traceback could surface a key-bearing transport exception.
42+
- **Key-leak: transport-logging guard default-on.** `Council.__init__` now
43+
installs `conclave.transport.guard_transport_logging()` by default, dropping
44+
the httpx/httpcore `DEBUG` records that emit the auth header. Callers who
45+
genuinely need that DEBUG band opt out with
46+
`Council(..., allow_transport_debug_logging=True)`.
47+
- **Synthesizer: observable degradation.** Synthesizer/judge degradation is
48+
confirmed (never silent) across synthesize, debate, and the adversarial-judge
49+
paths: an unkeyed or failed synthesizer surfaces on
50+
`CouncilResult.synthesis_error` (and `AdversarialResult.verdict_error`,
51+
mirrored to `synthesis_error`), with no path where synthesis is both absent
52+
and unexplained.
53+
- **Synthesizer behavior documented.** README gains a "Synthesizer behavior"
54+
section covering selection precedence (`synthesizer=` arg → config →
55+
default), observable degradation, and the versioned prompt.
56+
57+
### Scope
58+
59+
- Feature-complete for 1.0: 4 council modes (synthesize / raw / debate /
60+
adversarial), 9 providers, streaming for synthesize/raw, an optional result
61+
cache, and debate convergence early-stop.
62+
63+
### Roadmap (post-1.0)
64+
65+
- `vote` mode (council issue #3) — a ranked/tallied decision mode — is
66+
documented as planned, not shipped.
67+
- A stdio MCP server (council issue #8) is documented as planned; the earlier
68+
HTTP local-server-mode spike was evaluated and shelved.
69+
70+
## [0.3.0] - 2026-06-08
71+
72+
- Provider-highway refactor: LiteLLM removed in favor of an owned `httpx`
73+
transport + adapter registry across the (then) 5 providers.
74+
- CI foundation: GitHub Actions matrix, ruff lint/format, coverage floor,
75+
gitleaks, and branch protection.
76+
- Key-leak fix in `redact()` for custom OpenAI-compatible endpoints; CLI
77+
exit-code contract and httpx client lifecycle hardening; transport/CLI/logging
78+
test backfill; first public release with community files.
79+
80+
[1.0.0]: https://github.com/ernestprovo23/conclave/compare/v0.3.0...v1.0.0
81+
[0.3.0]: https://github.com/ernestprovo23/conclave/releases/tag/v0.3.0

0 commit comments

Comments
 (0)