Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions .github/scripts/prepare-release.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
set -euo pipefail

# Tag events have no branch filter. Check the peeled commit against main explicitly.
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
if ! git merge-base --is-ancestor HEAD origin/main; then
echo "::error title=Release outside main::Release tags must point to a commit on main."
exit 1
fi

case "$TAG_NAME" in
core-v*)
target=core
version="${TAG_NAME#core-v}"
;;
mmf-v*)
target=mmf
version="${TAG_NAME#mmf-v}"
;;
v*)
target=all
version="${TAG_NAME#v}"
;;
*)
echo "::error title=Invalid release tag::Expected core-v<semver>, mmf-v<semver>, or v<semver>."
exit 1
;;
esac

semver_pattern='^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?$'
if [[ ! "$version" =~ $semver_pattern ]]; then
echo "::error title=Invalid release version::Expected a SemVer version without build metadata."
exit 1
fi

core_version="$version"
if [[ "$target" == mmf ]]; then
# Reuse the closest core release in this commit's history, including legacy joint releases.
if ! core_tag=$(git describe --tags --abbrev=0 --match 'core-v[0-9]*' --match 'v[0-9]*' HEAD); then
echo "::error title=Missing core release::Publish BufferQueue from main before publishing MMF alone."
exit 1
fi
core_version="${core_tag#core-}"
core_version="${core_version#v}"
if [[ ! "$core_version" =~ $semver_pattern ]]; then
echo "::error title=Invalid core release::The nearest core release tag is not valid SemVer."
exit 1
fi
if [[ "$version" != *-* && "$core_version" == *-* ]]; then
echo "::error title=Prerelease core dependency::A stable MMF release requires a stable core release."
exit 1
fi
if ! git diff --quiet "$core_tag" HEAD -- src/BufferQueue; then
echo "::error title=Unreleased core changes::Publish BufferQueue first, or use a joint v<semver> tag."
exit 1
fi
# A tag alone is not evidence that its dependency reached the package registry.
core_path_version=$(printf '%s' "$core_version" | tr '[:upper:]' '[:lower:]')
curl --fail --silent --show-error --location --retry 3 --output /dev/null \
"https://api.nuget.org/v3-flatcontainer/bufferqueue/$core_path_version/bufferqueue.$core_path_version.nupkg"
fi

# Per-project versions prevent an MMF-only tag from inventing a same-version core dependency.
# This file exists only in the runner's temporary directory; project files stay version-free.
release_props="$RUNNER_TEMP/BufferQueue.Release.props"
cat > "$release_props" <<EOF_PROPS
<Project>
<PropertyGroup Condition="'\$(MSBuildProjectName)' == 'BufferQueue'">
<Version>$core_version</Version>
<PackageVersion>$core_version</PackageVersion>
</PropertyGroup>
<PropertyGroup Condition="'\$(MSBuildProjectName)' == 'BufferQueue.MemoryMappedFile'">
<Version>$version</Version>
<PackageVersion>$version</PackageVersion>
</PropertyGroup>
</Project>
EOF_PROPS

{
echo "target=$target"
echo "version=$version"
echo "core_version=$core_version"
} >> "$GITHUB_OUTPUT"
echo "RELEASE_PROPS=$release_props" >> "$GITHUB_ENV"
98 changes: 69 additions & 29 deletions .github/workflows/dotnet-publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ on:
push:
tags:
- "v*"
- "core-v*"
- "mmf-v*"

permissions:
contents: read
Expand Down Expand Up @@ -33,6 +35,15 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v7
with:
fetch-depth: 0

- name: Validate main-only release tag and resolve versions
id: release
shell: bash
env:
TAG_NAME: ${{ github.ref_name }}
run: bash .github/scripts/prepare-release.sh

- name: Setup .NET SDKs
uses: actions/setup-dotnet@v6
Expand All @@ -49,33 +60,17 @@ jobs:
restore-keys: |
${{ runner.os }}-nuget-

- name: Resolve package version
id: version
shell: bash
env:
TAG_NAME: ${{ github.ref_name }}
run: |
version="${TAG_NAME#v}"
semver_pattern='^(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?$'

if [[ "$version" == "$TAG_NAME" || ! "$version" =~ $semver_pattern ]]; then
echo "::error title=Invalid release tag::Expected v<semver>, received '$TAG_NAME'."
exit 1
fi

echo "value=$version" >> "$GITHUB_OUTPUT"

- name: Restore
run: dotnet restore BufferQueue.slnx
run: >-
dotnet restore BufferQueue.slnx
-p:CustomAfterMicrosoftCommonProps="$RELEASE_PROPS"

- name: Build
env:
VERSION: ${{ steps.version.outputs.value }}
run: >-
dotnet build BufferQueue.slnx
--configuration Release
--no-restore
-p:Version="$VERSION"
-p:CustomAfterMicrosoftCommonProps="$RELEASE_PROPS"
-p:ContinuousIntegrationBuild=true

- name: Test
Expand All @@ -84,50 +79,94 @@ jobs:
--configuration Release
--no-build
--no-restore
-p:CustomAfterMicrosoftCommonProps="$RELEASE_PROPS"

- name: Pack BufferQueue
env:
VERSION: ${{ steps.version.outputs.value }}
if: steps.release.outputs.target != 'mmf'
run: >-
dotnet pack "$CORE_PROJECT"
--configuration Release
--no-build
--no-restore
--output artifacts/packages
-p:PackageVersion="$VERSION"
-p:CustomAfterMicrosoftCommonProps="$RELEASE_PROPS"
-p:Authors="$PACKAGE_AUTHORS"
-p:RepositoryUrl="$REPOSITORY_URL"
-p:RepositoryType="$REPOSITORY_TYPE"
-p:PackageLicenseUrl="$PACKAGE_LICENSE_URL"
-p:ContinuousIntegrationBuild=true

- name: Pack BufferQueue.MemoryMappedFile
env:
VERSION: ${{ steps.version.outputs.value }}
if: steps.release.outputs.target != 'core'
run: >-
dotnet pack "$MMF_PROJECT"
--configuration Release
--no-build
--no-restore
--output artifacts/packages
-p:PackageVersion="$VERSION"
-p:CustomAfterMicrosoftCommonProps="$RELEASE_PROPS"
-p:Authors="$PACKAGE_AUTHORS"
-p:RepositoryUrl="$REPOSITORY_URL"
-p:RepositoryType="$REPOSITORY_TYPE"
-p:PackageLicenseUrl="$PACKAGE_LICENSE_URL"
-p:ContinuousIntegrationBuild=true

- name: Verify selected package files and dependency versions
shell: python
env:
RELEASE_TARGET: ${{ steps.release.outputs.target }}
RELEASE_VERSION: ${{ steps.release.outputs.version }}
CORE_VERSION: ${{ steps.release.outputs.core_version }}
run: |
import os
import pathlib
import xml.etree.ElementTree as ET
import zipfile

target = os.environ["RELEASE_TARGET"]
version = os.environ["RELEASE_VERSION"]
packages = []
if target != "mmf":
packages.append("BufferQueue")
if target != "core":
packages.append("BufferQueue.MemoryMappedFile")
directory = pathlib.Path("artifacts/packages")
expected = {f"{package}.{version}.nupkg" for package in packages}
actual = {path.name for path in directory.glob("*.nupkg")}
if actual != expected:
raise SystemExit(f"Unexpected package selection: expected {expected}, found {actual}")
for package in packages:
with zipfile.ZipFile(directory / f"{package}.{version}.nupkg") as archive:
nuspecs = [name for name in archive.namelist() if name.endswith(".nuspec")]
if len(nuspecs) != 1:
raise SystemExit(f"Expected one nuspec in {package}")
metadata = ET.fromstring(archive.read(nuspecs[0]))
text = lambda name: metadata.find(f".//{{*}}{name}").text
if text("id") != package or text("version") != version:
raise SystemExit(f"Unexpected identity or version in {package}")
for framework in ("net8.0", "net10.0"):
archive.getinfo(f"lib/{framework}/{package}.dll")
archive.getinfo("README.md")
if package == "BufferQueue.MemoryMappedFile":
dependencies = [item for item in metadata.findall(".//{*}dependency")
if item.get("id") == "BufferQueue"]
if len(dependencies) != 2 or any(
item.get("version") != os.environ["CORE_VERSION"] for item in dependencies
):
raise SystemExit("MMF must depend on the resolved core version for both frameworks")

- name: Upload NuGet packages
uses: actions/upload-artifact@v7
with:
name: bufferqueue-nuget-${{ steps.version.outputs.value }}
name: bufferqueue-nuget-${{ steps.release.outputs.target }}-${{ steps.release.outputs.version }}
path: artifacts/packages/*.nupkg
if-no-files-found: error

- name: Publish BufferQueue
if: steps.release.outputs.target != 'mmf'
env:
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}
VERSION: ${{ steps.version.outputs.value }}
VERSION: ${{ steps.release.outputs.version }}
run: |
package="artifacts/packages/BufferQueue.$VERSION.nupkg"

Expand All @@ -139,9 +178,10 @@ jobs:
dotnet nuget push "$package" --api-key "$NUGET_API_KEY" --source "$NUGET_SOURCE"

- name: Publish BufferQueue.MemoryMappedFile
if: steps.release.outputs.target != 'core'
env:
NUGET_API_KEY: ${{ secrets.NUGET_API_KEY }}
VERSION: ${{ steps.version.outputs.value }}
VERSION: ${{ steps.release.outputs.version }}
run: |
package="artifacts/packages/BufferQueue.MemoryMappedFile.$VERSION.nupkg"

Expand Down
5 changes: 4 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,10 @@ semantic, persistence format, or consumer lifecycle.
solution and project inputs.
- Build and test both supported target frameworks. Keep the two test projects independently visible in CI so core and
MMF failures are easy to identify.
- Release tags use `v<semver>`. A publication workflow must validate the tag, build and test before packing, upload
- Release tags use `core-v<semver>` for BufferQueue, `mmf-v<semver>` for MemoryMappedFile, or `v<semver>` for both.
Tags must point to a commit already on `main`; reject unmerged-branch tags. Derive versions from tags, never commit
version numbers to project files. Follow [package release guidance](docs/releases.md) for MMF dependency versions.
A publication workflow must validate the tag, build and test before packing, upload
package artifacts, verify expected package files, and use only `NUGET_API_KEY` for NuGet publication.
- Preserve package metadata and package README inclusion when changing the packaging workflow.

Expand Down
2 changes: 2 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ The articles below describe the checked-in implementation and complement the pac

## Reading order

For package publishing and tag conventions, see [package releases](releases.md).

Read [architecture and registrations](design/architecture.md) first to understand what is shared
between storage implementations. Continue with [partitioning and concurrency](design/partitioning-and-concurrency.md)
to understand producer routing and partition ownership, then read the [consumer model](design/consumer-model.md)
Expand Down
2 changes: 2 additions & 0 deletions docs/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@

## 阅读顺序

包发布流程和 tag 规则见[包发布](releases.zh-CN.md)。

先阅读[架构与注册](design/architecture.zh-CN.md),了解两种存储实现共享的边界。随后阅读
[Partition 与并发](design/partitioning-and-concurrency.zh-CN.md),先理解 producer 路由和 partition 所属关系,
再阅读 [Consumer 模型与投递](design/consumer-model.zh-CN.md) 后修改通用队列行为。随后根据所修改的实现阅读对应存储文章:
Expand Down
43 changes: 43 additions & 0 deletions docs/releases.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Package releases

[简体中文](releases.zh-CN.md)

Package versions come from release tags, not project files.

| Tag | Published packages |
| --- | --- |
| `core-v<semver>` | `BufferQueue` only |
| `mmf-v<semver>` | `BufferQueue.MemoryMappedFile` only |
| `v<semver>` | Both packages at the same version; retained for joint releases |

**Release tags must point to a commit already on `main`.** Tags on unmerged branches are rejected before building or
publishing. A historical commit on `main` is allowed; the tag need not point to its latest commit. GitHub tag events
have no branch restriction, so the workflow fetches `main` and explicitly checks commit ancestry.

The version suffix must be SemVer, optionally with a prerelease suffix, without build metadata. For example,
`core-v2.1.1` selects only `BufferQueue`; `mmf-v2.1.2` selects only `BufferQueue.MemoryMappedFile`.
The aliases identify packages: `core` includes shared queue,
consumer, routing, and DI APIs as well as Memory storage.

## MMF dependencies

The MMF package depends on `BufferQueue`; independent package versions do not imply independent source compatibility.
For `mmf-v<semver>`, CI resolves the closest ancestor Core release tag (`core-v<semver>` or legacy `v<semver>`) and uses
that version for the Core dependency. That Core package must already be downloadable from NuGet. Core sources under
`src/BufferQueue/` must match that release, and a stable MMF release cannot depend on a prerelease Core package.

If Core sources have changed, publish Core first, then create the MMF tag, or use a joint `v<semver>` tag to publish both.
For a joint release, both package versions and the MMF dependency use the tag version. A Core-only tag never publishes
MMF automatically.

## Workflow

The publishing workflow validates the tag and `main` ancestry, restores, builds, and tests the whole solution on
.NET 8 and .NET 10, then packs only the selected packages. It verifies the selected package files, identities,
framework assemblies, package README, and MMF dependency versions before uploading artifacts and publishing to NuGet.
A shared concurrency group serializes publishing runs.

Versions are injected per project through an MSBuild props file generated under the runner's temporary directory.
No version numbers or generated version files are committed to the project. Publication uses only `NUGET_API_KEY`.
Updating this configuration does not create a tag or publish a package; publication starts when an authorized release
tag is pushed. A GitHub Release and its package assets should be completed as part of the same release operation.
38 changes: 38 additions & 0 deletions docs/releases.zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# 包发布

[English](releases.md)

包版本从发布 tag 解析,不写入项目文件。

| Tag | 发布的包 |
| --- | --- |
| `core-v<semver>` | 仅 `BufferQueue` |
| `mmf-v<semver>` | 仅 `BufferQueue.MemoryMappedFile` |
| `v<semver>` | 两个包使用相同版本,保留用于联合发布 |

**发布 tag 必须指向已经进入 `main` 的提交。** 未合并分支上的 tag 会在构建、发布前被拒绝。
允许指向 `main` 的历史提交,不要求必须是最新提交。GitHub 的 tag 事件不限制分支,因此工作流会拉取
`main` 并显式检查提交祖先关系。

版本后缀必须符合 SemVer,可带预发布后缀,不支持 build metadata。例如,`core-v2.1.1` 只选择
`BufferQueue`,`mmf-v2.1.2` 只选择 `BufferQueue.MemoryMappedFile`。
别名对应包:`core` 除了 Memory 存储,还包含共享的队列、Consumer、路由和 DI API。

## MMF 依赖

MMF 包依赖 `BufferQueue`;包版本独立不代表源码兼容性独立。对于 `mmf-v<semver>`,CI 会找到当前提交历史中
最近的 Core 发布 tag(`core-v<semver>` 或旧的 `v<semver>`),使用它的版本作为 Core 依赖。
该 Core 包必须已经能从 NuGet 下载,`src/BufferQueue/` 下的源码必须与对应发布一致,稳定版 MMF 不能依赖预发布版 Core。

如果 Core 源码已经变化,应先发布 Core,再创建 MMF tag;也可以用联合发布的 `v<semver>` 同时发布两个包。
联合发布时,两个包及 MMF 的 Core 依赖都使用 tag 中的版本。仅发布 Core 不会自动发布 MMF。

## 工作流

发布工作流先校验 tag 和 `main` 祖先关系,然后在 .NET 8 和 .NET 10 上还原、构建、测试整个解决方案,
只打包所选的包。上传产物和推送 NuGet 前,会验证所选包文件、包标识、目标框架程序集、包 README 和 MMF 依赖版本。
共享 concurrency group 保证发布工作流串行执行。

版本通过 runner 临时目录中生成的 MSBuild props 文件按项目注入,不向项目提交版本号或生成的版本文件。
发布仅使用 `NUGET_API_KEY`。更新这份配置不会创建 tag 或发布包;只有推送经过授权的发布 tag 才会触发发布。
同一次发布操作还应完成对应 GitHub Release 及包附件。
Loading