Skip to content

chore: align ci titles and group tests #225

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mogita merged 2 commits into from
Mar 20, 2026
Merged

chore: align ci titles and group tests #225

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
cf91fcc
ci: align ci titles
mogita Mar 19, 2026
22e873a
chore: add makefile to uniform test and lint commands
mogita Mar 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 33 additions & 49 deletions .github/workflows/run_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,8 @@ jobs:
uses: actions/checkout@v5
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
- name: Run Ruff linter
run: uv run ruff check .
- name: Run Ruff formatter
run: uv run ruff format --check .
- name: Run linter
run: make lint

typecheck:
name: Type Check (ty)
Expand All @@ -37,31 +35,14 @@ jobs:
uses: actions/checkout@v5
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
- name: Run ty type checker
run: |
uvx ty check getstream/ \
--exclude "getstream/models/" \
--exclude "getstream/video/rtc/pb/" \
--exclude "**/rest_client.py" \
--exclude "**/async_rest_client.py" \
--exclude "getstream/chat/channel.py" \
--exclude "getstream/chat/async_channel.py" \
--exclude "getstream/chat/client.py" \
--exclude "getstream/chat/async_client.py" \
--exclude "getstream/common/client.py" \
--exclude "getstream/common/async_client.py" \
--exclude "getstream/moderation/client.py" \
--exclude "getstream/moderation/async_client.py" \
--exclude "getstream/video/client.py" \
--exclude "getstream/video/async_client.py" \
--exclude "getstream/video/call.py" \
--exclude "getstream/video/async_call.py" \
--exclude "getstream/feeds/client.py" \
--exclude "getstream/feeds/feeds.py" \
--exclude "getstream/stream.py"
- name: Run type checker
run: make typecheck

test:
name: Test "${{ inputs.marker }}"
# ── Non-video tests (chat, feeds, etc.) ────────────────────────────
# Uses STREAM_CHAT_* credentials which do NOT have video enabled.
# Video paths and manual test paths are defined in the Makefile.
test-non-video:
name: Non-video tests (${{ matrix.python-version }})
environment:
name: ci
runs-on: ubuntu-latest
Expand All @@ -77,32 +58,35 @@ jobs:
uses: ./.github/actions/python-uv-setup
with:
python-version: ${{ matrix.python-version }}
- name: Run non-video tests
- name: Run tests
env:
STREAM_API_KEY: ${{ vars.STREAM_CHAT_API_KEY }}
STREAM_API_SECRET: ${{ secrets.STREAM_CHAT_API_SECRET }}
STREAM_BASE_URL: ${{ vars.STREAM_CHAT_BASE_URL }}
run: |
uv run pytest -m "${{ inputs.marker }}" tests/ getstream/ \
--ignore=tests/rtc \
--ignore=tests/test_video_examples.py \
--ignore=tests/test_video_integration.py \
--ignore=tests/test_video_openai.py \
--ignore=tests/test_signaling.py \
--ignore=tests/test_audio_stream_track.py \
--ignore=getstream/video
- name: Run video tests
run: make test MARKER="${{ inputs.marker }}"

# ── Video tests (video-enabled credentials) ─────────────────────
# Uses STREAM_* credentials which have video enabled.
test-video:
name: Video tests (${{ matrix.python-version }})
environment:
name: ci
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13"]
timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@v5
- name: Install dependencies
uses: ./.github/actions/python-uv-setup
with:
python-version: ${{ matrix.python-version }}
- name: Run tests
env:
STREAM_API_KEY: ${{ vars.STREAM_API_KEY }}
STREAM_API_SECRET: ${{ secrets.STREAM_API_SECRET }}
STREAM_BASE_URL: ${{ vars.STREAM_BASE_URL }}
run: |
uv run pytest -m "${{ inputs.marker }}" \
tests/rtc \
tests/test_video_examples.py \
tests/test_video_integration.py \
tests/test_video_openai.py \
tests/test_signaling.py \
tests/test_audio_stream_track.py \
getstream/video
run: make test-video MARKER="${{ inputs.marker }}"
52 changes: 32 additions & 20 deletions DEVELOPMENT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,59 +1,71 @@
# Getstream Python SDK

### Running tests
### Setup

Clone the repo and sync
Clone the repo and install dependencies:

```
git clone git@github.com:GetStream/stream-py.git
uv sync --no-sources --all-packages --all-extras --dev
cp .env.example .env # fill in your Stream credentials
pre-commit install # optional: enable commit hooks
```

Env setup
### Running tests

Run `make help` to see all available targets. The most common ones:

```
cp .env.example .env
make test # non-video tests (chat, feeds, moderation, etc.)
make test-video # video/WebRTC tests only
make test-all # both of the above
```
Comment on lines +3 to 22
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Mar 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fix the new Markdown hierarchy/language lint regressions.

These sections read as top-level siblings of ## Release and ## OpenAPI & Protobuf, so they should be ## headings instead of ###. The added command snippets are also all shell examples, so fencing them as shell will clear the MD001/MD040 warnings and keep the document structure consistent.

Also applies to: 27-64

🧰 Tools
🪛 markdownlint-cli2 (0.21.0)

[warning] 3-3: Heading levels should only increment by one level at a time
Expected: h2; Actual: h3

(MD001, heading-increment)

[warning] 7-7: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

[warning] 18-18: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@DEVELOPMENT.md` around lines 3 - 22, Change the two top-level sections
currently using "###" headings ("### Setup" and "### Running tests") to "##" so
they are siblings of "## Release" and "## OpenAPI & Protobuf", and update the
three fenced code blocks under those headings to use shell language identifiers
(```shell) so markdown lint rules MD001/MD040 are satisfied; apply the same
heading and fence fixes to the other occurrences mentioned (the later blocks
around the remainder of the file).


Run tests
Non-video and video tests are split because they require different Stream credentials.
The `MARKER` variable defaults to `"not integration"`. Override it for integration tests:

```
uv run pytest -m "not integration" tests/ getstream/
make test-integration # runs both groups with -m "integration"
make test MARKER="integration" # or target a single group
```

### Commit hook
Two manual tests exist for local telemetry inspection (excluded from CI):

```
pre-commit install
make test-jaeger # requires local Jaeger (docker run ... jaegertracing/all-in-one)
make test-prometheus # requires getstream[telemetry] deps
```

### Check

Shortcut to ruff, ty (type checker) and non integration tests:
### Linting and type checking

```
uv run python dev.py
make lint # ruff check + format check
make typecheck # ty type checker (excludes generated code)
```

### Formatting
To auto-fix lint issues and format:

```
uv run ruff check --fix
make format
```

### Type checking (ty)

Type checking is run via the `ty` type checker, excluding generated code:
Run lint, typecheck, and non-video tests in one go:

```
uv run python dev.py ty
make check
```

Or manually (note: requires exclude flags for generated code - see dev.py for the full list):
### Code generation

```
uvx ty check getstream/ --exclude "getstream/models/" --exclude "getstream/video/rtc/pb/" ...
make regen # regenerate OpenAPI + WebRTC protobuf code
```

### Legacy: dev.py

`dev.py` is an older CLI tool that predates the Makefile. It still works but does not
handle the video/non-video test split or manual test exclusions. Prefer `make` targets.

## Release

Create a new release on Github, CI handles the rest. If you do need to do it manually follow these instructions:
Expand Down
124 changes: 124 additions & 0 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,124 @@
.DEFAULT_GOAL := help

# ── Pytest marker expression ───────────────────────────────────────
# Override on the command line: make test MARKER="integration"
# Default excludes integration tests for everyday local development.
MARKER ?= not integration

# ── Video-related test paths (single source of truth) ──────────────
# When you add a new video test file, add it here. Both test (via
# --ignore) and test-video (via positional args) stay in sync.
VIDEO_PATHS := \
getstream/video \
tests/rtc \
tests/test_audio_stream_track.py \
tests/test_connection_manager.py \
tests/test_connection_utils.py \
tests/test_signaling.py \
tests/test_video_examples.py \
tests/test_video_integration.py \
tests/test_video_openai.py \
tests/test_webrtc_generation.py

# ── Manual tests (require local infrastructure, never run in CI) ───
MANUAL_PATHS := \
tests/test_tracing_jaeger_manual.py \
tests/test_metrics_prometheus_manual.py

# ── Derived ignore flags ──────────────────────────────────────────
VIDEO_IGNORE := $(addprefix --ignore=,$(VIDEO_PATHS))
MANUAL_IGNORE := $(addprefix --ignore=,$(MANUAL_PATHS))

# ── Typecheck exclusions ──────────────────────────────────────────
TY_EXCLUDES := \
--exclude "getstream/models/" \
--exclude "getstream/video/rtc/pb/" \
--exclude "**/rest_client.py" \
--exclude "**/async_rest_client.py" \
--exclude "getstream/chat/channel.py" \
--exclude "getstream/chat/async_channel.py" \
--exclude "getstream/chat/client.py" \
--exclude "getstream/chat/async_client.py" \
--exclude "getstream/common/client.py" \
--exclude "getstream/common/async_client.py" \
--exclude "getstream/moderation/client.py" \
--exclude "getstream/moderation/async_client.py" \
--exclude "getstream/video/client.py" \
--exclude "getstream/video/async_client.py" \
--exclude "getstream/video/call.py" \
--exclude "getstream/video/async_call.py" \
--exclude "getstream/feeds/client.py" \
--exclude "getstream/feeds/feeds.py" \
--exclude "getstream/stream.py"

# ── Targets ───────────────────────────────────────────────────────

.PHONY: test test-video test-all test-integration test-jaeger test-prometheus lint format typecheck check regen help

## Run non-video tests (chat, feeds, moderation, etc.)
test:
uv run pytest -m "$(MARKER)" tests/ getstream/ $(VIDEO_IGNORE) $(MANUAL_IGNORE)

## Run video/WebRTC tests only
test-video:
uv run pytest -m "$(MARKER)" $(VIDEO_PATHS)

## Run all tests (non-video + video), excluding manual tests
test-all:
uv run pytest -m "$(MARKER)" tests/ getstream/ $(MANUAL_IGNORE)

## Run integration tests for both groups
test-integration:
$(MAKE) test MARKER="integration"
$(MAKE) test-video MARKER="integration"

## Run the Jaeger tracing manual test (requires local Jaeger on :4317)
test-jaeger:
uv run pytest -m "integration" tests/test_tracing_jaeger_manual.py

## Run the Prometheus metrics manual test (requires telemetry deps)
test-prometheus:
uv run pytest -m "integration" tests/test_metrics_prometheus_manual.py

## Run Ruff linter and formatter check
lint:
uv run ruff check .
uv run ruff format --check .

## Auto-fix lint issues and format
format:
uv run ruff check --fix .
uv run ruff format .

## Run ty type checker
typecheck:
uvx ty check getstream/ $(TY_EXCLUDES)

## Run full check: lint + typecheck + non-video tests
check: lint typecheck test

## Regenerate all generated code (OpenAPI + WebRTC protobuf)
regen:
./generate.sh
./generate_webrtc.sh

## Show available targets
help:
@echo "Usage: make <target> [MARKER=\"...\"]"
@echo ""
@echo "Test targets:"
@echo " test Non-video tests (default MARKER='not integration')"
@echo " test-video Video/WebRTC tests only"
@echo " test-all All tests except manual infrastructure tests"
@echo " test-integration Integration tests (both non-video and video)"
@echo " test-jaeger Jaeger tracing manual test (needs Docker Jaeger)"
@echo " test-prometheus Prometheus metrics manual test (needs telemetry deps)"
@echo ""
@echo "Quality targets:"
@echo " lint Ruff linter + format check"
@echo " format Auto-fix lint issues and format code"
@echo " typecheck ty type checker"
@echo " check Full check: lint + typecheck + non-video tests"
@echo ""
@echo "Code generation:"
@echo " regen Regenerate OpenAPI + WebRTC protobuf code"
5 changes: 5 additions & 0 deletions dev.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,11 @@
"""
Development CLI tool for getstream SDK.
Essential dev commands for testing, linting, and type checking.
NOTE: Prefer using the Makefile instead (run `make help` for available targets).
The Makefile is a superset of this script.
This script is kept for backwards compatibility.
"""

import os
Expand Down
Loading