fix(perf): exclude non-stream requests from TTFT/TPOT/ITL metrics

[qiumuyang]

Summary

When a benchmark run mixes stream: true and stream: false requests, streaming-specific metrics (TTFT / TPOT / ITL) are now computed over stream requests only. Pure-stream and pure-non-stream runs are unchanged.

Motivation

For non-stream requests, first_chunk_latency = query_latency (entire request time) and time_per_output_token = 0. When mixed with real stream requests this inflates TTFT and deflates TPOT.

Mixed runs are reachable via workload_trace (per-request stream from original trace) and line_by_line (JSONL body stream field overrides CLI flag).

Example

8 streaming (TTFT ~50 ms) + 2 non-streaming (latency ~2.5 s):

Metric	Before	After (stream only)
Avg TTFT	(8×0.05 + 2×2.5) / 10 = 0.54 s	0.05 s
Avg TPOT	deflated by two 0 contributions	stream-only mean

Changes

File	What changed
benchmark_util.py	Add is_stream flag and is_stream_body() helper; stream-only accumulators for TTFT/TPOT/ITL with non-stream fallback; expose stream/non-stream counts.
default_api.py	Set is_stream = True in SSE branch; body-based fallback on failure.
openai_responses_api.py	Set is_stream = True in SSE branch; body-based fallback on failure.
http_client.py	Body-based is_stream on timeout/connection-error paths.
db_util.py	Persist is_stream column; percentile TTFT/TPOT/ITL/decode use stream rows only (fallback for pure non-stream).
perf_constants.py	STREAM_REQUESTS / NON_STREAM_REQUESTS constants.
perf_models.py	Count fields in BenchmarkSummary; conditional display when mixed.

Testing

$ pytest tests/perf/test_stream_metrics.py -q
9 passed

[gemini-code-assist]

Code Review

This pull request introduces stream and non-stream metric bucketing to prevent non-stream requests from polluting streaming-specific metrics like TTFT, TPOT, and ITL. It adds an is_stream flag to BenchmarkData, updates MetricsAccumulator and database utilities to calculate separate averages and percentiles, and includes comprehensive unit tests. The review feedback points out a potential AttributeError in the new is_stream_body helper if the request body is not a dictionary (e.g., when it is None or a string on failure paths), and suggests adding a defensive type check.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

The helper function is_stream_body assumes that body is always a dictionary. However, if request generation fails or connection testing fails, body can be None or a non-dictionary type (such as a string). This will cause an AttributeError when calling body.get('stream').

We should add a defensive check to ensure body is a dictionary before calling .get().

Suggested change

	def is_stream_body(body: dict) -> bool:
	"""Best-effort stream classification from the request body.
	Used only when no response is available (error/timeout paths).
	"""
	return body.get('stream') is True
	def is_stream_body(body: Any) -> bool:
	"""Best-effort stream classification from the request body.
	Used only when no response is available (error/timeout paths).
	"""
	if isinstance(body, dict):
	return body.get('stream') is True
	return False

[Yunnglin]

LGTM. Bucketing TTFT/TPOT by the actually-streamed response is the right fix, and the pure-non-stream fallback keeps existing runs unchanged.

Pushed two small refinements to the branch (e1517dad):

1. Dropped the redundant ITL stream bucket — non-stream requests contribute an empty ITL list, so it's already stream-only by construction (TTFT/TPOT/decode correctly stay on the stream subset).
2. Hardened is_stream_body against non-dict bodies on error/timeout paths.

Tests (9 passed) + make lint clean. Thanks!