redai-infra · Pinoeer-kingxi · Jul 29, 2026 · Jul 29, 2026 · Jul 29, 2026 · Jul 29, 2026
@@ -0,0 +1,97 @@
+# Task 21: Hybrid-async multimodal pipeline benchmark results
+
+## Steady-state result
+
+The formal steady-state campaign completed on commit `e2be8cd158609cc2dfae72b7ba92df72cacb3091` using physical GPUs `0,1,2,6,7`: actor TP2 x CP2 x DP1 on four GPUs and one rollout engine on one GPU. Each B/P/P+R/P+S condition used two paired seeds (`20260816`, `20260817`), 40 optimizer updates, 10 warmup updates, and measured windows `10-19`, `20-29`, and `30-39`.
+
+| Condition | Steady token throughput (mean, range) | Samples/s (mean) | Geomean vs reference |
+|---|---:|---:|---:|
+| B | 1,209.50 (1,144.11-1,274.89) token/s | 1.562 | reference |
+| P | 1,197.49 (1,184.17-1,210.81) token/s | 1.516 | -0.85% token/s vs B |
+| P+R | 1,565.68 (1,559.71-1,571.65) token/s | 1.988 | **+29.64% token/s vs B**; +30.75% vs P |
+| P+S | 1,188.79 (1,142.59-1,234.99) token/s | 1.498 | -1.64% token/s vs B |
+
+P+R is positive in both paired seeds (`+36.33%`, `+23.28%` token/s vs B), with a 21.50% geometric-mean step-time reduction. The ProcessorPool-only and chunk-overlap paths do not show a sustained throughput gain in this campaign; their values remain reported as attributable ablations rather than being combined into the P+R claim.
+
+The three warmup-excluded window means (token/s) were:
+
+| Condition | 10-19 | 20-29 | 30-39 |
+|---|---:|---:|---:|
+| B | 1,221.46 | 1,188.81 | 1,219.31 |
+| P | 1,202.85 | 1,204.60 | 1,185.82 |
+| P+R | 1,556.00 | 1,565.52 | 1,575.02 |
+| P+S | 1,198.25 | 1,178.24 | 1,190.35 |
+
+## 40-step training curves
+
+The gradient norm, training loss, and raw reward curves below come from the same eight formal real-training runs used for the steady-state throughput result. They show every optimizer step from `0` through `39`; the shaded `0-9` interval is warmup, thick lines are the two-seed means, thin lines are the individual seeds, and bands span the two seed values at each step. The values are plotted directly from the TensorBoard scalar exports without smoothing or interpolation. The underlying per-step data are available in [`task21_steady_training_curves.csv`](steady-state/task21_steady_training_curves.csv).
+
+| Condition | Steady grad norm mean (max) | Steady loss mean | Steady raw reward mean |
+| --- | ---: | ---: | ---: |
+| B | 0.67076 (2.20789) | 0.01818 | 0.42181 |
+| P | 0.75002 (3.24743) | 0.01381 | 0.41393 |
+| P+R | 1.43510 (43.00600) | 0.01920 | 0.41706 |
+| P+S | 0.62810 (2.39713) | 0.01823 | 0.41888 |
+
+![40-step gradient norm](steady-state/task21_steady_grad_norm.png)
+
+P+R seed `20260817` has one finite gradient-norm spike at step `21` (`43.00600`). It returns to the surrounding range on the next step, and the corresponding loss (`0.02013`) and raw reward (`0.39453`) remain within the observed trajectories rather than diverging.
+
+![40-step training loss](steady-state/task21_steady_loss.png)
+
+![40-step raw reward](steady-state/task21_steady_reward.png)
+
+Every formal run conserved 256 actor samples per update and exactly 40 optimizer updates; measured image count was 1 per sample, metrics were finite, and all five selected GPUs have steady NVML samples. The analyzer reports `measurement_scope=steady_state`, `validation=passed`, two independent seeds, and 30 measured optimizer steps per run. The reviewable campaign report and paired CSVs are checked in under [`steady-state/`](steady-state/); the report records the benchmark commit separately from later documentation and merge-conflict fixes.
+
+## Protocol
+
+- Performance-code commit: `780c742793fea54acdb28cb40ffb591fb909ba51`.
+- Base commit: `9a5674afde12f608698ab4f60cdb9849a0eb6cb3`.
+- Hardware: physical RTX A6000 GPUs `1,2,3,4,6`; four actor GPUs (`TP=2`, `CP=2`) plus one rollout GPU.
+- Workload: Qwen3-VL-8B-Instruct, OpenR1-Multimodal, 256 generated samples per run, response cap 1024, one real optimizer step.
+- Design: paired seeds `20260811`–`20260814`, ABBAABBA order; baseline and experiment differ only in `--hybrid-pipeline-overlap`.
+- Statistics: four fresh-process paired runs, each containing one optimizer step. The analyzer validated every run and all preregistered first-step targets.
+
+These results measure first-step latency and throughput under the fixed Qwen3-VL,
+TP2 x CP2 x DP1 profile. They do not establish steady-state training
+throughput. A steady-state claim requires multi-step runs with warmup excluded
+and later measurement windows declared before analysis.
+
+## Result summary
+
+| Metric | Baseline | Experiment | Result |
+|---|---:|---:|---:|
+| step token throughput, arithmetic mean | 575.68 token/s | 644.69 token/s | paired geometric-mean speedup **12.01%** |
+| hybrid phase-1 time, arithmetic mean | 165.38 s | 130.38 s | paired geometric-mean reduction **21.21%** |
+| end-to-end step time, arithmetic mean | 342.24 s | 305.61 s | **10.70% lower** |
+| producer/actor overlap | 0/4 runs | 4/4 runs | **100%** experiment overlap |
+| first-step-window GPU utilization, arithmetic mean | 40.01% | 44.37% | +4.36 percentage points |
+| sampled peak VRAM, maximum | 47,380 MiB | 47,140 MiB | no regression |
+
+Every paired run exceeded the preregistered 5% throughput target: `10.22%`, `12.21%`, `12.80%`, and `12.82%`. Every pair also reduced phase-1 time by `19.11%`–`23.53%`.
+
+The throughput plot shows both total-token and response-token throughput for each matched seed. The experiment point is above its baseline point in every pair.
+
+![Step throughput](task21_step_throughput.png)
+
+The phase-1 plot shows the producer/actor overlap interval, phase-1 latency, and producer lead. The experiment overlaps producer work with actor forward in all four runs, while the baseline deliberately disables that overlap.
+
+![Phase-1 overlap](task21_phase1_overlap.png)
+
+The GPU plot combines the 500 ms NVML time series with first-step-window utilization, idle ratio, and sampled peak VRAM. It describes these fresh-process runs only.
+
+![GPU utilization and VRAM](task21_gpu_util_vram.png)
+
+The quality plot pairs reward, response length, truncation, loss, gradient norm, and PPO KL by seed. Workload, reward, response length, and truncation are exact within each pair; loss differs by at most `6.21e-5`, PPO KL remains within `2.37e-11`, and every value is finite. Seed `20260814` shows a disclosed gradient-norm difference (`1.0033` vs `0.4775`) caused by different asynchronous microbatch grouping; the deterministic replay parity run controls grouping and bounds the patched-path gradient-norm difference to `0.207%`.
+
+![Correctness and quality](task21_correctness_quality.png)
+
+The window summary shows each paired throughput speedup and the geometric mean against the preregistered 5% threshold.
+
+![Window and paired summary](task21_window_summary.png)
+
+## Correctness evidence
+
+The final replay parity dataset contains 1,024 samples across four actor ranks. Baseline/control/experiment agree exactly for tokens, response lengths, loss masks, rewards, raw rewards, advantages, returns, truncation flags, multimodal tensors, and dynamic-microbatch schedules. Loss is exactly `0.03446025028824806`; experiment log-probability maximum absolute difference is `2.3842e-7`; gradient-norm symmetric relative difference is `0.207%` (below the BF16 0.5% guardrail).
+
+Full raw logs, manifests, TensorBoard events, timeline JSONL, NVML CSV, summaries, and generated figures are stored under `/data01/LWX/relax-task21/`. Compact paired data and checksums are included in this directory.
@@ -0,0 +1,10 @@
+5016e5fc77093f2df3d85942e8abf95999738c75191c17a37176d53bbfc8a4b3  /data01/LWX/relax-task21/comparisons/FORMAL-FINAL8-Q3VL8B-R1024-C4-DET-GPU12346-FIRSTSTEP/comparison_summary.json
+6e72053891db350c2863bbf3508188a3740d13d176cd86eaf0b56261d3875938  /data01/LWX/relax-task21/comparisons/FORMAL-FINAL8-Q3VL8B-R1024-C4-DET-GPU12346-FIRSTSTEP/console-final.log
+025300b86959bc7c0dfb7dd941989423be873a3b8b06452c1697a8886da73018  docs/assets/task21-hybrid-mm-forward-pipeline/formal-paired-results.csv
+da9b541bc2fbd26d5a4730eb9096d3b152d9470d2dd81c03f5104b4a62d03306  docs/assets/task21-hybrid-mm-forward-pipeline/paired_run_summary.csv
+7408c899219b63619335ebf42f6374421df46a664b7d22c4a45ed03567b38f5e  docs/assets/task21-hybrid-mm-forward-pipeline/task21_correctness_quality.png
+38c6f272370cb0e12abb1d9a403596681ae4f150f30624e35d0b85943adf8c91  docs/assets/task21-hybrid-mm-forward-pipeline/task21_gpu_util_vram.png
+e1256d0fdd7b5b22f81411ba8e20224f26f58e0955142352a37b492aa9b6a708  docs/assets/task21-hybrid-mm-forward-pipeline/task21_phase1_overlap.png
+bbad23dc4059ffc0610786b4d042b9af37558a8753cb298e5bf5429e44011134  docs/assets/task21-hybrid-mm-forward-pipeline/task21_step_throughput.png
+c989433666edfcecbe5d0761c0e03b41d104772a232c15fb6abdfe5c6699bd0a  docs/assets/task21-hybrid-mm-forward-pipeline/task21_window_summary.png
+ea8cfbd0958c16f54ef799aecf9a9bc45601b7861e8dd01ba7dd8a8ed033eb64  docs/assets/task21-hybrid-mm-forward-pipeline/window_speedup_summary.csv
@@ -0,0 +1,5 @@
+seed,baseline_step_token_per_s,experiment_step_token_per_s,throughput_speedup_percent,baseline_phase1_s,experiment_phase1_s,phase1_reduction_percent,baseline_step_time_s,experiment_step_time_s,baseline_peak_vram_mib,experiment_peak_vram_mib,raw_reward,truncated_ratio,baseline_loss,experiment_loss,baseline_grad_norm,experiment_grad_norm
+20260811,597.4306030273438,658.458984375,10.215141480601897,169.09609985351562,136.78187561035156,19.10997608528948,348.0404357910156,315.78277587890625,47380.0,47140.0,0.296875,0.1875,0.013692490756511688,0.013677351176738739,0.4145539104938507,0.4117335081100464
+20260812,576.8270874023438,647.2715454101562,12.21240464365998,171.17022705078125,136.36746215820312,20.33225374074731,353.73858642578125,315.24017333984375,46818.0,46898.0,0.41796875,0.10546875,0.012462313286960125,0.012400214560329914,0.44756120443344116,0.4443718492984772
+20260813,559.2531127929688,630.8582763671875,12.803713012273654,162.420166015625,124.19634246826172,23.53391483646563,342.5908508300781,303.7052917480469,46912.0,47104.0,0.35546875,0.11328125,-0.021680761128664017,-0.021680761128664017,0.6372919082641602,0.6365648508071899
+20260814,569.1944580078125,642.1531372070312,12.817882917302992,158.83006286621094,124.18817901611328,21.81065928260567,324.609619140625,287.7288818359375,46916.0,46824.0,0.38671875,0.1640625,0.004201771225780249,0.004198123700916767,1.0032663345336914,0.4775027334690094
@@ -0,0 +1,5 @@
+actor_fetch_samples:baseline,actor_fetch_samples:experiment,actor_fetch_samples:relative_delta,actor_multimodal_tensor_bytes:baseline,actor_multimodal_tensor_bytes:experiment,actor_multimodal_tensor_bytes:relative_delta,actor_response_tokens:baseline,actor_response_tokens:experiment,actor_response_tokens:relative_delta,actor_total_tokens:baseline,actor_total_tokens:experiment,actor_total_tokens:relative_delta,nvml_peak_memory_mib:baseline,nvml_peak_memory_mib:delta,nvml_peak_memory_mib:experiment,perf/hybrid_phase1_time:baseline,perf/hybrid_phase1_time:experiment,perf/hybrid_phase1_time:improvement,perf/step_resp_token_per_s:baseline,perf/step_resp_token_per_s:experiment,perf/step_resp_token_per_s:improvement,perf/step_time:baseline,perf/step_time:experiment,perf/step_time:improvement,perf/step_time:p95_baseline,perf/step_time:p95_experiment,perf/step_time:p95_regression,perf/step_token_per_s:baseline,perf/step_token_per_s:experiment,perf/step_token_per_s:improvement,perf/wall_clock_samples_per_s:baseline,perf/wall_clock_samples_per_s:experiment,perf/wall_clock_samples_per_s:improvement,producer_lead_at_first_forward:baseline,producer_lead_at_first_forward:delta,producer_lead_at_first_forward:experiment,rollout/raw_reward:baseline,rollout/raw_reward:delta,rollout/raw_reward:experiment,rollout/truncated_ratio:baseline,rollout/truncated_ratio:delta,rollout/truncated_ratio:experiment,seed,train/grad_norm:baseline,train/grad_norm:delta,train/grad_norm:experiment,train/loss:baseline,train/loss:delta,train/loss:experiment,train/pg_clipfrac:baseline,train/pg_clipfrac:delta,train/pg_clipfrac:experiment,train/ppo_kl:baseline,train/ppo_kl:delta,train/ppo_kl:experiment
+256,256,0.0,1327110144,1327110144,0.0,119338,119338,0.0,207930,207930,0.0,47380.0,-240.0,47140.0,169.09609985351562,136.78187561035156,0.19109976085289482,342.8854675292969,377.91168212890625,0.10215135348836757,348.0404357910156,315.78277587890625,0.09268365567579862,348.0404357910156,315.78277587890625,-0.09268365567579862,597.4306030273438,658.458984375,0.10215141480601897,0.7355467172030487,0.8106838610417711,0.10215142299109847,0.0,0.0,0.0,0.296875,0.0,0.296875,0.1875,0.0,0.1875,20260811,0.4145539104938507,-0.0028204023838043213,0.4117335081100464,0.013692490756511688,-1.5139579772949219e-05,0.013677351176738739,0.0,0.0,0.0,-2.8720626137762606e-12,2.8720626137762606e-12,0.0
+256,256,0.0,1468078080,1468078080,0.0,109958,109958,0.0,204046,204046,0.0,46818.0,80.0,46898.0,171.17022705078125,136.36746215820312,0.20332253740747308,310.8453674316406,348.80706787109375,0.12212406687322264,353.73858642578125,315.24017333984375,0.10883294772823704,353.73858642578125,315.24017333984375,-0.10883294772823704,576.8270874023438,647.2715454101562,0.1221240464365998,0.7236982614383574,0.8120792387841379,0.12212407028603667,0.0,0.0,0.0,0.41796875,0.0,0.41796875,0.10546875,0.0,0.10546875,20260812,0.44756120443344116,-0.0031893551349639893,0.4443718492984772,0.012462313286960125,-6.209872663021088e-05,0.012400214560329914,0.0,0.0,0.0,0.0,0.0,0.0
+256,256,0.0,1581717504,1581717504,0.0,94331,94331,0.0,191595,191595,0.0,46912.0,192.0,47104.0,162.420166015625,124.19634246826172,0.2353391483646563,275.345947265625,310.6004638671875,0.12803717269734372,342.5908508300781,303.7052917480469,0.1135043711407171,342.5908508300781,303.7052917480469,-0.1135043711407171,559.2531127929688,630.8582763671875,0.12803713012273654,0.7472470422946981,0.842922421688908,0.1280371469927848,0.0,0.0,0.0,0.35546875,0.0,0.35546875,0.11328125,0.0,0.11328125,20260813,0.6372919082641602,-0.0007270574569702148,0.6365648508071899,-0.021680761128664017,0.0,-0.021680761128664017,0.0,0.0,0.0,-2.2905178188176167e-12,2.2905178188176167e-12,0.0
+256,256,0.0,958666752,958666752,0.0,109918,109918,0.0,184766,184766,0.0,46916.0,-92.0,46824.0,158.83006286621094,124.18817901611328,0.2181065928260567,338.615966796875,382.0193786621094,0.12817886963750502,324.609619140625,287.7288818359375,0.11361566364646236,324.609619140625,287.7288818359375,-0.11361566364646236,569.1944580078125,642.1531372070312,0.12817882917302992,0.7886395993986166,0.8897264618223858,0.12817878090429868,0.0,0.0,0.0,0.38671875,0.0,0.38671875,0.1640625,0.0,0.1640625,20260814,1.0032663345336914,-0.525763601064682,0.4775027334690094,0.004201771225780249,-3.6475248634815216e-06,0.004198123700916767,0.0,0.0,0.0,-2.20278361523496e-11,2.3654630120981124e-11,1.6267939686315236e-12
@@ -0,0 +1,3 @@
+candidate,nvml/peak_memory_used_mib:candidate,nvml/peak_memory_used_mib:reference,nvml/steady_mean_gpu_util_percent:candidate,nvml/steady_mean_gpu_util_percent:reference,perf/step_resp_token_per_s:candidate,perf/step_resp_token_per_s:improvement,perf/step_resp_token_per_s:reference,perf/step_time:candidate,perf/step_time:reduction,perf/step_time:reference,perf/step_token_per_s:candidate,perf/step_token_per_s:improvement,perf/step_token_per_s:reference,perf/wall_clock_samples_per_s:candidate,perf/wall_clock_samples_per_s:improvement,perf/wall_clock_samples_per_s:reference,reference,rollout/image_count/mean:candidate,rollout/image_count/mean:delta,rollout/image_count/mean:reference,rollout/raw_reward:candidate,rollout/raw_reward:delta,rollout/raw_reward:reference,seed,train/grad_norm:candidate,train/grad_norm:delta,train/grad_norm:reference,train/loss:candidate,train/loss:delta,train/loss:reference,train/ppo_kl:candidate,train/ppo_kl:delta,train/ppo_kl:reference
+P+R,48393.0,46741.0,83.92935579272213,77.59728409534975,799.7521633699627,0.41034894101985686,567.0597822349148,129.96772181193035,0.24067978916955646,171.16325887044272,1559.7070045237251,0.36325051708314526,1144.108866990144,1.9697198383645176,0.3169674476415332,1.4956480829438525,B,1.0,0.0,1.0,0.41822916666666665,-0.00911458333333337,0.42734375,20260816,0.725117082397143,0.0517819702625274,0.6733351121346156,0.016472606247407384,-0.0018613979414415852,0.01833400418884897,0.0,1.3656563582533274e-13,-1.3656563582533274e-13
+P+R,48335.0,47354.0,85.42939171473519,74.53677597955706,804.4803081955772,0.2334115741806173,652.2399538288842,127.61269302368164,0.1883688675515388,157.22991383870442,1571.6453641874045,0.23276455720934708,1274.894995152354,2.006070038444318,0.2320867941367446,1.62818889707349,B,1.0,0.0,1.0,0.4158854166666667,-0.0003906249999999778,0.41627604166666665,20260817,2.1450912207365036,1.4769096086422602,0.6681816120942433,0.021927844686433672,0.003910016796241204,0.01801782789019247,0.0,3.738612998103903e-12,-3.738612998103903e-12
@@ -0,0 +1,3 @@
+candidate,nvml/peak_memory_used_mib:candidate,nvml/peak_memory_used_mib:reference,nvml/steady_mean_gpu_util_percent:candidate,nvml/steady_mean_gpu_util_percent:reference,perf/step_resp_token_per_s:candidate,perf/step_resp_token_per_s:improvement,perf/step_resp_token_per_s:reference,perf/step_time:candidate,perf/step_time:reduction,perf/step_time:reference,perf/step_token_per_s:candidate,perf/step_token_per_s:improvement,perf/step_token_per_s:reference,perf/wall_clock_samples_per_s:candidate,perf/wall_clock_samples_per_s:improvement,perf/wall_clock_samples_per_s:reference,reference,rollout/image_count/mean:candidate,rollout/image_count/mean:delta,rollout/image_count/mean:reference,rollout/raw_reward:candidate,rollout/raw_reward:delta,rollout/raw_reward:reference,seed,train/grad_norm:candidate,train/grad_norm:delta,train/grad_norm:reference,train/loss:candidate,train/loss:delta,train/loss:reference,train/ppo_kl:candidate,train/ppo_kl:delta,train/ppo_kl:reference
+P+R,48393.0,46725.0,83.92935579272213,78.87460953143773,799.7521633699627,0.29281391681209845,618.6135165856209,129.96772181193035,0.2207435029812811,166.78426462809244,1559.7070045237251,0.28814826316148245,1210.8132651561089,1.9697198383645176,0.28327451080074684,1.534916981352211,P,1.0,0.0,1.0,0.41822916666666665,-0.0013020833333333703,0.41953125,20260816,0.725117082397143,0.057410457730293185,0.6677066246668498,0.016472606247407384,0.006813187155057675,0.009659419092349708,0.0,0.0,0.0
+P+R,48335.0,47779.0,85.42939171473519,79.27988663017983,804.4803081955772,0.3154239474112164,611.574929724225,127.61269302368164,0.2536193243751721,170.9753443400065,1571.6453641874045,0.32721025428832173,1184.1721077043321,2.006070038444318,0.3397988890358883,1.4972919106447946,P,1.0,0.0,1.0,0.4158854166666667,0.007552083333333348,0.4083333333333333,20260817,2.1450912207365036,1.312765329082807,0.8323258916536967,0.021927844686433672,0.00397575895379608,0.017952085732637593,0.0,6.887873663538197e-12,-6.887873663538197e-12