v0.6.0

Overview

⭐️ Highlights

Trinity-RFT v0.6.0 adds SGLang support, optimizes fully asynchronous parameter synchronization and scheduling to reduce bubbles, improves MoE training stability, and upgrades the veRL backend to v0.8.0.

Explorer

1. Add SGLang inference engine support, including SGLang OpenAI API responses with prompt_token_ids, token_ids, and routed_experts.
2. Upgrade vLLM to v0.23.0 and SGLang to v0.5.13.
3. Refactor rollout model creation and management to support cross-node deployment of rollout models.
4. Optimize fully asynchronous parameter synchronization with partial rollouts.
5. Refactor weight synchronization with vLLM's WeightTransferEngine.
6. Improve MoE rollout support by returning routed experts from vLLM/SGLang OpenAI API paths.
7. Add data, pipeline, and expert parallel settings for rollout models to improve rollout performance.
8. Enhance multi-turn and multi-modal rollout support.
9. Add rollout performance benchmarking tools.

Trainer

1. Upgrade veRL to v0.8.0 while keeping v0.7.1 supported.
2. Improve Qwen3.5 compatibility with FSDP and Megatron backends.
3. Improve MoE model training stability and reduce OOM risk in FSDP/Megatron training paths.

Buffer

1. Refactor Buffer with an asynchronous implementation.
2. Update the SQL schema and add filtering support to the Experience Viewer.
3. Optimize the Experience Viewer.

Others

1. Add Connect the Dots (CoD) example.
2. Add InterCode-SQL example/benchmark.
3. Add the GiGPO and DAPO algorithms.
4. Improve Docker and remote GPU unit test workflows.

🚨 Breaking Changes

Due to database schema changes, the database files in the existing checkpoint folder cannot be recognized by the new version. To continue running previous experiments, please remove the corresponding .db files (typically located in the checkpoint_root_dir/project/name/buffer directory).

What's Changed

• Fix opd key error typo by @garyzhang99 in #527
• Update vllm to v0.19.1 by @pan-x-c in #529
• Refactor WorkflowRuner and Scheduler to handling partially completed tasks by @pan-x-c in #530
• Optimize Explorer by @pan-x-c in #531
• Add Performance tools by @pan-x-c in #532
• Fix: skip trainer config validation in bench mode by @shiweijiezero in #535
• Support SGLang Inference Engine by @pan-x-c in #533
• Fix(trainer,checkpoint,explorer): resolve Qwen3.5/vLLM compatibility bugs and harden runtime paths by @chenyushuo in #534
• SGLang OpenAI API Server returns prompt_token_ids and token_ids by @pan-x-c in #536
• Refactor Inference Model Creation and Management by @pan-x-c in #537
• Update vLLM to 0.21.0, SGLang to 0.5.12 by @pan-x-c in #538
• Train Qwen3.5 with megatron by @pan-x-c in #540
• fix(trainer): add torch.cuda.empty_cache() after FSDP update_actor by @Leon-Algo in #541
• Bug fix on update_weight for MoE model. by @chenyushuo in #543
• Support return routed experts for MoE Models by @pan-x-c in #542
• Refactor Monitor Metric Aggregation by @pan-x-c in #539
• Enhance multi-turn multi-modal sft by @chenyushuo in #545
• Update Megatron by @pan-x-c in #546
• fix(trainer): add empty_cache() after compute_ref_log_prob to prevent OOM by @Leon-Algo in #548
• Fix Megatron OOM when calculating entropy by @pan-x-c in #550
• Refactor Docker environment and unit test workflow by @pan-x-c in #554
• Fix Tinker API by @pan-x-c in #553
• Enhance multi-modal for vllm model by @chenyushuo in #547
• Update SQL Schema and Add Filter to Experience Viewer by @pan-x-c in #555
• DAPO Integration in Trinity: Algorithm Registration, Dynamic Sampling Pipeline, Overlong Handling, and Reviewer-Focused Docs by @artaasd95 in #552
• fix mm_utils by @chenyushuo in #558
• fix(sglang): fix shutdown Gloo connection error by @MengsD in #559
• docs: use TINKER_API_KEY in tinker docs by @pan-x-c with @Copilot in #564
• fix: propagate the optional 'group' config field across paths, namespaces & docs by @shiweijiezero in #562
• fix(sglang): fix sglang oom (Qwen3.5-35B-A3B) by @MengsD in #561
• Return routed experts for vLLM OpenAI API by @pan-x-c in #565
• Refactor Buffer into async implementation by @pan-x-c in #557
• explorer: fix glibc getenv SIGSEGV when creating runners at startup by @shiweijiezero in #567
• fix(example): fix dapo-math reward issue by @MengsD in #566
• Add the GiGPO algorithm [FEATURE] by @artaasd95 in #560
• fix(model): add routed_experts to dummy experiences by @MengsD in #568
• Optimize Docker unittest tool by @pan-x-c in #569
• Run unittest on remote GPU server by @pan-x-c in #571
• Update VERL to 0.8.0 by @pan-x-c in #570
• Update news by @yanxi-chen in #574
• Add InterCode-SQL example and benchmark by @PatrickG1014 in #572
• Fix verl FSDP can't save checkpoints when offloading is enable by @pan-x-c in #577
• Refactor weight synchronization with vLLM's WeightTransferEngine by @pan-x-c in #578
• Add data/pipeline/expert parallel settings for rollout model by @chenyushuo in #576
• Add fully async mode without any bubble by @pan-x-c in #579
• Optimize viewer and add tests by @pan-x-c in #581
• Add Connect the Dots (CoD) overview under examples/research_cod by @shiweijiezero in #583
• Update Connect the Dots (CoD) overview README and figures by @shiweijiezero in #585
• Update vLLM to v0.23.0, SGLang to v0.5.13 by @pan-x-c in #586
• Fix qwen3.5 for verl==0.8.0 by @chenyushuo in #589
• trinity convert supports converting multiple steps by @pan-x-c in #587
• Add CPT Algorithm && Formatter by @chenyushuo in #592
• fix(buffer): reshuffle with correct epoch in ShuffleSelector by @MengsD in #593
• patch _build_module by @chenyushuo in #595
• Release v0.6.0 by @pan-x-c in #594

New Contributors

• @Leon-Algo made their first contribution in #541
• @artaasd95 made their first contribution in #552
• @MengsD made their first contribution in #559
• @PatrickG1014 made their first contribution in #572

Full Changelog: v0.5.2...v0.6.0

v0.5.2

Overview

1. Added support for training Qwen3.5 series models.
2. Upgraded core dependencies: transformers to v5.3.0, verl to v0.7.1, and vLLM to v0.19.0.
3. Enabled the use of auxiliary models within the experience pipeline.
4. Added support for stream mode in the rollout API.
5. Enabled integration with external APIs (e.g., OpenAI) during training and benchmarking.
6. Added experience data visualization capabilities.
7. Optimized serialization and deserialization of experience data.

What's Changed

• Update Readme and docs by @hiyuchang in #508
• Support vLLM v0.16.0 by @pan-x-c in #510
• [Example] Clip_B and Clip_V from entropy dynamics by @hiyuchang in #509
• Support Transformers V5 by @chenyushuo in #512
• Using auxiliary models in experience pipeline & OpenAI API supports stream mode by @pan-x-c in #513
• Optimize serialize / deserialize of Experience by @pan-x-c in #514
• Support Qwen3.5 by @chenyushuo in #515
• Bug fix: vlm model with seq parallel by @chenyushuo in #517
• Add SQL Buffer Viewer to CLI by @pan-x-c in #520
• Update news in March by @hiyuchang in #522
• Rename TaskSelector to DataSelector. by @chenyushuo in #519
• Add openai_api model for explorer by @hiyuchang in #523
• feat: upgrade veRL to v0.7.1 with trainer file migration by @chenyushuo in #525
• Fix: dummy response text when prompt truncation is activated by @yanxi-chen in #524
• Bumping version to 0.5.2 by @pan-x-c in #526

Full Changelog: v0.5.1...v0.5.2

v0.5.1

Overview

1. Enhanced support for multi-modal models (including Qwen2.5 VL, Qwen3 VL and Kimi-VL-A3B-Thinking series)
2. Refactored trinity command line interface using typer
3. Added a log management tool and fixed bugs in the logging system.
4. Added Jensen-Shannon Divergence for on-policy distillation.
5. Fixed bugs in model weight synchronization and over-rollout.

What's Changed

• Update algorithm List in README by @pan-x-c in #498
• Refactor Launcher with Typer by @pan-x-c in #502
• Fix memory resume by @hiyuchang in #505
• Fix logger in debug mode by @pan-x-c in #504
• Fix Logger in Workflow by @pan-x-c in #506
• Fix over_rollout by @luyi256 in #500
• Enhance support for VL models by @chenyushuo in #501
• jsd implement for opd by @kokolerk in #499
• Add log manager to track experiement logs by @pan-x-c in #507

New Contributors

• @luyi256 made their first contribution in #500

Full Changelog: v0.5.0...v0.5.1

v0.5.0

Overview

⭐️ Highlights

1. Added colocate mode for single-GPU deployment.
2. Refactored the Synchronizer and introduced a new model weight synchronization style: TRAINER_DRIVEN.
3. Automatically provide parallelism setting suggestions based on cluster resources.

Explorer

1. Added Multi-Threaded / Asynchronous Workflow Runner.
2. Supported colocating the Rollout Model with the Trainer in single-GPU scenarios.
3. Refined explorer-side metrics computation.
4. Supported vLLM up to v0.15.1 (v0.10.2 remains supported).
5. Fixed bugs related to multi-turn truncation.

Trainer

1. Supported sharing GPU with Explorer in single-GPU scenarios.

Buffer

1. Moved Selector into TaskFileReader.

Others

1. Added a Huggingface-format checkpoint conversion tool to the trinity CLI.
2. Refactored the configuration validation system.
3. Provided parallelism setting suggestions to help avoid OOM issues.

What's Changed

• Update Doc links to agentscope-ai by @chenyushuo in #480
• Add convert mode to launcher by @chenyushuo in #483
• Simplify InferenceModel by @chenyushuo in #485
• Move Selector into TaskFileReader by @chenyushuo in #486
• Clear and fix metrics computation by @hiyuchang in #484
• feature(workflows): added a agentic workflow for sudoku by @AdnanQureshi3 in #481
• Add config_validator.py and refactor config by @chenyushuo in #487
• Support vLLM v0.14.1 by @pan-x-c in #488
• Fix multi-turn truncation by @hiyuchang in #489
• Add Thread / Async Workflow Runner by @pan-x-c in #491
• Add GPUMemoryValidator by @chenyushuo in #490
• Fix can_repeat in MultiTurnWorkflow and Metrics Reference Doc by @hiyuchang in #492
• Update Tuner Interface with logger and TuFT support by @pan-x-c in #493
• Support colocating Explorer and Training in a Single GPU by @pan-x-c in #495
• Add new sync style TRAINER_DRIVEN. by @chenyushuo in #494
• Release v0.5.0 by @pan-x-c in #496

Full Changelog: v0.4.1...v0.5.0

v0.4.1

Overview

⭐️ Trinity-RFT repository has moved to agentscope-ai organization. You can find our new repository here🔗 and documentation here📚

This is a minor release that includes several bug fixes, feature improvements, and dependency upgrades.

Explorer

1. Add OpenAI API support for Tinker backend, now users can use tinker backend to run agentic RL examples.
2. Enhance the AgentScope Workflow Adapter to support features of AgentScope Tuner.

Trainer

1. Update veRL to v0.7.0, which includes various performance improvements and bug fixes.
2. Fix bugs in multi-stage resume and last checkpoint saving.
3. Avoid preserving checkpoints for weight synchronization purpose to reduce storage usage.

Buffer

1. Fix batch size mismatch issue in SQL buffer.

Others

1. Introducing R3L, a systematic reflect-then-retry RL mechanism with efficient language-guided exploration and stable off-policy learning. [github repo], [paper]
2. Improve documentation.

🚨 Breaking Changes

veRL has been upgraded to v0.7.0, and v0.5.0 is no longer supported.

What's Changed

• Bug fix in multi stage resume by @chenyushuo in #462
• Bug fix in last checkpoint save. by @chenyushuo in #460
• Add model_name for auxiliary models by @hiyuchang in #461
• Improve contributing.md by @yxdyc in #464
• Fix learn_to_ask and tinker example. by @chenyushuo in #466
• Add default batch size for SQLReader by @chenyushuo in #467
• Enhance AgentScope Workflow Adapter by @pan-x-c in #465
• Fix process_messages_to_experience in MultiTurnWorkflow by @hiyuchang in #468
• [Doc] Update docker install guide by @pan-x-c in #469
• Support dynamic lora updating by @pan-x-c in #472
• Add low-quality experience filter operator #470 by @AdnanQureshi3 in #473
• Add openai client support for tinker backend by @chenyushuo in #475
• Improve documentation for CPU-only users and first-time contributors by @P09s in #474
• Remove checkpoints saved for sync purpose by @pan-x-c in #476
• Add R3L to readme and doc by @yanxi-chen in #477
• Update veRL to 0.7.0 by @chenyushuo in #471
• Support add / remove lora adapter by @pan-x-c in #478
• Rename warmup_style to lr_scheduler_type by @chenyushuo in #479

New Contributors

• @AdnanQureshi3 made their first contribution in #473
• @P09s made their first contribution in #474

Full Changelog: v0.4.0...v0.4.1

v0.4.0

Overview

⭐️ Highlights

Add Tinker backend for users without GPUs to leverage Trinity-RFT. See example for more details.

Explorer

1. Add Tinker SamplingClient backend for users without GPUs.
2. Support vLLM v0.12.0 (v0.10.2 ~ v0.11.0 are still supported).
3. Add a tinker-compatible sample API to the vLLM backend.
4. Enhance serve mode for online RL.
5. Fix several bugs in the vLLM OpenAI API.

Trainer

1. Add Tinker TrainingClient backend for users without GPUs.
2. Add a switch in PPOPolicyLossFn to ignore explorer-generated logprobs.

Buffer

1. Support staleness control, which mitigates the negative effects of excessively off-policy data.
2. Add a Streamlit viewer to visualize the experience data.

Others

1. Add benchmark comparisons with veRL and rLLM.
2. Refactor registration system to avoid loading all modules during initialization.
3. Add algorithms: SAPO, on-policy distillation.
4. Enhance debug mode; add --module viewer to visualize experience data generated during debugging.
5. Add SwanLab monitor.
6. Add tutorial on aligning configuration with veRL.
7. Add tutorial on choosing model context length based on GPU and model size.
8. Optimize README and Sphinx docs.

🚨 Breaking Changes

1. The schema of SQL experience buffer is changed. Experience data saved in previous version cannot be used.
2. The registration system has been refactored. Developers no longer need to use @REGISTRY.register_module to register modules. See Developer Guide for details.
3. Tinker requires Python >= 3.11. (For users who do not use tinker, Python 3.10 is still supported)
4. vLLM 0.12.0 requires CUDA >= 12.9. (For users using vLLM 0.11.0 or lower, CUDA 12.8 is still supported)
5. Refactor SampleStrategy, add kwargs to its inputs, and change the output type from Experiences to List[Experience].
6. Experiences (not Experience) is going to be deprecated

What's Changed

• Fix docker build action by @pan-x-c in #415
• Add benchmark scripts for Guru-Math by @chenyushuo in #417
• Add bench results for frozenlake and alfworld by @hiyuchang in #416
• Add corrected kl with importance sampling by @garyzhang99 in #419
• Add SAPO algorithm by @garyzhang99 in #422
• Enhance debug mode by @pan-x-c in #421
• Update README with supported algorithm and news by @hiyuchang in #424
• Fix openai api history by @pan-x-c in #428
• Add Experience Viewer by @pan-x-c in #427
• Add sequence mask for grpo by @garyzhang99 in #420
• Fix the mismatch between vLLM OpenAI API and vLLM generate by @pan-x-c in #431
• Add Guru-Math report. by @chenyushuo in #432
• Refactor and Merge several compute_score functions by @hiyuchang in #430
• Add scripts to search context length capacity on given settings. by @chenyushuo in #423
• [Doc] Align with verl by @hiyuchang in #433
• [Doc] add a new page with example list from the dataset perspective by @HYLcool in #434
• General Multi-turn FrozenLake by @pan-x-c in #429
• Fix sft warmup yaml by @hiyuchang in #435
• Update Benchmark doc and FAQ by @pan-x-c in #436
• Fix typo in doc and check selector by @hiyuchang in #437
• Add report for gsm8k alignment experiment. by @chenyushuo in #439
• Alfworld Concatenated Multi-turn RFT SFT format AND settings. by @kokolerk in #442
• BOTS reference evaluation results collection by @ShenQianli in #440
• add fallback_to_policy_gradient option by @binary-husky in #443
• Support vLLM v0.12.0 by @pan-x-c in #438
• Optimize registry by @hiyuchang in #441
• Add staleness control by @chenyushuo in #445
• Implement On-Policy Distillation by @garyzhang99 in #444
• Fix some typos by @hiyuchang in #447
• Explorer API collects feedback from agent applications by @pan-x-c in #295
• Check Explorer GPU Number by @hiyuchang in #453
• impl swanlab monitor by @binary-husky in #450
• Pre release 0.4.0 by @pan-x-c in #455
• improve ray stat by @binary-husky in #454
• Implement Tinker compatible sample API by @pan-x-c in #456
• Enhance AgentScope Workflow Adapter by @pan-x-c in #457
• Add GSPO-style REC variant by @yanxi-chen in #380
• Add tinker backend. by @chenyushuo in #448
• Release v0.4.0 by @pan-x-c in #459

New Contributors

• @kokolerk made their first contribution in #442
• @binary-husky made their first contribution in #443

Full Changelog: v0.3.3...v0.4.0

v0.3.3

Overview

This is a bug fix update that addresses many bugs present in versions 0.3. We recommend all users currently using versions 0.3.0/0.3.1/0.3.2 upgrade to this new version.

Explorer

1. Over Rollout: This mechanism allows the explorer to proceed with fewer tasks than the full batch size. It effectively increases throughput in scenarios where some tasks take significantly longer to complete than others.
2. Make the prompt truncation configurable
3. Fix logprobs calculation when temperature is not 1.0
4. Support rope scaling
5. Support loading custom chat template files
6. Support recording workflow running status
7. Optimize the aggregation of workflow metrics

Trainer

1. Update ppo policy loss calculation
2. Fix loss aggregation for kl loss and entropy loss
3. Optimize Trainer checkpoints saving

Buffer

1. Support registering custom Task Reader
2. Support token-level reward
3. Fix some bugs

Others

1. Add "Learn to Ask" and "Frozen Lake" examples
2. Update Dockerfile

What's Changed

• Fix doc/config issues about prompt/response/sequence lengths by @yanxi-chen in #370
• Limit the number of historical document versions by @pan-x-c in #372
• Add learn_to_ask example by @chenyushuo in #356
• Update main readme by @yanxi-chen in #374
• Fix default_sampling_params and simple_workflow by @chenyushuo in #373
• Fix metrics in trainer by @chenyushuo in #381
• Minor updates for BOTS example by @HYLcool in #385
• Fix vLLM prompt logprobs calculation by @pan-x-c in #384
• Fix alfworld dataset loading to use correct train/test split by @shiweijiezero in #378
• Bug fix when set total_steps. by @chenyushuo in #386
• Add more useful options in OptimizerConfig by @garyzhang99 in #371
• Add rope_scaling and rope_theta to config by @chenyushuo in #390
• Add chat_template_path and trust_remote_code by @hiyuchang in #379
• Add loss_agg_mode for kl and entropy_loss by @pan-x-c in #388
• Bug fix in benchmark ckpt loading and megatron hf save by @chenyushuo in #392
• Support Over Rollout by @pan-x-c in #376
• Make Alfworld Rollout Parallel by @hiyuchang in #393
• Make buffer reader registerable by @pan-x-c in #395
• [Example] Frozen_Lake by @hiyuchang in #375
• Add token_level_reward to Experience by @chenyushuo in #404
• Support recording workflow running status by @pan-x-c in #397
• Bug fix for trainer_state saving by @chenyushuo in #408
• Fix dynamic timeout by @pan-x-c in #409
• Typo fix by @chenyushuo in #411
• Fix repeat times in evaluation by @chenyushuo in #410
• Add truncate_status to experience by @hiyuchang in #407
• Add trainer_strategy and save_hf_checkpoint by @pan-x-c in #412
• Save bf16 model by @chenyushuo in #414
• Add Docker image build and push workflow by @pan-x-c in #413

Full Changelog: v0.3.2...v0.3.3

v0.3.2

Overview

This is a bug fix update that addresses many bugs present in versions 0.3. We recommend all users currently using versions 0.3.0 and 0.3.1 upgrade to this new version.

Buffer

1. Support task scheduler and selector in task dataset
2. Add BOTS: Online RL task selection for efficient LLM fine-tuning (paper).
3. Extract the PriorityFunction of the replay buffer as a customizable module.

Explorer

1. Update vLLM to v0.11.0
2. Fix logprobs, top_k, top_p, temperature mismatch when using vLLM's OpenAI API server
3. Fix torch cache conflicts when enforce_eager is False
4. Simplify Workflow Inferface

Other Modules

1. Optimize monitor metrics organization
2. Optimize and simplify Config and config manager
3. Add more algorithm and examples

What's Changed

• Simplify trainer config by @hiyuchang in #329
• Simplify Workflow Interface by @pan-x-c in #330
• Update off-policy RFT documentation by @yanxi-chen in #335
• Simplify Buffer and Explorer Config by @pan-x-c in #333
• Update rec example by @yanxi-chen in #337
• Sync the config manager with the latest codebase by @hiyuchang in #332
• Add taskset scheduler by @chenyushuo in #326
• Split Storage Config by @pan-x-c in #338
• Fix some metrics by @hiyuchang in #339
• Refactor Priority Function by @chenyushuo in #344
• Bug fix in unittest by @chenyushuo in #346
• [Example] Mix algorithm with VLM by @hiyuchang in #342
• Add example for experience replay by @yanxi-chen in #345
• Fix Bug: group_rewards Un-assigned value by @shiweijiezero in #343
• Fix openai client logprobs calculation by @pan-x-c in #347
• Fix openai API server setup by @pan-x-c in #348
• Update readme by @yanxi-chen in #349
• Update vLLM to 0.11 by @pan-x-c in #350
• bug fix in plugin_loader by @chenyushuo in #354
• Fix/email search by @chenyushuo in #351
• Enable System Metrics Recording in MLFLOW by @hiyuchang in #362
• Add generation args to ModelConfig by @chenyushuo in #357
• Add std_thresholdoption to StepWiseGRPOAdvantageFn, to filter out zero-grad group samples. by @garyzhang99 in #363
• Fix microbatch loss scale when loss_agg_mode is "token-mean" by @yanxi-chen in #336
• Add weight_decay in OptimizerConfig by @hiyuchang in #364
• Fix taskset and eval_tasksets path by @hiyuchang in #367
• Fix torch compile cache dir conflicts when enforce_eager=False by @pan-x-c in #368
• Examples for BOTS by @ShenQianli in #353
• Release Trinity-RFT 0.3.2 by @pan-x-c in #369

New Contributors

• @ShenQianli made their first contribution in #353

Full Changelog: v0.3.1...v0.3.2

v0.3.1

Overview

Agentic RL

1. Add more agentic RL examples using agent frameworks (e.g. AgentScope)
2. Provide Debug mode for workflow developers
3. Add examples for RL in non-verifiable domain: trainable RULER reward, rubric-as-reward

Framework Enhancement

1. Support multi-stage training
2. Support using environment variables in configuration file
3. Support LoRA
4. Enhance checkpoint saving process
5. Enhance experience replay mechanism for priority queue buffer
6. Add algorithms: group-relative REINFORCE variants
7. Update vLLM to 0.10.2

Documentation

1. Add Chinese Docs
2. Rewrite Developer Guide

What's Changed

• Support new task ordering methods by @HYLcool in #265
• Fix problem in get_openr1_dataset.py in chord example by @garyzhang99 in #266
• Replace /PATH/TO to ${oc.env:TRINITY_XXX} by @chenyushuo in #267
• Support multi-stage training by @pan-x-c in #268
• [Example] Policy model as its own reward model by @hiyuchang in #270
• Dev/update agentscope react example version by @garyzhang99 in #275
• Normalize Trainer by @pan-x-c in #271
• Refactor workflow to async workflow by @chenyushuo in #276
• Support Chinese Docs by @hiyuchang in #277
• Support vLLM 0.10.2 by @pan-x-c in #278
• AgentScopeV1.0 WebSearch Workflow(simple react + search api) by @garyzhang99 in #264
• Add enable_activation_offload configuration option by @nkkarpov in #281
• Update data-juicer version in toml by @chenyushuo in #286
• Improvement in config by @chenyushuo in #288
• Add loss-agg-mode for policy loss by @hiyuchang in #294
• Explorer provides OpenAI API compatible inference service by @pan-x-c in #289
• Fix absmethod in workflow by @chenyushuo in #297
• Add LoRA mode by @hiyuchang in #291
• Enhance support for multi-modal models by @pan-x-c in #298
• Group-relative REINFORCE Families by @yaochaorui in #292
• Optimize Installation and Development Doc by @pan-x-c in #301
• Update Readme by @pan-x-c in #302
• Refactor checkpoint save by @chenyushuo in #299
• Refactor AgentScope ReAct Agent workflow example by @pan-x-c in #303
• Update Workflow Developement Tutorial by @pan-x-c in #310
• Update chord with tooluse example by @garyzhang99 in #313
• Debug Mode for workflow developers by @pan-x-c in #314
• [BUGFIX]Fix tokenizer bug when getting action masks with enable_thinking arguments by @garyzhang99 in #316
• Add task_count by @hiyuchang in #307
• Update FAQ by @hiyuchang in #320
• Refactoring EmailSearchWorkflow and EmailSearchAgent to adapt to the latest version of Agentscope. by @chenyushuo in #321
• Non-verifiable Medicine QA Task by @hiyuchang in #317
• Add batch level std calculation by @garyzhang99 in #311
• Implement serial saving by @chenyushuo in #322
• Enhance experience replay for priority queue buffer by @yanxi-chen in #306
• Fix example config typo by @garyzhang99 in #323
• Simplify Config by @pan-x-c in #325
• Update config from ray cluster by @hiyuchang in #324
• Support AgentScope Workflow Function by @pan-x-c in #327

Full Changelog: v0.3.0...v0.3.1

v0.3.0

Overview

Framework Development

Buffer Module

1. Use Operator Interface to replace the original AddStrategy. Operator can perform various transformations on experience data in a pipeline manner. [Breaking Change]
2. Add TaskPipeline and ExperiencePipeline for task and experience data preprocessing.
3. Support calling Data-Juicer services in both TaskPipeline and ExperiencePipeline, and resolve some dependency conflicts.
4. Refactor SQL/FILE storage. SQL can store SFT/DPO/Rollout/Experience data. SQL and FILE support parsing multi-turn SFT data with tools. [Breaking Change]

Trainer Module

1. Support FSDP2 backend
2. Support Megatron backend
3. Support Qwen2.5 VL muti-modal models [Experimental]

Explorer Module

1. Support Qwen2.5 VL multi-modal models [Experimental]
2. Workflow supports running in async mode.
3. ModelWrapper provides openai.AsyncOpenAI interface.

Utils Module

1. Enhance logger and support printing logs of different actors to different files under the checkpoint dir
2. Enhance wandb and mlflow monitor

New Algorithms

1. AsymRE
2. sPPO
3. RULER
4. TOPR and CISPO

New Workflows

1. General Multi-turn Email Search

Others

1. Support uv
2. Refactor README and documents
3. Fix many bugs

What's Changed

• Support FSDP2 by @chenyushuo in #204
• Update veRL to 0.5.0 by @pan-x-c in #195
• Enhance Buffer Data Processing Pipeline by @pan-x-c in #175
• Remove AddStrategy by @pan-x-c in #211
• Add sft tools example and tests by @garyzhang99 in #210
• Cleaning up the old-version data processor by @HYLcool in #213
• Update main readme for v0.2.1 by @yanxi-chen in #216
• Add data_process figure by @hiyuchang in #218
• Enhance Logger by @pan-x-c in #217
• Bug fix for experience display in wandb by @chenyushuo in #224
• Fix links in readme by @yanxi-chen in #225
• Fix alfworld prompt typo by @garyzhang99 in #222
• Enhance SFT/DPO reader by @pan-x-c in #226
• AsymRE by @yaochaorui in #187
• [example] An email search workflow by @hiyuchang in #230
• Bug fix for FSDP offload by @chenyushuo in #233
• Refactor Storage by @pan-x-c in #227
• sPPO by @yaochaorui in #232
• Specify attempt in error message in run_with_retry by @vadimkantorov in #241
• Add issue template by @pan-x-c in #246
• [Example] GRPO on GSM8K with RULER reward by @hiyuchang in #239
• Add TOPR and CISPO algorithm by @garyzhang99 in #185
• Support Multi turn SFT with tools by @pan-x-c in #245
• Support Megatron by @chenyushuo in #219
• Support Multi-Modal LLM by @hiyuchang in #234
• Add log_table function for mlflow by @hiyuchang in #249
• Update the example of human in the loop by @HYLcool in #247
• Keep SQL Experience Buffer behavior consistent with previous versions by @pan-x-c in #248
• Update MoE training in example by @chenyushuo in #251
• SFT tools schema fix by @garyzhang99 in #252
• Merge verl-related config into default config by @hiyuchang in #256
• Update config manager for 0.3.0 by @chenyushuo in #257

Full Changelog: v0.2.1...v0.3.0