feat: add CVRP benchmark (VehicleRouting/CVRP)

[zzy2007zzy2007]

Summary

• 在 VehicleRouting 套件下新增 CVRP
  基准（benchmarks/VehicleRouting/CVRP）
• 12 个确定性 TSPLIB 风格实例（seed 42），评测仅用 Python
  标准库
• 近最优确定性参考解（GRASP 多起点 + 局部搜索 +
  LNS）；刻意弱化的规范 baseline（随机顺序最近插入），给
  agent 留出明确提升空间（55.59 → 100）
• 含 unified-task 元数据（frontier_eval/）与容器评测
  Dockerfile

Background

CVRP 是经典 NP-hard 物流优化问题：最小化容量约束车队的总配
送距离。实例坐标聚类模拟城市客户分布（Reality
Gap）；成本最小化工程价值明确（Economic Value）。

Verification

• 原生：python verification/evaluator.py baseline/solver.py → combined_score 55.59, valid 1.0（12
  实例）
• unified 适配（仓库根）：python -m frontier_eval task=unified task.benchmark=VehicleRouting/CVRP algorithm.iterations=0
• Docker：docker build -t cvrp-benchmark -f verification/docker/Dockerfile . && docker run --rm cvrp-benchmark
• agent 实测（deepseek-v4-flash）：openevolve 96.38 /
  ShinkaEvolve 99.31 / AB-MCTS 98.70，对 reference 100

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a comprehensive benchmark for the Capacitated Vehicle Routing Problem (CVRP). It provides a standardized environment for evaluating optimization algorithms (specifically targeting LLM-based solvers) using industry-standard TSPLIB-style instances and a robust evaluation pipeline.
• Modified File Structure & Modifications:
  • benchmarks/VehicleRouting/CVRP/README.md & README_zh-CN.md: Comprehensive documentation of the benchmark, including structure, requirements, scoring, and reference scores.
  • benchmarks/VehicleRouting/CVRP/Task.md & Task_zh-CN.md: Detailed task descriptions for the solver, defining the I/O contract and validation rules.
  • benchmarks/VehicleRouting/CVRP/baseline/solver.py: Implements a "Random-order cheapest insertion" baseline solver with the required EVOLVE-BLOCK structure.
  • benchmarks/VehicleRouting/CVRP/data/instances/: Contains 12 deterministically generated .vrp instances.
  • benchmarks/VehicleRouting/CVRP/verification/: Contains the core infrastructure including evaluator.py (scoring), ref_solver.py (high-quality reference), and generate_instances.py.
  • benchmarks/VehicleRouting/CVRP/.gitignore: Standard Python exclusions.

2. AI Content Analysis

• Estimated AI Component: 40%
• Reasoning & Evidence:
  • Boilerplate & Documentation: The structure of the README.md and Task.md follows a highly consistent, professional template likely generated or polished by AI.
  • Baseline Solver: The solve function in baseline/solver.py uses very standard, idiomatic Python for heuristics (e.g., best_place inner function). The logic is correct but follows a "textbook" implementation style common in AI training data.
  • Domain Nuance: The parse_instance function shows significant domain-specific knowledge (handling NODE_COORD_SECTION, DEMAND_SECTION, and Euclidean distance rounding), which suggests human architectural oversight or specific prompting.

3. Engineering & Economic Assessment

• Engineering Reality Check: High. This is not a "toy example." It uses TSPLIB formats, implements a deterministic reference solver (GRASP + LNS), and handles critical edge cases like capacity violations and customer coverage. The inclusion of a Dockerfile and environment variables for timeouts/scaling demonstrates production-grade thinking.
• Economic Value: High. Combinatorial optimization is a high-value domain in logistics and supply chain. Providing a standardized way to benchmark LLMs against classic heuristics (like Clarke-Wright or LNS) helps quantify the actual utility of AI in operations research.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: VehicleRouting/CVRP
  • Execution & Dependencies: The .md files provide explicit commands for both local Python execution and Docker-based evaluation. It correctly notes that only the Python standard library is required for the evaluator.
• Documentation Quality: Excellent. The dual-language documentation is thorough. It includes optimization hints, scoring formulas, and pre-measured reference scores for various models (DeepSeek-V4). No significant grammatical errors were detected.
• Organizational Structure: Logical and Scalable. The separation of baseline, data, and verification is clean. The data/instances folder allows for easy expansion with more complex problems.

5. Security & Privacy Check

• Sensitive Files: Clean. .gitignore correctly excludes __pycache__. No API keys or .env files were found.
• Absolute Paths: None detected. The code uses pathlib.Path and relative sys-args for file I/O.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 为容量约束车辆路径问题 (CVRP) 引入了一个全面的 Benchmark。它提供了一个标准化的环境，用于评估优化算法（特别是针对基于 LLM 的求解器），使用了行业标准的 TSPLIB 风格实例和稳健的评估流水线。
• 修改的文件结构与变更摘要:
  • benchmarks/VehicleRouting/CVRP/README.md & README_zh-CN.md: 完整的 Benchmark 文档，包括结构、需求、评分标准和参考分数。
  • benchmarks/VehicleRouting/CVRP/Task.md & Task_zh-CN.md: 详细的求解器任务描述，定义了输入输出契约和验证规则。
  • benchmarks/VehicleRouting/CVRP/baseline/solver.py: 实现了一个“随机顺序最近插入”的基准求解器，包含所需的 EVOLVE-BLOCK 结构。
  • benchmarks/VehicleRouting/CVRP/data/instances/: 包含 12 个确定性生成的 .vrp 实例。
  • benchmarks/VehicleRouting/CVRP/verification/: 包含核心基础设施，包括 evaluator.py（评分）、ref_solver.py（高质量参考求解器）和 generate_instances.py。
  • benchmarks/VehicleRouting/CVRP/.gitignore: 标准 Python 忽略文件。

2. AI 成分分析

• 预估 AI 含量: 40%
• 判断依据与证据:
  • 模板与文档: README.md 和 Task.md 的结构遵循高度一致且专业的模板，很可能是由 AI 生成或润色的。
  • 基准求解器: baseline/solver.py 中的 solve 函数使用了非常标准、地道的 Python 启发式代码（例如 best_place 内置函数）。逻辑正确，但遵循了 AI 训练数据中常见的“教科书式”实现风格。
  • 领域细微差别: parse_instance 函数显示了显著的领域特定知识（处理 NODE_COORD_SECTION、DEMAND_SECTION 和欧几里得距离舍入），这表明了人工架构监督或特定的提示工程。

3. 工程与经济评估

• 工程现实检验: 高。这绝非“玩具示例”。它使用了 TSPLIB 格式，实现了确定性参考求解器（GRASP + LNS），并处理了诸如容量违规和客户覆盖等关键边缘情况。包含 Dockerfile 和用于超时/缩放的环境变量展示了生产级的思考。
• 经济价值: 高。组合优化是物流和供应链中的高价值领域。提供一种标准化的方法来衡量 LLM 与经典启发式算法（如 Clarke-Wright 或 LNS）的性能，有助于量化 AI 在运筹学中的实际效用。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是。
  • task_name: VehicleRouting/CVRP
  • 运行与依赖: .md 文件为本地 Python 运行和基于 Docker 的评测提供了明确的命令。正确注明了评测器仅需 Python 标准库。
• 文档质量: 优秀。双语文档非常详尽。包括优化提示、评分公式以及各种模型（如 DeepSeek-V4）的预测参考分数。未检测到明显的语法错误。
• 组织结构: 逻辑清晰且具备可扩展性。baseline、data 和 verification 的分离非常干净。data/instances 文件夹便于未来扩展更复杂的问题。

5. 安全与隐私检查

• 敏感文件: 未发现异常。.gitignore 正确排除了 __pycache__。未发现 API 密钥或 .env 文件。
• 绝对路径: 未检测到。代码使用 pathlib.Path 和相对系统参数进行文件 I/O。

[wrh-human]

PR #102 Review Comment

Thank you for submitting this CVRP benchmark PR. The task documentation, baseline, reference solver, unified metadata, and Dockerfile are all well organized, and I verified locally that the baseline reproduces at 55.59, the generated instances are byte-identical, and reference.json can be fully regenerated by the reference solver. Overall, CVRP is a non-numerical optimization task with clear economic value and fits Frontier-Eng's purpose.

However, the current implementation has several issues that affect the credibility of the evaluation, so I do not recommend merging it yet.

Main issues:

1. The evaluation sandbox copies the entire benchmark, so candidate code can directly import verification/ref_solver.py and call the reference solver. I built a minimal adversarial candidate and measured 99.57 / valid=1.0; the unified path also does not block it and reports no readonly_violation. This makes the score meaningless and is a blocking issue.
2. The evaluation set contains only 12 public, fixed, small instances with no held-out set or runtime-generated instances. A candidate can solve instances offline and hardcode routes by name, so the benchmark cannot demonstrate that an Agent learned a generalizable solving method.
3. constraints.txt says candidates must not read data/reference.json, but the evaluator writes the full reference distances into artifacts and returns them through the unified path, contradicting the constraint.
4. Constraints are not actually verified: there is no check that code outside the EVOLVE-BLOCK is unchanged, no prevention of reading/importing verification code or the reference, and no check for hardcoding or determinism.
5. There is no baseline/result_log.txt, no Agent run command, configuration, output logs, or multi-seed statistics; the "best" scores in the README are insufficient evidence.
6. The Dockerfile is only a manual evaluation path; the unified runtime still defaults to process mode, so container isolation is not actually integrated.

Please complete the following changes before continuing the review:

• Copy only the files the candidate needs into the evaluation sandbox; do not copy verification/ or data/reference.json.
• Add hidden/held-out instances, or generate and score instances at evaluation time, to prevent hardcoding by instance name.
• Remove reference information from the evaluator artifacts.
• Add verifiers that check that code outside the markers is unchanged, prohibit reading/importing evaluation code and the reference, and prohibit instance-level hardcoding.
• Add complete run records, fixed seeds, and multi-run statistics for the baseline and Agents.
• Actually integrate Docker isolation into the unified runtime.
• Add unit tests for the evaluator, validator, and reference solver.

Thank you again for your contribution. After completing the above modifications, the review can continue.