Yunnglin · Jul 23, 2026 · Jul 23, 2026 · Jul 23, 2026
diff --git a/docs/en/benchmarks/job_bench.md b/docs/en/benchmarks/job_bench.md
@@ -0,0 +1,158 @@
+# JobBench
+
+
+## Overview
+
+JobBench evaluates agentic systems on realistic professional work tasks that require reading reference files, producing
+deliverables, and reconciling multi-source information. This adapter uses the ModelScope dataset
+`evalscope/job-bench`.
+
+## Task Description
+
+- **Task Type**: Agentic professional work / deliverable generation
+- **Input**: Workplace-style task prompt with optional reference files under `reference_files/`
+- **Output**: Final deliverable files written to `jobbench_output/`
+- **Dataset**: ModelScope `evalscope/job-bench`
+- **Metric**: Weighted LLM-judge rubric score (`normalized_score`)
+
+## Evaluation Notes
+
+- The default evaluation split is `main`.
+- Configure `judge_model_args` for rubric scoring.
+- `normalized_score` is the primary weighted score (`total_score / max_score`); `pass_rate` is the unweighted
+  proportion of fully passed rubrics; `total_score` is the raw sum of passed rubric weights.
+- Docker runs use `python:3.11-slim-bookworm` by default. For formal evaluation, provide an image with the Office,
+  PDF, and spreadsheet tools required by the tasks.
+
+
+## Properties
+
+| Property | Value |
+|----------|-------|
+| **Benchmark Name** | `job_bench` |
+| **Dataset ID** | [evalscope/job-bench](https://modelscope.cn/datasets/evalscope/job-bench/summary) |
+| **Paper** | N/A |
+| **Tags** | `Agent`, `Knowledge`, `MultiTurn` |
+| **Metrics** | `normalized_score`, `pass_rate`, `total_score` |
+| **Default Shots** | 0-shot |
+| **Evaluation Split** | `main` |
+
+
+## Data Statistics
+
+| Metric | Value |
+|--------|-------|
+| Total Samples | 65 |
+| Prompt Length (Mean) | 3344.32 chars |
+| Prompt Length (Min/Max) | 2000 / 4920 chars |
+
+## Sample Example
+
+**Subset**: `default`
+
+```json
+{
+  "input": [
+    {
+      "id": "203fd1aa",
+      "content": "You are preparing the Statistical Analysis Plan for Phase III trial XR-2847 evaluating cardiovascular disease prevention. The sponsor requires validation of statistical assumptions against empirical data and regulatory alignment before protoc ... [TRUNCATED 2026 chars] ... erables.\n\nWrite every final deliverable file under `jobbench_output`. Do not put intermediate scratch files there.\nYour final message may summarize what you produced, but files requested by the task must be actual files in\n`jobbench_output`.\n"
+    }
+  ],
+  "id": 0,
+  "group_id": 0,
+  "tools": [
+    {
+      "name": "bash",
+      "description": "Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.",
+      "parameters": {
+        "properties": {
+          "command": {
+            "type": "string",
+            "description": "The bash command to execute."
+          },
+          "timeout": {
+            "type": "number",
+            "description": "Maximum execution time in seconds (default: 60).",
+            "default": 60
+          }
+        },
+        "required": [
+          "command"
+        ]
+      }
+    },
+    {
+      "name": "python_exec",
+      "description": "Execute Python source code inside the sandbox environment. Returns stdout and stderr output.",
+      "parameters": {
+        "properties": {
+          "code": {
+            "type": "string",
+            "description": "Python source code to execute."
+          },
+          "timeout": {
+            "type": "number",
+            "description": "Maximum execution time in seconds (default: 60).",
+            "default": 60
+          }
+        },
+        "required": [
+          "code"
+        ]
+      }
+    }
+  ],
+  "metadata": {
+    "task_id": "biostatisticians__task1",
+    "reference_files": [
+      "dataset/biostatisticians/task1/task_folder/Clinical_Study_Proposal_CVD_Prevention.csv",
+      "dataset/biostatisticians/task1/task_folder/framingham.csv"
+    ],
+    "rubric_json": "{\n  \"rubrics\": [\n    {\n      \"rubric\": \"Does the analysis calculate the observed 10-year CHD event rate in the eligible population as 20.2% (236/1166) and identify this as higher than the assumed 15% control group rate?\",\n      \"weight\": 10,\n ... [TRUNCATED 4642 chars] ... ge criterion and by the sysBP criterion separately\",\n        \"The flow shows the final eligible population count (1,166)\",\n        \"The flow enables verification of the filtering process and assessment of generalizability\"\n      ]\n    }\n  ]\n}"
+  }
+}
+```
+
+*Note: Some content was truncated for display.*
+
+## Prompt Template
+
+**Prompt Template:**
+```text
+{question}
+```
+
+## Usage
+
+### Using CLI
+
+```bash
+evalscope eval \
+    --model YOUR_MODEL \
+    --api-url OPENAI_API_COMPAT_URL \
+    --api-key EMPTY_TOKEN \
+    --datasets job_bench \
+    --agent-config '{"mode":"native","strategy":"function_calling","max_steps":250}' \
+    --limit 10  # Remove this line for formal evaluation
+```
+
+### Using Python
+
+```python
+from evalscope import TaskConfig, run_task
+from evalscope.api.agent import NativeAgentConfig
+
+task_cfg = TaskConfig(
+    model='YOUR_MODEL',
+    api_url='OPENAI_API_COMPAT_URL',
+    api_key='EMPTY_TOKEN',
+    datasets=['job_bench'],
+    agent_config=NativeAgentConfig(
+        strategy='function_calling',
+        max_steps=250,
+    ),
+    limit=10,  # Remove this line for formal evaluation
+)
+
+run_task(task_cfg=task_cfg)
+```
diff --git a/docs/en/get_started/supported_dataset/agent.md b/docs/en/get_started/supported_dataset/agent.md
@@ -14,6 +14,7 @@ Below is the list of supported AGENT benchmarks. Click on a benchmark name for d
 | `gaia` | [GAIA](../../benchmarks/gaia.md) | `Agent`, `MultiTurn`, `Reasoning` |
 | `gdpval` | [GDPval](../../benchmarks/gdpval.md) | `Agent`, `Knowledge`, `MultiTurn` |
 | `general_fc` | [General-FunctionCalling](../../benchmarks/general_fc.md) | `Agent`, `Custom`, `FunctionCalling` |
+| `job_bench` | [JobBench](../../benchmarks/job_bench.md) | `Agent`, `Knowledge`, `MultiTurn` |
 | `k2_verifier` | [K2-Vendor-Verifier](../../benchmarks/k2_verifier.md) | `Agent`, `FunctionCalling` |
 | `kimi_verifier` | [Kimi-Vendor-Verifier (Param Compliance)](../../benchmarks/kimi_verifier.md) | `Agent`, `FunctionCalling` |
 | `mcp_atlas` | [MCP-Atlas](../../benchmarks/mcp_atlas.md) | `Agent`, `MultiTurn` |
@@ -48,6 +49,7 @@ Below is the list of supported AGENT benchmarks. Click on a benchmark name for d
 ../../benchmarks/gaia.md
 ../../benchmarks/gdpval.md
 ../../benchmarks/general_fc.md
+../../benchmarks/job_bench.md
 ../../benchmarks/k2_verifier.md
 ../../benchmarks/kimi_verifier.md
 ../../benchmarks/mcp_atlas.md

diff --git a/docs/zh/benchmarks/job_bench.md b/docs/zh/benchmarks/job_bench.md
@@ -0,0 +1,153 @@
+# JobBench
+
+
+## 概述
+
+JobBench 评估智能体系统在真实专业工作场景中的表现，这些任务要求系统能够阅读参考文件、生成交付成果，并整合多源信息。本适配器使用 ModelScope 数据集 `evalscope/job-bench`。
+
+## 任务描述
+
+- **任务类型**：智能体专业工作 / 交付成果生成
+- **输入**：包含可选参考文件（位于 `reference_files/` 目录下）的职场风格任务提示
+- **输出**：最终交付成果文件写入 `jobbench_output/` 目录
+- **数据集**：ModelScope `evalscope/job-bench`
+- **评估指标**：加权 LLM 评分标准得分（`normalized_score`）
+
+## 评估说明
+
+- 默认评估划分（split）为 `main`。
+- 请配置 `judge_model_args` 以进行评分标准打分。
+- `normalized_score` 是主要的加权得分（`total_score / max_score`）；`pass_rate` 是完全通过评分项的未加权比例；`total_score` 是通过评分项权重的原始总和。
+- Docker 运行默认使用 `python:3.11-slim-bookworm` 镜像。正式评估时，请提供包含任务所需 Office、PDF 和电子表格工具的镜像。
+
+## 属性
+
+| 属性 | 值 |
+|----------|-------|
+| **基准测试名称** | `job_bench` |
+| **数据集ID** | [evalscope/job-bench](https://modelscope.cn/datasets/evalscope/job-bench/summary) |
+| **论文** | 无 |
+| **标签** | `Agent`, `Knowledge`, `MultiTurn` |
+| **指标** | `normalized_score`, `pass_rate`, `total_score` |
+| **默认示例数** | 0-shot |
+| **评估划分** | `main` |
+
+
+## 数据统计
+
+| 指标 | 值 |
+|--------|-------|
+| 总样本数 | 65 |
+| 提示词长度（平均） | 3344.32 字符 |
+| 提示词长度（最小/最大） | 2000 / 4920 字符 |
+
+## 样例示例
+
+**子集**: `default`
+
+```json
+{
+  "input": [
+    {
+      "id": "203fd1aa",
+      "content": "You are preparing the Statistical Analysis Plan for Phase III trial XR-2847 evaluating cardiovascular disease prevention. The sponsor requires validation of statistical assumptions against empirical data and regulatory alignment before protoc ... [TRUNCATED 2026 chars] ... erables.\n\nWrite every final deliverable file under `jobbench_output`. Do not put intermediate scratch files there.\nYour final message may summarize what you produced, but files requested by the task must be actual files in\n`jobbench_output`.\n"
+    }
+  ],
+  "id": 0,
+  "group_id": 0,
+  "tools": [
+    {
+      "name": "bash",
+      "description": "Execute a bash command inside the sandbox environment. Returns the combined stdout / stderr output of the command.",
+      "parameters": {
+        "properties": {
+          "command": {
+            "type": "string",
+            "description": "The bash command to execute."
+          },
+          "timeout": {
+            "type": "number",
+            "description": "Maximum execution time in seconds (default: 60).",
+            "default": 60
+          }
+        },
+        "required": [
+          "command"
+        ]
+      }
+    },
+    {
+      "name": "python_exec",
+      "description": "Execute Python source code inside the sandbox environment. Returns stdout and stderr output.",
+      "parameters": {
+        "properties": {
+          "code": {
+            "type": "string",
+            "description": "Python source code to execute."
+          },
+          "timeout": {
+            "type": "number",
+            "description": "Maximum execution time in seconds (default: 60).",
+            "default": 60
+          }
+        },
+        "required": [
+          "code"
+        ]
+      }
+    }
+  ],
+  "metadata": {
+    "task_id": "biostatisticians__task1",
+    "reference_files": [
+      "dataset/biostatisticians/task1/task_folder/Clinical_Study_Proposal_CVD_Prevention.csv",
+      "dataset/biostatisticians/task1/task_folder/framingham.csv"
+    ],
+    "rubric_json": "{\n  \"rubrics\": [\n    {\n      \"rubric\": \"Does the analysis calculate the observed 10-year CHD event rate in the eligible population as 20.2% (236/1166) and identify this as higher than the assumed 15% control group rate?\",\n      \"weight\": 10,\n ... [TRUNCATED 4642 chars] ... ge criterion and by the sysBP criterion separately\",\n        \"The flow shows the final eligible population count (1,166)\",\n        \"The flow enables verification of the filtering process and assessment of generalizability\"\n      ]\n    }\n  ]\n}"
+  }
+}
+```
+
+*注：部分内容因展示需要已被截断。*
+
+## 提示模板
+
+**提示模板：**
+```text
+{question}
+```
+
+## 使用方法
+
+### 使用 CLI
+
+```bash
+evalscope eval \
+    --model YOUR_MODEL \
+    --api-url OPENAI_API_COMPAT_URL \
+    --api-key EMPTY_TOKEN \
+    --datasets job_bench \
+    --agent-config '{"mode":"native","strategy":"function_calling","max_steps":250}' \
+    --limit 10  # 正式评估时请删除此行
+```
+
+### 使用 Python
+
+```python
+from evalscope import TaskConfig, run_task
+from evalscope.api.agent import NativeAgentConfig
+
+task_cfg = TaskConfig(
+    model='YOUR_MODEL',
+    api_url='OPENAI_API_COMPAT_URL',
+    api_key='EMPTY_TOKEN',
+    datasets=['job_bench'],
+    agent_config=NativeAgentConfig(
+        strategy='function_calling',
+        max_steps=250,
+    ),
+    limit=10,  # 正式评估时请删除此行
+)
+
+run_task(task_cfg=task_cfg)
+```
diff --git a/docs/zh/get_started/supported_dataset/agent.md b/docs/zh/get_started/supported_dataset/agent.md
@@ -14,6 +14,7 @@
 | `gaia` | [GAIA](../../benchmarks/gaia.md) | `Agent`, `MultiTurn`, `Reasoning` |
 | `gdpval` | [GDPval](../../benchmarks/gdpval.md) | `Agent`, `Knowledge`, `MultiTurn` |
 | `general_fc` | [General-FunctionCalling](../../benchmarks/general_fc.md) | `Agent`, `Custom`, `FunctionCalling` |
+| `job_bench` | [JobBench](../../benchmarks/job_bench.md) | `Agent`, `Knowledge`, `MultiTurn` |
 | `k2_verifier` | [K2-Vendor-Verifier](../../benchmarks/k2_verifier.md) | `Agent`, `FunctionCalling` |
 | `kimi_verifier` | [Kimi-Vendor-Verifier (Param Compliance)](../../benchmarks/kimi_verifier.md) | `Agent`, `FunctionCalling` |
 | `mcp_atlas` | [MCP-Atlas](../../benchmarks/mcp_atlas.md) | `Agent`, `MultiTurn` |
@@ -48,6 +49,7 @@
 ../../benchmarks/gaia.md
 ../../benchmarks/gdpval.md
 ../../benchmarks/general_fc.md
+../../benchmarks/job_bench.md
 ../../benchmarks/k2_verifier.md
 ../../benchmarks/kimi_verifier.md
 ../../benchmarks/mcp_atlas.md