【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py #422

Kozmosa · 2025-12-06T15:47:29Z

Summary

Added built-in decorator AgentUnittestGenerator to both run_model entrypoints (run_model.py, run_model.py) to generate standalone unittest scripts from model.py + meta files.
Use jinja2.Template for codegen templating (placeholders: target device and dummy/numpy flags);
Provided smoke script to exercise the decorator on one CV and one NLP Torch sample (test_agent_unittest_generator.sh); scripts output <sample>_test.py beside each sample.

How to Use

Prepare a decorator config (base64 JSON) pointing to the built-in class:

{
  "decorator_class_name": "AgentUnittestGenerator",
  "decorator_config": {
    "model_path": "/abs/path/to/sample",
    "force_device": "auto",      // cpu|cuda|auto (Torch); cpu|gpu|auto (Paddle)
    "output_path": null,         // default: <sample>/<sample_name>_test.py
    "use_dummy_inputs": false    // Torch only
    // Paddle: use_numpy flag is supported instead
  }
}

Invoke run_model:
- Torch: python -m graph_net.torch.run_model --model-path /abs/path/to/sample --decorator-config <base64>
- Paddle: python -m graph_net.paddle.run_model --model-path /abs/path/to/sample --decorator-config <base64>
The generated unittest is self-contained and runnable: python <sample_name>_test.py.

Validation (manual)

CV (Torch): sample resnet18 → generated resnet18_test.py runs on CPU.
NLP (Torch): sample albert-base-v2 → generated albert-base-v2_test.py runs on CPU.
Paddle side: generation logic mirrors Torch; not explicitly run in this check.

Notes

New dependency: jinja2 (required to render templates).
Backward compatibility: _get_decorator still accepts both argparse.Namespace and dict configs.

Sample Artifact

Generated Torch unittest example: albert-base-v2_test.py (created via the decorator).

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ bash graph_net/test/test_agent
_unittest_generator.sh
[AgentTest] running CV (torchvision/resnet18) sample at /home/kozmosa/code/GraphNet/samples/torchvision/resnet18
model_path='/home/kozmosa/code/GraphNet/samples/torchvision/resnet18'
[Agent] unittest 已生成: /home/kozmosa/code/GraphNet/samples/torchvision/resnet18/resnet18_test.py (device=cuda)
use_dummy_inputs=False
[AgentTest] running NLP (transformers-auto-model/albert-base-v2) sample at /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2
model_path='/home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2'
[Agent] unittest 已生成: /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2/albert-base-v2_test.py (device=cuda)
use_dummy_inputs=False
[AgentTest] done. Generated *_test.py files should now exist beside the samples.

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ python /home/kozmosa/code/Graph
Net/samples/torchvision/resnet18/resnet18_test.py
.
----------------------------------------------------------------------
Ran 1 test in 6.119s

OK

(graphnet) kozmosa@dev_kozmosa:~/code/GraphNet$ python /home/kozmosa/code/GraphNet/samples/transformers-auto-model/albert-base-v2/albert-base-v2_test.py
.
----------------------------------------------------------------------
Ran 1 test in 2.871s

OK

resnet18_test.py

import importlib.util
import os
import tempfile
import unittest

import torch
from graph_net.torch import utils


def _load_graph_module(model_path: str, target_device: str):
    source_path = os.path.join(model_path, "model.py")
    with open(source_path, "r", encoding="utf-8") as f:
        code = f.read()

    if target_device != "cuda":
        code = utils.modify_code_by_device(code, target_device)

    tmp_dir = tempfile.mkdtemp(prefix="agent_unittest_")
    tmp_file = os.path.join(tmp_dir, "model_tmp.py")
    with open(tmp_file, "w", encoding="utf-8") as f:
        f.write(code)

    spec = importlib.util.spec_from_file_location("agent_graph_module", tmp_file)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module.GraphModule


class AgentGraphTest(unittest.TestCase):
    def setUp(self):
        self.model_path = os.path.dirname(__file__)
        self.target_device = "cuda"
        self.GraphModule = _load_graph_module(self.model_path, self.target_device)
        self.meta = utils.load_converted_from_text(self.model_path)
        self.use_dummy_inputs = False

    def _with_device(self, info):
        cloned = {"info": dict(info["info"]), "data": info.get("data")}
        cloned["info"]["device"] = self.target_device
        return cloned

    def test_forward_runs(self):
        model = self.GraphModule()
        weight_info = self.meta["weight_info"]

        def _build_tensor(val):
            wrapped = self._with_device(val)
            return (
                utils.get_dummy_tensor(wrapped)
                if self.use_dummy_inputs
                else utils.replay_tensor(wrapped)
            )

        state_dict = {k: _build_tensor(v) for k, v in weight_info.items()}
        model.__graph_net_file_path__ = self.model_path
        output = model(**state_dict)
        self.assertIsNotNone(output)


if __name__ == "__main__":
    unittest.main()

albert-base-v2_test.py

import importlib.util
import os
import tempfile
import unittest

import torch
from graph_net.torch import utils


def _load_graph_module(model_path: str, target_device: str):
    source_path = os.path.join(model_path, "model.py")
    with open(source_path, "r", encoding="utf-8") as f:
        code = f.read()

    if target_device != "cuda":
        code = utils.modify_code_by_device(code, target_device)

    tmp_dir = tempfile.mkdtemp(prefix="agent_unittest_")
    tmp_file = os.path.join(tmp_dir, "model_tmp.py")
    with open(tmp_file, "w", encoding="utf-8") as f:
        f.write(code)

    spec = importlib.util.spec_from_file_location("agent_graph_module", tmp_file)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module.GraphModule


class AgentGraphTest(unittest.TestCase):
    def setUp(self):
        self.model_path = os.path.dirname(__file__)
        self.target_device = "cuda"
        self.GraphModule = _load_graph_module(self.model_path, self.target_device)
        self.meta = utils.load_converted_from_text(self.model_path)
        self.use_dummy_inputs = False

    def _with_device(self, info):
        cloned = {"info": dict(info["info"]), "data": info.get("data")}
        cloned["info"]["device"] = self.target_device
        return cloned

    def test_forward_runs(self):
        model = self.GraphModule()
        weight_info = self.meta["weight_info"]

        def _build_tensor(val):
            wrapped = self._with_device(val)
            return (
                utils.get_dummy_tensor(wrapped)
                if self.use_dummy_inputs
                else utils.replay_tensor(wrapped)
            )

        state_dict = {k: _build_tensor(v) for k, v in weight_info.items()}
        model.__graph_net_file_path__ = self.model_path
        output = model(**state_dict)
        self.assertIsNotNone(output)


if __name__ == "__main__":
    unittest.main()

CLAassistant · 2025-12-06T15:47:35Z

All committers have signed the CLA.

paddle-bot · 2025-12-06T15:47:36Z

Thanks for your contribution!

…e and torch.

Xreki · 2025-12-07T03:05:02Z

收到您的PR，感谢您的贡献。
在次跟您解释下，这个题目需要生成一个不依赖graph_net环境和样本本身的单测，即这个单测拿到任何一个只有python和torch的环境，都可以直接执行。

JewelRoam · 2025-12-08T03:57:35Z

整体还是很清晰的，PR说明很详细（赞），有几个小改进点：

拆分出独立组件，通过config中"handler_path"/"handler_class_name"(model_path_handler)或"decorator_path"(run_model)指定，尽可能不要修改入口脚本（model_path_handler/run_model）
我们目前逐步过渡到使用graph_net.model_path_handler代替run_model，需要辛苦入口脚本尽可能选用model_path_handler（参数配置类似，修改成本应该不高）
参照上面Xreki提到的，单测中不依赖graph_net，不应该需要import graph_net utils
与子图生成流程对齐，支持在config中指定"output_dir"，设置单测保存位置
使用简洁的英文注释，不要使用中文

graph_net/paddle/run_model.py

lixinqi · 2025-12-12T08:04:57Z

graph_net/paddle/run_model.py

+import unittest
+
+import paddle
+from graph_net.paddle import utils


这是不可以的。单测脚本应该能脱离graph_net单独工作。

已经按照建议进行修改，最新的提交中，生成的单侧脚本不再依赖graph_net

lixinqi · 2025-12-12T08:05:52Z

graph_net/test/test_agent_unittest_generator.sh

+encode_cfg() {
+  MODEL_PATH="$1" python - <<'PY'
+import base64, json, os
+cfg = {
+    "decorator_class_name": "AgentUnittestGenerator",
+    "decorator_config": {
+        "model_path": os.environ["MODEL_PATH"],
+        "force_device": "auto",
+        "output_path": None,
+        "use_dummy_inputs": False,
+    },
+}
+print(base64.b64encode(json.dumps(cfg).encode()).decode())
+PY
+}


可以参考https://github.com/PaddlePaddle/GraphNet/blob/develop/graph_net/bash_templates/model_path_handler_sh.txt 简化这里的逻辑

已经按照建议进行修改了

… AgentUnittestGenerator, adopt jinja2 as render engine

paddle-bot bot added the contributor External developers label Dec 6, 2025

feat: add AgentUnittestGenerator class for run_model.py in both paddl…

b2a820e

…e and torch.

Kozmosa force-pushed the develop branch from 076e07c to b2a820e Compare December 6, 2025 15:53

luotao1 mentioned this pull request Dec 8, 2025

【Hackathon 9th】开源贡献个人挑战赛冲刺赛 PaddlePaddle/Paddle#76333

Open

lixinqi reviewed Dec 12, 2025

View reviewed changes

Merge branch 'PaddlePaddle:develop' into develop

675631e

luotao1 changed the title ~~【Hackathon 9th Sprint No.87】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py~~ 【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py Dec 15, 2025

feat: refactor code to improve style, use a seperate method to inject…

05c4cd3

… AgentUnittestGenerator, adopt jinja2 as render engine

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py #422

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py #422

Uh oh!

Kozmosa commented Dec 6, 2025 •

edited

Loading

Uh oh!

CLAassistant commented Dec 6, 2025 •

edited

Loading

Uh oh!

paddle-bot bot commented Dec 6, 2025

Uh oh!

Xreki commented Dec 7, 2025

Uh oh!

JewelRoam commented Dec 8, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lixinqi Dec 12, 2025

Uh oh!

Kozmosa Dec 15, 2025

Uh oh!

lixinqi Dec 12, 2025

Uh oh!

Kozmosa Dec 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py #422

Are you sure you want to change the base?

【Hackathon 9th Sprint No.86】feat: implement AgentUnittestGenerator for both paddle/torch run_model.py #422

Uh oh!

Conversation

Kozmosa commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddle-bot bot commented Dec 6, 2025

Uh oh!

Xreki commented Dec 7, 2025

Uh oh!

JewelRoam commented Dec 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lixinqi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Kozmosa Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

lixinqi Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Kozmosa Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Kozmosa commented Dec 6, 2025 •

edited

Loading

CLAassistant commented Dec 6, 2025 •

edited

Loading

JewelRoam commented Dec 8, 2025 •

edited

Loading