[DRAFT] # PR: Support Repository-based MCP Plugins & Options in Codex CLI Generator

[silviojr]

This PR extends the Codex CLI generator to support cloning plugin repositories containing bundled MCP servers (such as the Data Agent Kit), auto-enabling them in config.toml, and passing configuration parameters (userConfig) to them.

1. Refactored Configuration Serialization

• Problem: Previously, config.toml was written immediately on initialization with only inline mcp_servers configuration. This prevented adding plugins cloned dynamically during setup.
• Solution: Refactored CodexCliGenerator to collect configurations (inline servers, repository-based servers, and cloned skills/plugins) into internal state dictionaries (self.inline_mcp_servers and self.enabled_plugins) first, and deferred writing the config.toml file to the end of the _setup method.

2. Supported List-Based mcp_servers Repositories

• Decision: Adopted the same configuration schema as ClaudeCodeGenerator to maintain cross-agent testing consistency.
• Implementation: Implemented _install_mcp_servers_from_repo which clones the plugin repository, registers it in marketplace.json, and records its ID and parameters in the enabled plugins list.

3. Enabled Plugins in config.toml

• Decision: Registered plugins in marketplace.json must be explicitly marked as enabled in config.toml to be loaded by Codex CLI.
• Implementation: Wrote [plugins."<plugin-id>"] sections with enabled = true and options inside config.toml. The plugin_id is constructed using the plugin-name@marketplace-name pattern.

4. Supported Plugin Options & Variables (userConfig)

• Problem: DAK's MCP servers rely on GCP project/location parameters to run queries (e.g. ${user_config.PROJECT_ID}).
• Solution: Extracted the config blocks from the YAML settings, passed them to self.enabled_plugins, and serialized them as inline TOML tables under options = { ... }.

Testing & Verification

1. Test Scenario Setup

We executed the evaluation pipeline using the Bullseye_PCA scenario:

• Run Config: Bullseye_PCA_codex_config.yaml (local)
• Model Config: codex_cli_dak_model.yaml (local)
  Specifies cloning and loading the dak plugin from its repository:

  setup:
    mcp_servers:
      - action: install_from_repo
        url: "https://github.com/gemini-cli-extensions/data-agent-kit-starter-pack.git"
        plugin: "dak"
        config:
          PROJECT_ID: "datagravity-demo"
          GCP_REGION: "us-central1"
          BIGQUERY_LOCATION: "US"

• Dataset Config: Bullseye_PCA.json (local)
  Requests creating a notebook and running a sample BigQuery query.

2. Sandbox Verification

After executing the setup sequence, we inspected the sandboxed directory .venv/fake_home_codex and verified:

1. Marketplace Configuration: The marketplace manifest file at .agents/plugins/marketplace.json was correctly created, pointing to the local cloned repository with the relative path starting with ./ (e.g. "path": "./.codex/plugins/data-agent-kit-starter-pack").
2. Plugin Registration & Installation: Running codex plugin list in the sandbox confirmed the plugin status was successfully transitioned to:
   dak@evalbench-local-marketplace | installed, enabled
3. Skills Installation: The 19 skills bundled inside the starter pack repository (such as developing-with-bigquery, notebook-guidance, etc.) were successfully copied to .codex/skills/.

3. Execution Verification

We executed the run and examined the outputs in results/aadf29ef-3145-4d8e-aca1-9e9bd09b58a9:

• Goal Completion Evaluation: PASS (100.0%)
• Actual Tool Execution: The agent successfully executed actual tool calls to the MCP server rather than falling back to text generation:
  • notebook__create_notebook (succeeded)
  • notebook__list_cells (succeeded)
  • notebook__insert_cell (succeeded, twice)
• Stats and Logs: Verified that all tool calls resolved and completed successfully with no goal/thread error logs in Codex CLI.