Skip to content

feat: Codex mode — exec_command/write_stdin unified-exec tool pair on the process manager #111

Description

@378-kaiabot

Note

Automated check by Pi, model aperture/synthetic/hf:moonshotai/Kimi-K3. Written against
aliou/pi-processes on branch main (16c1080, v0.12.0) from a read of the current sources.

Summary

Add an optional "Codex mode" that registers OpenAI Codex's unified-exec tool pair —
exec_command and write_stdin — on top of the existing ProcessManager, and disables the
built-in bash tool while active. Models trained on Codex's tool surface get the exact
interface they know (blocking-with-yield shell, session handles, reattachable output streams),
and the process manager already provides ~90% of the machinery: detached process groups,
combined-ordered logs, stdin writes, signal sends.

The interface to replicate

From openai/codex (unified_exec feature; codex-rs/core/src/tools/handlers/shell_spec.rs,
codex-rs/core/src/unified_exec/*):

  • exec_commandcmd, workdir, shell, login, yield_time_ms (default 10000,
    clamp 250–30000), max_output_tokens (default 10000). Blocks up to yield_time_ms collecting
    output; returns the exit code if finished, else a numeric session_id.
  • write_stdinsession_id, chars (empty = poll, waits 5000–300000 ms; non-empty
    writes wait ~250 ms, cap 30000), max_output_tokens. Ctrl-C is sent as \x03.
  • Both render a header: Chunk ID, Wall time, Process exited with code N /
    Process running with session ID N, Original token count, Output:.

The shape that matters for models: one tool for "run this", one tool for "keep talking to it",
with output returned as deltas rather than snapshots.

Design against the current codebase

Where it lives

Inside the existing processes extension — a new tools/codex/ directory next to the other
tool modules, registered from registerProcessTool's caller when a new
codex.enabled config flag (ProcessConfig, extensions/processes/config/types.ts) is set.
Same ProcessManager instance as the process tool; no new package, no protocol channels.
The flag needs migration 003 under extensions/processes/config/migrations/, a
PROCESS_CONFIG_VERSION bump in 002-stamp-config-version.ts, and pnpm gen:schema.

One manager addition: a byte-cursor read over combined.log

getCombinedOutputSince(
  id: string,
  byteOffset: number,
): { lines: Array<{ type: "stdout" | "stderr"; text: string }>; nextOffset: number } | null

ProcessLogStore already writes *-combined.log with 1:/2: stream tags and caps it at
64 MiB via truncate-and-restart (MAX_LOG_FILE_BYTES). The new method reads forward from
byteOffset and returns the consumed size as nextOffset; if the file shrank below the offset
(a truncate happened), it resets to 0 and the caller naturally re-reads from the truncation
marker line. The tool layer keeps Map<sessionId, { processId, cursor }> — the only
per-session state.

Why not drain process_output_changed events (appendedText deltas)? The event buffer is
consumed on every emit whether or not anyone listens, so a session that goes quiet between
polls would force the tool layer to accumulate unbounded output in memory. The on-disk log is
already bounded; a cursor costs nothing while idle.

exec_command: blocking yield, tool-layer only

  1. manager.start(name, cmd, workdir) — synchronous spawn, returns the opaque process id.
  2. Poll getCombinedOutputSince on a ~250 ms interval (Codex's minimum clamp), checking
    manager.get(id)?.status against LIVE_STATUSES each pass.
  3. Exited within yield_time_ms → return collected output + exit code, no session_id.
  4. Still running → mint a session id for the map entry, return output-so-far + the id.

write_stdin: write / poll / Ctrl-C

  • Non-empty chars: manager.writeToStdin(processId, chars) ({ end: true } gives EOF),
    then collect deltas for ~250 ms.
  • Empty chars: poll getCombinedOutputSince until output arrives or the deadline
    (5000–300000 ms).
  • \x03: compose from the existing signal-parameterized kill —
    manager.kill(id, { signal: "SIGINT", timeoutMs: 500 }), escalate to SIGTERM, then the
    default SIGKILL path. No PTY, no manager change.

Known limitation: pipe-spawned processes have no controlling terminal, and a
non-interactive bash -lc sets SIGINT to IGNORE for job-control-less commands — a
sleep loop can survive SIGINT while default-disposition programs (cat) die. The
escalation chain covers the common case; document the rest.

Output clip + format

max_output_tokens * 4 bytes, matching the existing 4-bytes/token convention; a pure
formatExecOutput(header fields, lines) renders the Codex header and is unit-testable.

Codex mode toggle

Pi's ExtensionAPI exposes getActiveTools() / setActiveTools(...); on session_start with
the flag on, filter bash out of the active set. Keep the process tool active — its exit
notifications and log watches are the agent's alternative to polling, and the Codex pair has no
equivalent.

Verification

E2E with existing fixtures in tests/e2e/scripts/:

  • emit-output.sh (short) → exit code + header, no session id.
  • continuous-output.sh (long) → live session_id, successive polls return only new output.
  • stdin-echo.sh (interactive) → write_stdin drives the REPL: input echoes, quit exits 0.

Unit tests for the cursor (offset advance, truncate reset, tag parsing), the formatter, and
the interrupt composition. Gates: pnpm typecheck, pnpm lint, pnpm test, pnpm test:e2e,
pnpm check:schema + a changeset.

Scope

  • In: both tools, cursor read, Codex formatting, config flag + migration, bash disable,
    Ctrl-C escalation, tests.
  • Out: PTY (tty param), Codex's sandbox/approval layer, remote exec-server, zsh-fork mode.

Run host: solar-al-khwarizmi · Session: 01a04286-fdf7-76d4-a0fe-9c6998f6db2a · Model: aperture/synthetic/hf:moonshotai/Kimi-K3

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions