Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ The running `pasclaw agent --session <id>` drains the queue at the top of its NE

Notable feature additions, newest first. Bug fixes and review follow-ups are in the [full git log](https://github.com/FMXExpress/PasClaw/commits/main).

- **2026-06-21** — `pasclaw relay` worker client + `cog-relay/` Replicate cog. Polls a remote gateway's `/v1/relay/poll`, forwards each job through the locally-configured provider (`Cfg.DefaultProvider` or `--provider NAME`), POSTs back to `/v1/relay/respond/<id>`. Bearer auth via `PASCLAW_GATEWAY_TOKEN` or `--gateway-token`; reconnects on SSE drop with 1s→30s backoff; refuses to forward to a provider that is itself a relay (would loop). `cog-relay/` exposes the same flow as a Replicate predictor so a deployment with one API-key Secret can lend its compute to any gateway. Lifecycle: one prediction = one polling window (default 300 s).
- **2026-06-21** — Relay pull-worker provider: external workers (WebGPU browser tab, phone with mlc-llm, desktop with llama.cpp) connect INBOUND to `pasclaw gateway`, advertise models via `X-Relay-Capabilities`, pull pending requests off PasClaw's in-process queue, run inference locally, POST results back. New endpoints `/v1/relay/poll` (SSE), `/v1/relay/respond/<id>`, `/v1/relay/status`; new catalog row `relay`. Unlocks WebGPU inference without exposed ports, cog-hosted PasClaw served by your phone, shared home GPU across devices. ([#318])
- **2026-06-20** — `pasclaw plan` + `pasclaw build --goal` pairing. `pasclaw plan -d "<task>"` runs in `pmPlan` mode with a dedicated `plan_write` tool and writes `workspace/PLAN.md` (`## Goal / Files / Steps / Open questions / Risks`). `pasclaw build` auto-loads PLAN.md as system-prompt context (stale-check + `--no-plan` opt-out) and archives consumed plans to `workspace/memory/plans/<ts>.md`. `build --goal` parses `## Goal` and seeds the Ralph judge loop. `cog-build/predict.py` gains a 4-choice `mode` Input (`build` / `plan` / `plan build` / `plan build goal`). ([#317])
- **2026-06-20** — Cloudflare AI Gateway catalog rows: `cloudflare` (compat OpenAI-shape endpoint, default `workers-ai/@cf/meta/llama-3.3-70b-instruct-fp8-fast` using the gateway's `<provider>/<model>` routing), `cloudflare-anthropic` and `cloudflare-gemini` (native passthroughs with each provider's own auth header). Catalog-only — existing provider units handle the wire shape once `api_base` is set. ([#316])
Expand Down
74 changes: 74 additions & 0 deletions cog-relay/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# PasClaw RELAY cog

A Replicate [cog](https://github.com/replicate/cog) that runs `pasclaw relay` -- a pull-worker for the [relay provider pattern](../docs/providers-relay.md). The cog acts as a remote inference back-end for a separate PasClaw gateway: it polls the gateway's `/v1/relay/poll`, drains inference jobs, runs them through a locally-configured provider (OpenAI / Anthropic / Gemini / Groq / OpenRouter / DeepSeek / Ollama / LM Studio / vLLM / custom), and POSTs the results back to `/v1/relay/respond/<id>`.

Sibling to [`/cog/`](../cog/) (one-shot `pasclaw agent`) and [`/cog-build/`](../cog-build/) (multi-iter `pasclaw build` with workspace.zip handshake).

## Why

Decouple LLM credentials from the gateway. Put a $500-budget API key on the cog -- the credential boundary -- and point a gateway running on a laptop, a home server, a CI runner, or a friend's machine at the cog. The gateway gets the LLM's capacity *without ever holding the key*. The key only exists inside Replicate's encrypted Secret store and the running cog container.

Three flows this unblocks:

1. **Shared family LLM**: one household deployment of cog-relay with the family's API key; each family member's PasClaw points at the same gateway. Per-device PasClaw never sees the key.
2. **Bursty CI**: a Replicate deployment is cheap when idle. Wrap predict() in a CI workflow that spins it up before kicking off a build, tears it down after. Gateway-side `pasclaw build` runs against it.
3. **Air-gapped gateway**: gateway runs on a network that can reach Replicate but not your provider's API directly. The cog bridges the gap.

## Inputs

| Name | Type | Default | Description |
|---|---|---|---|
| `gateway_url` | str | -- | **Required.** Remote PasClaw gateway base URL. Worker opens SSE to `<url>/v1/relay/poll`. |
| `gateway_token` | Secret | -- | **Required.** Bearer for `/v1/relay/*`. The same `PASCLAW_GATEWAY_TOKEN` the gateway is configured with. |
| `lifetime_seconds` | int | 300 | How long the worker polls before predict() returns. One prediction = one polling window. |
| `worker_id` | str | "" | Worker identity shown on the gateway's `/v1/relay/status` panel. Empty = auto-generated `cog-relay-<random>`. |
| `*_api_key` | `Optional[Secret]` | None | Cloud provider creds: openai / anthropic / gemini / groq / openrouter / deepseek. Stored encrypted, masked in logs. |
| `ollama_url` / `lmstudio_url` / `vllm_url` | str | "" | Self-hosted OpenAI-compatible server URLs reachable from the cog container. |
| `custom_provider_kind` | str | "" | Catalog kind for any OpenAI-compatible endpoint not surfaced as a dedicated input: `mistral` / `xai` / `cerebras` / `moonshot` / `qwen` / `zhipu` / `perplexity` / `nvidia` / `volcengine` / `minimax` / `novita` / `litellm` / `mimo`, or `openai-compat` for in-house endpoints. |
| `custom_provider_url` / `custom_provider_key` / `custom_provider_model` | str / Secret / str | -- | The generic escape hatch. |
| `provider` | str | "" | Which configured provider to forward jobs through. Empty = first configured wins. |
| `model` | str | "" | Capability the worker advertises to the gateway's queue. Empty = provider's default. Pass a deliberate non-default string to make this worker handle only requests for that model id. |

**At least one provider must be configured** -- the worker forwards jobs *through* a provider, so it needs one. Either a cloud key, a local-server URL, or the `custom_provider_*` set.

## Output

A single string: the worker's captured stdout. Each completed dispatch logs one `relay worker: completed <id>` line, so a simple `grep -c 'completed'` on the output gives the jobs-served count for the window.

## Continuous coverage

One prediction is one polling window. For continuous worker coverage, wrap predict() in a loop:

```python
import replicate, time

while True:
out = replicate.run(
"your-handle/pasclaw-relay",
input={
"gateway_url": "https://gateway.example.com:8888",
"gateway_token": {"$secret": "GATEWAY_TOKEN"},
"openai_api_key": {"$secret": "OPENAI_KEY"},
"lifetime_seconds": 300,
},
)
served = out.count("relay worker: completed")
print(f"served {served} jobs this window")
time.sleep(1)
```

Or push the cog as a Replicate deployment and use the deployment's queue auto-scaling.

## Sandbox / shell / tools

The worker is a thin `Provider.Chat()` bridge -- it never runs tools, never touches the filesystem outside its scratch dir, never executes shell. Tool *definitions* arrive on the wire as part of each request envelope (so the provider can advertise them to the model) and any tool *calls* in the model's reply are forwarded verbatim back to the gateway; the gateway-side PasClaw owns the agent loop and dispatches tools there.

So sandbox configuration is irrelevant on the cog side. The shell/sandbox toggles a normal `pasclaw agent` cog uses don't apply.

## How it works (one prediction lifecycle)

1. Predictor builds a `config.json` with the chosen provider in `$PASCLAW_HOME`.
2. Spawns `pasclaw relay --gateway-url ... --gateway-token ...`.
3. The Pascal worker opens a long-poll SSE GET to `<gateway>/v1/relay/poll` with `Authorization: Bearer <token>`, `X-Relay-Worker-Id: <id>`, `X-Relay-Capabilities: <model>`.
4. As each `data: {...}` event arrives, the worker decodes the [request envelope](../docs/providers-relay.md#request-envelope), calls `Provider.Chat(messages, tools, model, options)`, encodes the result as the [response envelope](../docs/providers-relay.md#response-envelope), and POSTs to `<gateway>/v1/relay/respond/<id>`.
5. After `lifetime_seconds`, the predictor sends SIGTERM. The worker drops the SSE socket; the gateway's poll handler sees the disconnect and requeues any in-flight job for the next polling worker. Predict() returns the captured log.
75 changes: 75 additions & 0 deletions cog-relay/cog.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Configuration for Cog (replicate.com) -- PasClaw RELAY mode.
#
# Sibling to /cog/ (one-shot `pasclaw agent`) and /cog-build/
# (multi-iter `pasclaw build` with workspace.zip handshake).
#
# This one runs `pasclaw relay` -- a pull-worker that connects
# OUTBOUND to a remote PasClaw gateway's /v1/relay/poll, drains
# inference jobs off the gateway's queue, runs each one through the
# cog's locally-configured provider (any of openai/anthropic/gemini/
# groq/openrouter/deepseek/ollama/lmstudio/vllm/custom), and POSTs
# the result back to /v1/relay/respond/<id>.
#
# Use case: lend a cog deployment's API-key + compute to a separate
# gateway instance (a home server, a CI runner, a friend's PasClaw)
# without exposing the API key directly. The gateway never sees the
# key; it just sees results coming back through the queue.
#
# Each predict() call runs the worker for `lifetime_seconds` (default
# 300 = 5 min) then exits. Drop a Replicate deployment in a wrapper
# loop or cron for continuous coverage; one prediction = one polling
# window.
#
# Layout:
# - cog-relay/cog.yaml : you're here. Same image as cog-build
# (FPC + PasClaw binary + bundled
# OpenSSL 1.0.2) minus pget -- no
# workspace.zip download path here.
# - cog-relay/predict.py : the Predictor.
# - cog-relay/requirements.txt : Python deps.

build:
gpu: false
base_image: "debian:bookworm-slim"
system_packages:
- "fpc"
- "fp-units-db"
- "fp-units-misc"
- "lazarus-src"
- "libsqlite3-dev"
- "make"
- "git"
- "curl"
- "ca-certificates"
- "patchelf"
- "libsqlite3-0"
- "tzdata"
python_version: "3.10"
python_requirements: "requirements.txt"
run:
# 1. Clone PasClaw + Indy
- git clone https://github.com/FMXExpress/PasClaw.git /src
- cd /src && make get-indy

# 2. Compile the PasClaw binary
- cd /src && make LAZUTILS_DIR=/usr/lib/lazarus/2.2.6/components/lazutils

# 3. Runtime dir
- mkdir -p /opt/pasclaw
- cp /src/build/pasclaw /opt/pasclaw/pasclaw

# 4. OpenSSL 1.0.2 next to the binary (Indy's TLS path needs it)
- mkdir -p /openssl /tmp/x
- curl -fsSL --retry 3 --max-time 90 "https://snapshot.debian.org/archive/debian-archive/20240331T102506Z/debian-security/pool/updates/main/o/openssl1.0/libssl1.0.2_1.0.2u-1~deb9u7_amd64.deb" -o /tmp/libssl.deb
- dpkg-deb -x /tmp/libssl.deb /tmp/x
- cp /tmp/x/usr/lib/x86_64-linux-gnu/libssl.so.1.0.2 /opt/pasclaw/
- cp /tmp/x/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.2 /opt/pasclaw/
- ln -sf libssl.so.1.0.2 /opt/pasclaw/libssl.so
- ln -sf libcrypto.so.1.0.2 /opt/pasclaw/libcrypto.so
- chmod +x /opt/pasclaw/pasclaw
- patchelf --set-rpath '$ORIGIN' /opt/pasclaw/pasclaw

# 5. Cleanup
- rm -rf /src /tmp/x /tmp/libssl.deb /openssl

run: "predict.py:Predictor"
Loading