Context
The current group() agent method groups artifacts AND synthesizes each group in one call (list[Group] with key + artifacts + content). This works but can't parallelize across groups since the agent does everything.
Proposal
Add a second group operation type: group + feed forward. The agent assigns artifacts to named groups, but does NOT synthesize them. The groups are then fed to downstream transforms for parallel processing.
# Current: group + synthesize (agent does both)
def group(self, artifacts: list[Artifact]) -> list[Group]
# New: group + feed forward (agent just routes, downstream processes)
def route(self, artifacts: list[Artifact]) -> dict[str, list[Artifact]]
This is fundamentally a different transform primitive — a "router" or "splitter" that creates named subsets for parallel downstream processing. It enables:
- LLM-driven semantic grouping (agent decides grouping via embeddings or classification)
- Parallel per-group synthesis via downstream MapSynthesis/ReduceSynthesis
- Dynamic pipeline branching based on content analysis
Relationship to current work
The group() method (group + synthesize in one call) ships first. This issue tracks adding the parallel-friendly variant as a follow-up.
Context
The current
group()agent method groups artifacts AND synthesizes each group in one call (list[Group]with key + artifacts + content). This works but can't parallelize across groups since the agent does everything.Proposal
Add a second group operation type: group + feed forward. The agent assigns artifacts to named groups, but does NOT synthesize them. The groups are then fed to downstream transforms for parallel processing.
This is fundamentally a different transform primitive — a "router" or "splitter" that creates named subsets for parallel downstream processing. It enables:
Relationship to current work
The
group()method (group + synthesize in one call) ships first. This issue tracks adding the parallel-friendly variant as a follow-up.