[Node B] Inference, AST Parsing & Prompting — Full 6-Month Track (Y. Jangra)

[mohityadav8]

Owner: Y. Jangra · Core Contributor / Inference Lead
Hardware: RTX 4060 · 8GB VRAM (Qwen2.5-Coder-7B 4-bit AWQ + BAAI/bge-large embeddings)
Scope: Model serving, prompt engineering, AST parsing, code retrieval, frontend dashboard components, benchmarking
Duration: 6 Phases / ~26 weeks
Labels: node-b, inference, epic, ast-retrieval, frontend

---

🎯 Epic Goal

Own everything that makes the agent actually think well: the model serving layer, every prompt template (PLAN/IMPLEMENT/REFLECT/strategy-shift), the Tree-sitter AST parser that powers graph-aware retrieval, and — in later phases — the live dashboard's frontend components. Also owns the final SWE-bench Lite benchmark run and honest public documentation of what does and doesn't work.

📅 Timeline (6 Months)

gantt
    title Node B — Y. Jangra — 6 Month Roadmap
    dateFormat  YYYY-MM-DD
    axisFormat  %b
    section Phase I - Single Node
    Ollama Serving + Prompt Templates      :p1, 2026-07-06, 21d
    section Phase II - Reaper Protocol
    Reflection + Strategy-Shift Prompts    :p2, after p1, 21d
    section Phase III - AST Retrieval
    Tree-sitter Parser + Call Graph Extract:p3, after p2, 21d
    section Phase IV - Three-Node Mesh
    gRPC InferenceService Worker           :p4, after p3, 21d
    section Phase V - War Room Dashboard
    Next.js Frontend Components            :p5, after p4, 21d
    section Phase VI - Benchmark & Ship
    SWE-bench Lite Run + Docs              :p6, after p5, 21d

Loading

🧭 Definition of Done (Epic-level)

• All 6 phase demo milestones below are recorded and linked in this issue
• Model runs stably at 4-bit quantization within 8GB VRAM budget across all phases
• AST graph retrieval demonstrably outperforms flat embeddings on the 20-bug eval set
• SWE-bench Lite 50-issue benchmark published with honest pass/fail numbers, including failures

---

🟦 PHASE I — Single-Node Agent Loop

Demo Milestone: Document exact VRAM and latency numbers from the run.

• 01. Set up Ollama serving Qwen2.5-Coder-7B-Instruct (4-bit AWQ) on Node B, confirm start under 4.5GB VRAM
  • Acceptance: nvidia-smi snapshot at idle and under load both attached to this task
• 02. Benchmark tokens/sec + max usable context at 4-bit on 8GB — write the numbers down
  • Acceptance: numbers recorded in /docs/model-selection.md with exact prompt lengths tested
• 03. InferenceClient wrapper — POST to Ollama, stream tokens, timeout after N seconds
  • Acceptance: timeout is configurable via hydra.config.json; timed-out request cleans up server-side generation
• 04. PLAN prompt — given issue + repo structure, output a numbered step list as JSON
  • Acceptance: prompt versioned as plan_v1.md, tested against 5 varied issue descriptions with valid JSON output every time
• 05. IMPLEMENT prompt — given step N + file contents, output a unified diff
  • Acceptance: output diff is directly git apply-able without post-processing in 90%+ of test cases
• 06. REFLECT prompt — given test failure output, identify the most likely root cause
  • Acceptance: prompt tested against 5 seeded failure types (syntax, type, assertion, import, runtime)
• 07. StructuredOutputParser — extract JSON from model output even when wrapped in markdown prose
  • Acceptance: handles at least: raw JSON, ```json fenced blocks, JSON with leading/trailing prose
• 08. Diff-apply utility — takes a unified diff string, validates it, applies it to the file
  • Acceptance: malformed diff is rejected with a clear error before touching any file, never partially applied
• 09. Prompt version control — every prompt is a named, versioned .md template, no hardcoded strings in Python
  • Acceptance: changing a prompt requires no Python code change, only a template file edit
• 10. Phase I demo — document exact VRAM and latency numbers from the run
  • Acceptance: numbers published alongside M. Yadav's terminal recording demo

---

🟥 PHASE II — The Reaper Protocol

Demo Milestone: Show the PostMortemReport output for the demo bug.

• 01. SelfReflectionPrompt — given N failed attempts, produce structured analysis of why each failed
  • Acceptance: output is structured JSON (per-attempt failure reason), not free text
• 02. ForcedStrategyShiftPrompt — inject full failure history, explicitly forbid the dead approach
  • Acceptance: tested that the model does NOT repeat a forbidden file/approach in its next output, verified programmatically
• 03. StackTraceParser — extract file, line, error type from Python/Node/Go stack traces into a structured dict
  • Acceptance: covers all 3 languages with unit tests using real captured tracebacks
• 04. FailureMemoryStore — in-session store of (attempt_n, strategy, result, error_signature) injected into every subsequent prompt
  • Acceptance: memory store correctly truncates/summarizes when it would exceed context budget
• 05. HypothesisGeneratorPrompt — given the failure tree so far, generate 3 alternative approaches, pick highest-confidence
  • Acceptance: the 3 generated approaches are demonstrably distinct from each other and from prior dead branches (not cosmetic rewording)
• 06. ApproachRankingPrompt — given candidate strategies, score each by estimated probability of success
  • Acceptance: ranking is consistent (same inputs → same ranking) across repeated calls at temperature 0
• 07. Controlled A/B test — 10 seeded bugs, baseline vs. Reaper agent, measure attempts-to-resolution and token spend
  • Acceptance: results tabulated and committed; Reaper variant shows measurable improvement or the gap is honestly reported
• 08. ErrorClassifier — buckets stderr into syntax/type/runtime/assertion/unknown for better prompt routing
  • Acceptance: classifier accuracy measured against a hand-labeled set of 30 real error messages
• 09. PostMortemReport — after a successful run, generate human-readable explanation of every strategy attempted and why each was abandoned
  • Acceptance: report reads coherently to someone who didn't watch the run live (tested by having a teammate read it cold)
• 10. Phase II demo — show the PostMortemReport output for the demo bug
  • Acceptance: report linked/attached to this issue

---

🟩 PHASE III — AST Graph Retrieval

Demo Milestone: Show hydra ast-graph output for the demo repo.

• 01. Wire Tree-sitter Python grammar — parse a file into FunctionDef, ClassDef, Import node list
  • Acceptance: correctly parses a file with a deliberate syntax error without crashing (Tree-sitter's key advantage over ast)
• 02. Extract call graph — for each function call, record which function it calls and in which file
  • Acceptance: validated against a hand-built 5-file toy repo with known call relationships
• 03. Extract import graph — for each import/from...import, record source-target file edge
  • Acceptance: handles relative imports, aliased imports, and __init__.py re-exports correctly
• 04. ASTHasher — stable hash per function/class body for change detection without full re-parse
  • Acceptance: whitespace-only changes do NOT change the hash; logic changes DO
• 05. Wire Tree-sitter TypeScript grammar — test the same parser pipeline on a .ts file
  • Acceptance: same call/import graph extraction works on TS as on Python, same test coverage
• 06. MetadataExtractor — function signatures, docstrings, type annotations stored alongside each node
  • Acceptance: metadata is what actually gets shown to the model in retrieved context, not just stored inertly
• 07. SyntaxErrorLocator — when the agent writes invalid syntax, pinpoint the exact broken bracket/token
  • Acceptance: error message includes line + column + a human-readable description, not just a Tree-sitter node dump
• 08. CrossLanguageEdge — record edges where a Python file calls a TypeScript REST endpoint (boundary detection)
  • Acceptance: at least one working example in the toy repo showing a detected cross-language edge
• 09. ASTVisualizerCLI — hydra ast-graph --repo PATH --file auth.py, pretty-print the call graph as ASCII
  • Acceptance: output is legible in a terminal, matches the style of the README's AST graph walk example
• 10. Phase III demo — show hydra ast-graph output for the demo repo
  • Acceptance: recorded/screenshotted and linked in this issue

---

🟨 PHASE IV — Three-Node Mesh

Demo Milestone: Show the cluster status CLI output during the run.

• 01. Wrap Ollama as a gRPC InferenceService server on Node B
  • Acceptance: service implements the .proto contract defined by M. Yadav's track exactly, no ad-hoc extensions
• 02. Implement streaming gRPC — stream inference tokens back to the supervisor as they generate
  • Acceptance: first token latency measured and documented separately from total generation time
• 03. BackpressureController on Node B — reject new requests with UNAVAILABLE if context is nearly full
  • Acceptance: rejection happens before generation starts, not mid-generation
• 04. Request queuing on Node B — buffer incoming tasks during model warmup so they don't timeout immediately
  • Acceptance: a request sent during a cold-start warmup succeeds instead of failing
• 05. Graceful shutdown on Node B — finish in-flight inference before stopping, don't cut mid-generation
  • Acceptance: SIGTERM during active generation completes the current response before exiting
• 06. Measure + document round-trip network latency added by the gRPC hop
  • Acceptance: number is explicitly called out in benchmark docs as "network overhead," separated from model latency
• 07. Automatic model reload on Node B if Ollama crashes — detect dead process, restart, re-warm
  • Acceptance: chaos test kills the Ollama process mid-run and confirms auto-recovery within a documented SLA
• 08. Integration test — kill Node B mid-inference, confirm supervisor emits NODE_LOST and pauses cleanly
  • Acceptance: test is automated in CI, not manual
• 09. InferenceMetricsExporter — publish tokens/sec, queue depth, context utilization over gRPC to supervisor
  • Acceptance: metrics feed directly into M. Yadav's ResourceMatrix without a translation layer
• 10. Phase IV demo — show the cluster status CLI output during the run
  • Acceptance: screenshot/recording showing Node B's live VRAM and model-loaded status

---

🟪 PHASE V — War Room Dashboard

Demo Milestone: Show the ConvergenceGauge going red and the REAPER_FIRED toast appearing.

• 01. Scaffold Next.js 14 app — App Router, shadcn/ui, Tailwind — minimal, dark, terminal-inspired
  • Acceptance: matches the visual direction described in README (dark theme, terminal aesthetic)
• 02. LiveLogStream — virtualized list of incoming agent thought/action/observation events, auto-scrolls
  • Acceptance: stays smooth (60fps) with 10,000+ events in the list, using virtualization not naive rendering
• 03. GenealogyTree component — interactive tree of all attempted branches, color-coded (green=pass, red=dead, amber=active)
  • Acceptance: renders the exact tree structure shown in README's genealogy example, clickable nodes show attempt detail
• 04. NodeMeterPanel — VRAM bar for Node B, container count for Node C, heartbeat indicator for all three
  • Acceptance: updates live via WebSocket, not polling
• 05. DiffViewer — side-by-side diff of current file edit with syntax highlighting (react-diff-viewer-continued)
  • Acceptance: correctly highlights Python and TypeScript diffs
• 06. ConvergenceGauge — real-time 0–1 gauge for current approach's convergence score, turns red near 0.75
  • Acceptance: color transition is smooth/gradient, not a hard snap, and threshold is read from live config not hardcoded
• 07. WarRoomMode — fullscreen split view: Genealogy + LiveLog + NodeMeters + ConvergenceGauge simultaneously
  • Acceptance: usable/legible at 1920x1080 and on a 13" laptop screen
• 08. MobileResponsiveness — dashboard readable on a phone screen for checking a run remotely
  • Acceptance: tested on an actual mobile viewport, not just browser devtools resize
• 09. Playwright end-to-end tests — connect to a mock WebSocket, feed scripted event stream, assert UI state
  • Acceptance: tests run in CI headless, cover at least genealogy tree update and Reaper toast appearance
• 10. Phase V demo — show the ConvergenceGauge going red and the REAPER_FIRED toast appearing
  • Acceptance: recorded clip, linked in this issue

---

🟧 PHASE VI — Benchmark, Harden, Ship

Demo Milestone: Publish the raw benchmark report alongside the release.

• 01. Select representative 50-issue slice from SWE-bench Lite — document selection methodology
  • Acceptance: methodology explains why these 50 (not cherry-picked for easy wins), committed to /benchmarks/
• 02. Run HydraNet against all 50 issues — record pass/fail/reaper-fired/timeout for each
  • Acceptance: raw per-issue log retained, not just aggregate numbers
• 03. Publish raw benchmark data — /benchmarks/results.json: issue ID, outcome, attempts, tokens used
  • Acceptance: file is machine-readable and matches a documented schema
• 04. Fair comparison table — HydraNet vs. Aider vs. OpenHands vs. Cline on the same 50 issues, using their published numbers where available
  • Acceptance: every comparison number is sourced with a link; no invented numbers for competitors
• 05. Architecture deep-dive doc — /docs/architecture.md, every decision + reasoning, including wrong paths taken
  • Acceptance: doc includes at least 2 documented "we tried X, it didn't work, here's why" sections — honesty over polish
• 06. Model selection guide — what fits in 6GB, what fits in 8GB, what to do with less
  • Acceptance: guide gives concrete model+quantization recommendations per VRAM tier, not vague advice
• 07. Getting Started in 30 minutes tutorial — assumes three laptops + a WiFi router, nothing else
  • Acceptance: tested by having someone unfamiliar with the project follow it cold and time it
• 08. Prompt engineering guide — how to get better results from HydraNet via better issue descriptions
  • Acceptance: includes concrete before/after issue description examples
• 09. Failure analysis doc — which bugs HydraNet consistently fails on, and known limitations
  • Acceptance: at least 5 concrete documented failure categories, not a vague disclaimer
• 10. Ship — post the benchmark report alongside the release
  • Acceptance: report linked from the v0.1.0 GitHub release notes

---

🔗 Dependencies

• Phase III AST retrieval feeds directly into M. Yadav's RetrievalAPI (Node A) — schema must be agreed before Phase III task 02
• Phase IV gRPC InferenceService must conform exactly to the .proto contract owned by M. Yadav's track
• Phase VI benchmark numbers block M. Yadav's Phase VI task 01 (ship) — this track's task 03 is a hard dependency for release

⚠️ Risks

• 7B model at 4-bit on 8GB leaves limited headroom for long-context retrieval-augmented prompts — watch context budget closely in Phase III
• SWE-bench Lite numbers may be underwhelming vs. cloud frontier models — commit now to publishing honestly (README already states this is not a 70%+ SWE-bench tool)