Skip to content

feat(web): worker sources graph, Workers on queue list, drop Submitted - #15

Merged
fahimfaisaal merged 75 commits into
mainfrom
feat/worker-view
Nov 9, 2026
Merged

fahimfaisaal merged 75 commits into
mainfrom
feat/worker-view

Conversation

@fahimfaisaal

Copy link
Copy Markdown
Member

Summary

Stacked on #14 (feat/job-page).

  • Worker page drops the Submitted card: varmq API 1.1.0 removed metrics.submitted from workers (a queue's submitted is its totals.submitted). Removed from api types, store record (older rows' value read and dropped) and fleet view; four cards in one row.
  • Sources graph on the worker page (varmq API 1.5.0, feat: worker strategy and sources in GET /workers/{name} (openapi 1.5.0) varmq#183): how the worker pulls.
    • Source nodes in pick order: store glyph (Redis, SQLite, memory) + name + FIFO/Priority badge, link to the queue page; "p N" chip under the priority strategy when set. Topic: broadcast glyph. ID-less storage: "custom storage", no link.
    • Edges join into the worker node, which names the strategy in words ("Priority: lowest number first", "Round robin", ...), or "Pulls directly" with one source.
    • Pure HTML/CSS, no JS; stacks at phone width. Hidden when data is stale.
    • Instances merged: first instance's sources in order, then what others add (key kind + Queue ID); an older pod's queue_ids join too. Strategy = first instance's.
    • Older varmq (no sources): nodes from queue_ids, no label, short note.
  • api + store: decode/store strategy, sources (priority *int64), omitempty, no StoreVersion bump.
  • Queue list: "Reported by" -> Workers: one chip per worker consuming the queue, linked, title "runs on ". No worker data (older varmq, producer-only) -> instance chips with a muted "instances" hint. Queue page header: muted "Read from instance-1, …" (workers are listed below it).
  • fleettest: SourceWorkers (3-source priority, 1-source, round-robin with topic + ID-less, older worker).
  • DESIGN.md, PRODUCT.md, README.

Evidence

  • go vet ./... clean; go test -race -count=1 ./... green (GOTOOLCHAIN=go1.27.0).
  • Tests in existing files: api decode with/without fields; store round trip + older row; fleet merge (instances, older pod, mixed fleet, kind key, no aliasing of the snapshot); web graph end to end (order, glyphs, links, chips, labels, topic, custom storage, legacy), label rules, Workers column + fallback, down pod skipped, live refresh names memory sources, hostile names/IDs + chip hrefs.
  • 6-lens review; findings fixed (mixed-fleet queues lost, legacy label).
  • Shots: Chrome 1280x800 DPR 2 (phone 375x812), vartop web over fake varmq.
Before After
Queue, dark before-queue-dark after-queue-dark
Queue, light before-queue-light after-queue-light
Queue, phone, dark before-queue-phone-dark after-queue-phone-dark
Queue, phone, light before-queue-phone-light after-queue-phone-light
Queues, dark before-queues-dark after-queues-dark
Queues, light before-queues-light after-queues-light
Queues, phone, dark before-queues-phone-dark after-queues-phone-dark
Queues, phone, light before-queues-phone-light after-queues-phone-light
Worker emails, light before-worker-emails-light after-worker-emails-light
Worker orders, dark before-worker-orders-dark after-worker-orders-dark
Worker orders, light before-worker-orders-light after-worker-orders-light
Worker orders, phone, dark before-worker-orders-phone-dark after-worker-orders-phone-dark
Worker orders, phone, light before-worker-orders-phone-light after-worker-orders-phone-light
Worker report maker, light before-worker-report-maker-light after-worker-report-maker-light
Worker settlements, light before-worker-settlements-light after-worker-settlements-light
Worker thumbnails, dark before-worker-thumbnails-dark after-worker-thumbnails-dark
Worker thumbnails, light before-worker-thumbnails-light after-worker-thumbnails-light
Worker thumbnails, phone, light before-worker-thumbnails-phone-light after-worker-thumbnails-phone-light

Merge Danger

  • Door: two-way. Display + decoded fields; store fields omitempty. Revert = drop commits.
  • Blast Radius: worker page and queue list. Needs varmq API 1.5.0 for strategy/priorities; older varmq degrades to queue_ids nodes.
  • Known gaps: instances with different sources are merged, not shown apart; differing strategies show the first. docs/images/worker-*.png still show the Submitted card. Two in-memory queues on different instances can share a name ("report-maker feat: add vartop web browser dashboard #1"), as before.

Remove EndpointSlice discovery, the kubernetes config block, wizard
prompts, KUBECONFIG in the collector lease, pod noun and k8s label mode.
Config with a kubernetes block now fails to load: "kubernetes discovery
was removed; use static or dns". Static and DNS unchanged. Drop
client-go deps. Lease keeps its unused kubeconfig column (written empty).

BREAKING CHANGE: kubernetes servers in config.json no longer load.
- config error names file path; text says endpoints or dns
- workers hint only on ErrConfigNotFound
- fix Close/PoolFor docs, stale Kubernetes comments, README typo
- test loadOrOnboard rejects kubernetes config
go directive stays 1.26.0 (contract needs 1.24). no fallout.
QueueSummary gains Paused (*bool: nil = varmq < 1.2.0) and PausedAt.
QueueAction PATCHes /queues/{id}/actions/{pause|resume}, decodes entry.
QueueActionUnsupported: router 404, 405, 501. JobStates adds cancelled.
One PATCH per unit reporting the queue; older unit (no paused) sent
nothing, no change. 501/405/router 404 = failed, not supported. JSON 404
= no change. Returns first ok entry for the badge. Group paused merged
from any copy that reports it. fake varmq: queue action route, shared
pause flag, ActionStatus.
Cancelled tab: count -, hollow bad dot, own timeline step; varmq 400 on
state=cancelled = note, not error. stateCount no longer falls through to
failed. POST /queues/{id}/actions/{pause|resume}: bind guards, confirm
panel, results, #qstate oob from PATCH answer. Buttons hidden when no
unit reports paused. Paused dot on queue list. Tabs scroll on phones.
api: paused decode, QueueAction path + body, unsupported mapping.
fleet: QueueAct fan-out, older unit skipped, 501/404/503/drop outcomes.
web: confirm then act, oob badge, list badge, older varmq hidden, 501
result, unreadable queue 409, cancelled tab, stale refusal.
Resume alone when every reporting instance says paused, Pause alone when
none does, both when mixed or unknown (nil paused). Group keeps per-unit
paused; QueueAct returns group as units answered, for the oob #qstate.
A unit that stores the flag but answers {id,error} (200) left the
group's paused nil: #qstate lost Pause/Resume after a good pause.
Take paused from the first answer that has one; keep the unit's own.
fleettest.Queue.ActionDrop: transport error on a queue action.
fleettest.Queue.JobsStatus; jobs route lists cancelled when Jobs has it.
200 means the flag was stored: that unit's paused, and the entry's when
no answer has paused, is the action's target. #qstate shows Resume
alone after such a pause, not both buttons. paused_at: g's on a repeat
pause, else unknown.
ListWorkflows, GetWorkflow, CancelWorkflow (undo=false opt). Router 404/405
-> RouteMissing, JSON 404 -> WorkflowNotFound, 409 -> WorkflowAmbiguous.
'.'/'..' IDs never sent: would name another route.
Group by (ID, created_at): durable one shared by units = one row, units as
On; in-memory twin = own row. Read skew: most advanced copy wins. List
merge reads each unit's first offset+limit+1, paged by route cap. Cancel
goes to first unit that takes it (stored with workflow), next on fail/404/409.
fleettest: workflow routes + SampleWorkflows (diamond w/ always, running
ETL w/ long edge, compensated saga, cancelled report).
- nav Workflows; list: state tabs, progress bar, duration, reported-by,
  merged; page 1 live, later pages browsed.
- detail: DAG laid out in Go (longest-path layers, dummy slots for long
  edges, barycenter sweeps), inline SVG. Always edges dashed, in-flight
  tasks pulse, edges into them flow; reduced motion off.
- workflow.js: pan/zoom (drag, ctrl+wheel, pinch, buttons, keys), trace
  up/downstream, task panel; view/panel/focus kept across polls.
- #live polls itself until workflow ends; #results hx-preserve.
- cancel / cancel without undo: confirm panel, bind() guards, re-read +
  created_at match, refused when ended; #wfctl oob.
- graph skipped past 5000 slots; table is fallback.
- liveClients shared by queue + workflow pages.
Cancel, zoom, panel close, graph well sit in #live (state follows record);
poll swap dropped their focus. workflow.js re-focuses same control.
Hint: Ctrl/Cmd + scroll.
Panel no longer covers graph: well narrows while open (below graph on
phone). Opening shows whole graph, else task lineage, else task +
downstream, else task centred; zoom stays >= 0.6.
Fan-in adds no dummies: r*r edges slipped under the cap (1M paths at
r=1000). Cap now tasks + edges + dummies; over it: note + table.
sortLayer re-placed whole graph per layer per sweep; now only the sorted
layer (column heights fixed). Deps filtered once.
FetchWorkflows stopped each unit at 10000 records; merged page at
offset=10000 then mixed truncated lists. Stop only past 10000, so a unit
yields up to 10000+limit and no request offset passes the cap.
- workflow.js loads on every workflow page (OwnPoll): a graph a later
  poll brings in still pans/zooms.
- focus restore keys task node, panel aside, panel links; oob afterSwap
  no longer re-inits a taken graph (dropped panel focus).
- stale at= with other workflows of the ID left: picker, not 404.
Edge cap (fan-in), BenchmarkLayoutDAG, hostile fields escaped, dup task
names, forms carry server, confirm posts confirm=1 back, cancel refused
by every unit keeps buttons, failing units keep polling, stale at, merge
ended-over-running tie, offset-cap paging, 503 cancel unit, guard route
tables. fleettest: WorkflowStatus knob, locked SetActionStatus.
Queue list rows and the worker page Queues line sort by name, namespace,
ID, as they show. GroupQueues keeps ID order for Group's search.
QueueConfirmView drops ID (Queue.ID holds it); ownersOf keeps every ID,
name() checks the backend; dup .qtitle CSS cut. Tests: confirm panel
names the queue, namespace cell and chip, workflow page owners, workers
on the queue order, older varmq worker lists none, redisq brace guard.
GET /queues/{id}/jobs/{jobId}: state, timeline with waited/ran/total
(live ones tick in job.js), errors, payload, result. Cancel behind bind
and confirm panel, only while pending or processing. Queue list and
workflow task rows/panel link job IDs; none for '.', '..', empty.
Worker page loses the Submitted card; a queue's submitted is its
totals.submitted on the queue pages.
Sources in pick order: store glyph, name, kind, priority chip under
priority strategy, link to queue page; topic and ID-less nodes. Worker
node says the strategy (2+ sources) or 'Pulls directly'. Older varmq:
nodes from queue_ids, a note. Queue list: Workers chips (by queue_ids),
instances as fallback. Queue page: 'Read from' line, muted.
@fahimfaisaal
fahimfaisaal changed the base branch from feat/job-page to main November 9, 2026 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant