requestor: lookout and lookout ingester changes

[nikola-jokic]

What type of PR is this?

What this PR does / why we need it

Which issue(s) this PR fixes

Fixes #

Special notes for your reviewer

[datadog-armadaproject]

⚠️ Warnings

🚦 3 Pipeline jobs failed

CI | All jobs succeeded

View in Datadog · View in GitHub Actions

CI | lint / Lint Go

View in Datadog · View in GitHub Actions

1 typecheck error in internal/lookoutingester/instructions/instructions_test.go:289:53: 'CancelledJob' type has no field or method 'CancelUser'

CI | test / Golang Unit Tests

View in Datadog · View in GitHub Actions

Build failed for package 'github.com/armadaproject/armada/internal/lookoutingester/instructions'.

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 5e1bb25 | Docs | View more details | Give us feedback!}

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[greptile-apps]

Greptile Summary

This PR extends Lookout requestor attribution across ingestion, persistence, schema, and query models.

• Adds preemption and reprioritization requestor fields to Lookout database schemas and migrations.
• Extracts cancellation, preemption, and reprioritization actors from events and persists them with field-specific overwrite semantics.
• Exposes the new fields through Lookout and query API models and adds ingestion/repository coverage.

Confidence Score: 4/5

The PR is not yet safe to merge because batch conflation can still misattribute requestors and attach cancellation metadata to a non-cancelled terminal job.

The current conflation code continues to overwrite first-write-wins cancellation and preemption actors with later batch values, while cancellation metadata is still merged after a terminal state blocks the corresponding cancellation transition.

Files Needing Attention: internal/lookoutingester/lookoutdb/insertion.go

Important Files Changed

Filename	Overview
internal/lookoutingester/lookoutdb/insertion.go	Adds requestor persistence and terminal-job handling, but both previously reported conflation defects remain at current HEAD.
internal/lookoutingester/instructions/instructions.go	Extracts cancellation, preemption, and reprioritization requestors into job update instructions.
internal/lookout/schema/migrations/036_add_preempt_and_reprioritize_users.sql	Adds nullable preemption and reprioritization requestor columns to the Lookout job table.
internal/lookout/repository/getjobs.go	Reads the new requestor columns and maps them into Lookout job models.
internal/server/queryapi/database/models.go	Extends query API database models with requestor attribution fields.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  E[Armada lifecycle event] --> C[Lookout instruction conversion]
  C --> B[Batch conflation]
  B --> P[(Lookout job row)]
  P --> Q[Lookout query repository]
  Q --> A[API job model]

Loading

_{Reviews (26): Last reviewed commit: "ok" | Re-trigger Greptile}

[Copilot]

🟡 Not ready to approve

Root-level Node files (package.json/package-lock.json) and a couple of small maintainability issues should be resolved/clarified before merge to avoid confusing or unintended build/dependency behavior.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR extends Armada’s Lookout/Lookout-ingester pipeline to persist and surface “actor” (requestor) information for job preemption and reprioritisation (and aligns cancellation handling), including DB schema changes, ingestion/conversion updates, repository query updates, and additional tests.

Changes:

• Add preempt_user and reprioritize_user fields end-to-end (ingester model → DB write path → schema/migration → Lookout repository reads → API model).
• Update event-to-instruction conversion to prefer trimmed requestor fields and ingest JobPreemptionRequested into PreemptUser.
• Adjust terminal-job filtering/conflation behavior and expand tests to preserve/store these actor fields correctly.

File summaries

File	Description
package.json	Introduces a repo-root Node manifest (currently empty).
package-lock.json	Introduces a repo-root npm lockfile.
internal/server/queryapi/query_api_test.go	Moves per-test DB context creation inside subtests.
internal/server/queryapi/database/models.go	Adds PreemptUser/ReprioritizeUser fields to query API DB model.
internal/scheduler/scheduling/preempting_queue_scheduler.go	Formatting-only changes (trailing commas/parentheses).
internal/scheduler/scheduling/preempting_queue_scheduler_test.go	Formatting-only changes (trailing commas/parentheses).
internal/lookoutingester/model/model.go	Adds PreemptUser/ReprioritizeUser to update instructions.
internal/lookoutingester/lookoutdb/insertion.go	Adds new columns to job updates, preserves/filters terminal-job actor updates, and updates SQL update semantics.
internal/lookoutingester/lookoutdb/insertion_test.go	Adds coverage for storing/preserving new actor fields and cancellation semantics.
internal/lookoutingester/instructions/instructions.go	Adds requestor normalization helper and converts preemption request events into PreemptUser.
internal/lookoutingester/instructions/instructions_test.go	Updates/extends converter tests for requestor fields and preemption request handling.
internal/lookouthc/schema/target_schema.sql	Adds new columns to target schema definition.
internal/lookouthc/schema/schema.go	Updates shape checks and repartitioning copy to include new columns.
internal/lookout/swagger.yaml	No functional change (whitespace-only).
internal/lookout/schema/migrations/035_add_preempt_and_reprioritize_users.sql	Adds DB migration for new columns.
internal/lookout/repository/util.go	Updates job simulator to use Requestor and adds reprioritisation-by actor support.
internal/lookout/repository/tables.go	Adds table/column mappings for new fields.
internal/lookout/repository/querybuilder.go	Adds new columns to GetJobs query selection.
internal/lookout/repository/getjobs.go	Scans/maps new columns into Lookout job model.
internal/lookout/repository/getjobs_test.go	Adds tests validating CancelUser, PreemptUser, and ReprioritizeUser are returned.
internal/lookout/repository/fieldparser.go	Normalizes last-transition timestamp formatting to UTC.
internal/lookout/model/model.go	Adds PreemptUser/ReprioritizeUser to Lookout job model.
internal/lookout/conversions/convert_test.go	Updates conversion tests to include CancelUser field.

Review details

Files not reviewed (1)

• internal/server/queryapi/database/models.go: Generated file

• Files reviewed: 20/23 changed files
• Comments generated: 4
• Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.