Add experimental OpenVINO GFX plugin with Metal and OpenCL backends

.github/workflows/linux.yml

@@ -123,6 +123,7 @@ jobs:
             -DBUILD_nvidia_plugin=OFF \
             -DBUILD_ollama_openvino=OFF \
             -DBUILD_llama_cpp_plugin=OFF \
+            -DBUILD_gfx_plugin=OFF \
             -DENABLE_INTEL_GPU=OFF \
             -DENABLE_OV_TF_FRONTEND=OFF \
             -DENABLE_OV_PADDLE_FRONTEND=OFF \

.github/workflows/mac.yml

@@ -101,12 +101,13 @@ jobs:
             -GNinja \
             -DCMAKE_VERBOSE_MAKEFILE=ON \
             -DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} \
-            -DBUILD_nvidia_plugin=OFF \
-            -DBUILD_ollama_openvino=OFF \
-            -DBUILD_llama_cpp_plugin=OFF \
-            -DOPENVINO_EXTRA_MODULES=${{ env.OPENVINO_CONTRIB_REPO }}/modules \
-            -DCMAKE_OSX_DEPLOYMENT_TARGET=${{ env.MACOSX_DEPLOYMENT_TARGET }} \
-            -DCMAKE_OSX_ARCHITECTURES=${{ env.OSX_ARCHITECTURES }} \
+            -DBUILD_nvidia_plugin=OFF \
+            -DBUILD_ollama_openvino=OFF \
+            -DBUILD_llama_cpp_plugin=OFF \
+            -DBUILD_gfx_plugin=OFF \
+            -DOPENVINO_EXTRA_MODULES=${{ env.OPENVINO_CONTRIB_REPO }}/modules \
+            -DCMAKE_OSX_DEPLOYMENT_TARGET=${{ env.MACOSX_DEPLOYMENT_TARGET }} \
+            -DCMAKE_OSX_ARCHITECTURES=${{ env.OSX_ARCHITECTURES }} \
             -DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
             -DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
             -DENABLE_OV_PADDLE_FRONTEND=OFF \

.github/workflows/windows.yml

@@ -143,12 +143,13 @@ jobs:
           & "C:\tools\opencv\build\setup_vars_opencv4.cmd"
           cmake -GNinja `
             -DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} `
-            -DBUILD_nvidia_plugin=OFF `
-            -DBUILD_ollama_openvino=OFF `
-            -DBUILD_llama_cpp_plugin=OFF `
-            -DENABLE_OV_TF_FRONTEND=OFF `
-            -DENABLE_OV_PADDLE_FRONTEND=OFF `
-            -DENABLE_OV_TF_LITE_FRONTEND=OFF `
+            -DBUILD_nvidia_plugin=OFF `
+            -DBUILD_ollama_openvino=OFF `
+            -DBUILD_llama_cpp_plugin=OFF `
+            -DBUILD_gfx_plugin=OFF `
+            -DENABLE_OV_TF_FRONTEND=OFF `
+            -DENABLE_OV_PADDLE_FRONTEND=OFF `
+            -DENABLE_OV_TF_LITE_FRONTEND=OFF `
             -DENABLE_OV_PYTORCH_FRONTEND=OFF `
             -DENABLE_INTEL_GPU=OFF `
             -DENABLE_SAMPLES=OFF `

.gitignore

@@ -2,4 +2,5 @@
 # SPDX-License-Identifier: Apache-2.0
 
 .vscode/
+build*
 /builds/

.gitmodules

@@ -0,0 +1,11 @@
+[submodule "modules/gfx_plugin/third_party/llvm-project"]
+	path = modules/gfx_plugin/third_party/llvm-project
+	url = https://github.com/llvm/llvm-project.git
+[submodule "modules/gfx_plugin/third_party/clvk"]
+	path = modules/gfx_plugin/third_party/clvk
+	url = https://github.com/kpet/clvk.git
+	branch = main
+[submodule "modules/gfx_plugin/third_party/clspv"]
+	path = modules/gfx_plugin/third_party/clspv
+	url = https://github.com/google/clspv.git
+	branch = llvm/release/22.x

modules/gfx_plugin/.codex/skills/gfx-plugin-dev/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "GFX Plugin Dev"
+  short_description: "Build and refactor the OpenVINO GFX plugin"
+  default_prompt: "Use $gfx-plugin-dev to implement or refactor a GFX plugin change."

modules/gfx_plugin/.codex/skills/gfx-plugin-docs/SKILL.md

@@ -0,0 +1,180 @@
+---
+name: gfx-plugin-docs
+description: Use when updating published documentation for modules/gfx_plugin after code changes, especially README, ARCHITECTURE, DEVELOPMENT, TESTING, USAGE, profiling, or microbench docs.
+---
+
+# GFX Plugin Docs
+
+This skill keeps `modules/gfx_plugin` documentation aligned with the current
+implementation.
+
+## Use This Skill When
+
+- The code changed and docs may be stale.
+- Public properties, device semantics, backend routes, supported shapes,
+  profiling, microbench output, or test layout changed.
+- The task asks to clean public docs, repo-local skills, or developer
+  instructions.
+
+## Published Docs
+
+Primary files:
+
+- `README.md`
+- `docs/ARCHITECTURE.md`
+- `docs/DEVELOPMENT.md`
+- `docs/TESTING.md`
+- `docs/USAGE.md`
+
+Operational files:
+
+- `docs/MICROBENCH_SCHEMA.md`
+- `docs/PROFILING_RUNBOOK.md`
+
+Repo-local skills can also be public orientation material:
+
+- `.codex/skills/gfx-plugin-dev/SKILL.md`
+- `.codex/skills/gfx-plugin-docs/SKILL.md`
+- `.codex/skills/gfx-plugin-testing/SKILL.md`
+- `.codex/skills/gfx-plugin-profiling/SKILL.md`
+- `.codex/skills/gfx-plugin-release-sync/SKILL.md`
+
+Do not edit `AGENTS.md` unless the user explicitly asks.
+
+## Update Strategy
+
+1. Inspect the live source tree and current diff first.
+2. Identify whether the change affects:
+   - backend availability, default backend resolution, or CMake-selected
+     backend registration/stub translation units
+   - compiler service, backend registry, lowering-plan, manifest, executable
+     bundle, pipeline-stage builder, pipeline-stage fusion selection,
+     compiler-side pipeline-stage I/O plan, runtime pipeline-stage plan,
+     memory plan, cache envelope, runtime executable descriptor builder,
+     stage-placement policy, stage compiler policy, fused-output lifetime plan,
+     runtime pipeline-stage materializer, runtime session, or runtime
+     descriptor behavior
+   - public compiled-model cache boundaries, including `ov::cache_dir`,
+     `export_model()`, `import_model()`, cache-envelope wire/store/load
+     contracts, cache import/repository behavior, backend payload codecs, and
+     whether the envelope is a plugin-owned public runtime contract or
+     internal-only
+   - descriptor-owned tensor binding contracts, generated source
+     `RuntimeParams` ownership, descriptor-owned `ConstTensor`
+     materialization, or fail-closed source-node boundary policy
+   - compiler-owned tensor-layout classification
+   - public properties
+   - `query_model()` or compile behavior
+   - MLIR support/lowering/backend hooks and backend-owned source planning
+   - Metal placement, MPSRT records, MPS/MPSGraph descriptors,
+     vendor primitive artifact materialization, shared MPSRT vendor contracts,
+     `VendorDescriptor` payloads, generated activation/elementwise/reduction/
+     Softmax/LogSoftmax MSL routes, `Swish` static/runtime beta contracts,
+     Pool2D vendor-route-only behavior, or MSL binding
+   - OpenCL source-artifact coverage, runtime-shape handling, static f32
+     scalars, constants, generated activation/elementwise units, generated f32
+     Conv2D/GroupConv2D units, generated f32 MatMul units,
+     generated f32/f16 Softmax units,
+     dynamic-static-rank Softmax units, generated f32/f16 Pool2D units,
+     generated f32/f16/i64 Range units, generated f32/f16 Interpolate units,
+     generated f32 numeric reduction units, generated boolean logical reduction
+     units, generated ShapeOf/Tile units, generated compare/select and
+     logical-bool elementwise units, the
+     `opencl_kernel_unit_catalog.*` route catalog, family-specific OpenCL
+     kernel-unit adapters, `Swish` default/static/runtime beta artifacts,
+     missing-route behavior for Transpose/Concat/Split and unsupported MatMul
+     variants, chunking, or boolean-buffer behavior
+   - OpenCL remote context/tensor behavior, external `cl_mem` validation, and
+     backend-unavailable adapters
+   - CLVK/CLSPV Raspberry OpenCL bundle wiring, OpenCL dynamic-loader search
+     order, or third-party submodule publication
+   - OpenCL runtime-bundle candidate ordering or bundled tool-path setup
+   - backend-owned OpenCL payload materialization in
+     `src/backends/opencl/compiler/opencl_kernel_artifacts.*`
+   - removal or reintroduction risk around `BackendLowering`,
+     `metal_lowering`, `mps_graph_attention_stage`, or source-signature ABI
+     fallback behavior
+   - backend stage placement, stage policy, parallelism, partitioning,
+     submission, caches, or workspace allocation
+   - stateful `ReadValue` / `Assign`
+   - output aliasing, compiler-owned pipeline-stage I/O planning, fused-output
+     lifetimes, pipeline-stage materialization, or source-node-aware output
+     resolution
+   - descriptor-backed view-only stages
+   - compare-runner, profiling, trace sinks, microbench, or target-profile
+     output
+   - test layout, controlled test `plugins.xml`, gtest matrix checks,
+     disabled-pattern hooks, or validation workflow
+3. Patch only docs/skills whose contract changed.
+4. Keep wording concrete and tied to actual files that exist.
+
+For documentation/security publication tasks, do not run build or test targets
+unless the user explicitly asks for that validation. Use source inspection,
+security/stale-reference grep, `git diff --check`, and staged diff review for
+the publication gate.
+
+## Mapping
+
+- `src/plugin/` or property changes:
+  - `README.md`
+  - `docs/USAGE.md`
+  - `docs/DEVELOPMENT.md`
+
+- `src/compiler/`, compiler backend policies, manifests, executable bundles, or
+  runtime descriptors:
+  - `README.md`
+  - `docs/ARCHITECTURE.md`
+  - `docs/DEVELOPMENT.md`
+  - `docs/TESTING.md`
+
+- `src/runtime/`, scheduling, caches, memory, or profiling:
+  - `README.md`
+  - `docs/ARCHITECTURE.md`
+  - `docs/DEVELOPMENT.md`
+  - `docs/TESTING.md`
+  - `docs/PROFILING_RUNBOOK.md` when counters or profiling flow changed
+
+- `src/mlir/`, `src/kernel_ir/`, or `src/transforms/`:
+  - `README.md` for user-visible support changes
+  - `docs/ARCHITECTURE.md`
+  - `docs/DEVELOPMENT.md`
+  - `docs/TESTING.md`
+
+- Metal backend or MPSRT changes:
+  - `README.md`
+  - `docs/ARCHITECTURE.md`
+  - `docs/DEVELOPMENT.md`
+  - `docs/TESTING.md`
+  - `docs/USAGE.md` for public properties or diagnostics
+
+- OpenCL backend or source-artifact changes:
+  - `README.md`
+  - `docs/ARCHITECTURE.md`
+  - `docs/DEVELOPMENT.md`
+  - `docs/TESTING.md`
+  - `docs/USAGE.md`
+
+- compare/profiling/microbench tools:
+  - `docs/TESTING.md`
+  - `docs/USAGE.md`
+  - `docs/MICROBENCH_SCHEMA.md`
+  - `docs/PROFILING_RUNBOOK.md`
+
+## Writing Rules
+
+- Do not describe removed architectures or removed backends as current.
+- Do not promise backend parity unless tests and code support it.
+- Use exact property names such as `GFX_BACKEND`, `ov::available_devices`, and
+  `ov::device::id`.
+- Mention concrete source or test files when that helps orientation.
+- Keep README public-facing; keep deep mechanics in `docs/ARCHITECTURE.md` and
+  `docs/DEVELOPMENT.md`.
+- Keep docs in English.
+- Keep local dumps, build artifacts, machine paths, sensitive access material, and agent notes
+  out of public docs and commits.
+
+## Output Expectations
+
+- State which docs changed and why.
+- State what stale or private content was removed.
+- If no doc change is needed after inspection, say that explicitly.

modules/gfx_plugin/.codex/skills/gfx-plugin-docs/agents/openai.yaml

@@ -0,0 +1,4 @@
+interface:
+  display_name: "GFX Plugin Docs"
+  short_description: "Keep GFX plugin docs aligned with code"
+  default_prompt: "Use $gfx-plugin-docs to sync GFX plugin documentation with the current code."

modules/gfx_plugin/.codex/skills/gfx-plugin-profiling/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: gfx-plugin-profiling
+description: Use when analyzing GFX plugin performance, profiling reports, microbench output, trace correlation, or backend overhead on macOS, Android, Linux, or Raspberry Pi for Metal and OpenCL paths.
+---
+
+# GFX Plugin Profiling
+
+This skill is for performance triage and profiling workflows in
+`modules/gfx_plugin/`.
+
+## Use This Skill When
+
+- The user asks about performance, bottlenecks, slow inference, submit overhead,
+  cache effects, transfer pressure, or synchronization cost.
+- The task mentions `GFX_PROFILING_REPORT`, `ov_gfx_microbench`, MB0-MB3,
+  Perfetto, AGI, Xcode/Instruments, or `perf`.
+- The task needs interpretation of profiling JSON, calibration artifacts, or
+  trace output.
+- The task compares `metal`, `opencl`, or `auto` and needs to confirm which
+  backend route actually ran.
+
+## Read First
+
+1. `docs/PROFILING_RUNBOOK.md`
+2. `docs/MICROBENCH_SCHEMA.md`
+3. `docs/USAGE.md`
+4. `docs/TESTING.md`
+5. `README.md`
+
+## Tooling Surface
+
+- `ov_gfx_microbench`
+- `GFX_PROFILING_REPORT`
+- `OV_GFX_PROFILE_TRACE` trace sinks registered through
+  `src/runtime/gfx_profiling_trace_sink.*`
+- `tools/gfx_profile_runbook.py`
+- `tools/gfx_microbench_smoke.py`
+- `tools/gfx_calibration_diff.py`
+- `tools/gfx_external_trace_summary.py`
+
+Native trace surfaces:
+
+- macOS: Instruments, `xcrun xctrace`, Xcode GPU capture, Metal-registered
+  `signpost` / `os_signpost` trace sinks
+- Android: Perfetto or AGI
+- Linux/Raspberry Pi: `perf stat`, `perf record`
+
+## Core Workflow
+
+1. Rebuild the relevant binary.
+2. Run `ov_gfx_microbench` and save report plus calibration JSON.
+3. Run the real workload with profiling enabled.
+4. Capture a platform trace only when plugin counters point to a category.
+5. Correlate:
+   - `analysis.triage_hints`
+   - `benchmarks[].derived`
+   - `benchmarks[].profile_digest`
+   - `GFX_PROFILING_REPORT.compile`
+   - `GFX_PROFILING_REPORT.extended`
+   - `GFX_PROFILING_REPORT.extended.target_profile`
+
+## Inspect First
+
+### Fixed Overhead
+
+- MB0 fixed overhead
+- `fixed_overhead_us`
+- `fixed_overhead_share`
+- first-to-steady ratios
+
+### Synchronization
+
+- wait segments
+- `wait_share_of_wall`
+- final fence wait counters
+- submit and barrier counts
+- dependency-window extension counters
+
+### Transfers
+
+- H2D/D2H bytes
+- upload/download spans
+- `transfer_share_of_wall`
+- remote tensor and output reuse paths
+
+### Binding Or Descriptor Churn
+
+- descriptor or binding update counters
+- prepared binding cache behavior
+- `binding_prepare_in_infer`
+- Metal `mpsrt_*` resource-binding counters
+- OpenCL program/kernel setup spans
+
+### Compile Or Cache Regression
+
+- `compile_ms`
+- pipeline creation counters
+- OpenCL program-cache behavior
+- Metal MSL/MPSRT prepared-pipeline cache counters
+- first-infer versus steady-state timing
+
+## Interpretation Rules
+
+- Treat microbench numbers as heuristics, not peak hardware claims.
+- Use `ov_gfx_compare_runner` for correctness and `ov_gfx_microbench` for
+  profiling triage.
+- Distinguish wall time, GPU time, and overhead-subtracted estimates.
+- Confirm `actual_backend`, `extended.target_profile`, or target backend
+  counters before comparing `auto`, Metal, and OpenCL runs.
+- Treat standalone OpenCL microbench output as kernel evidence only; plugin
+  performance claims require execution through the GFX OpenCL backend.
+
+## Platform Notes
+
+### macOS
+
+- Prefer signposts or `xctrace` when correlating CPU and Metal execution.
+- Compare command-buffer timing with plugin stage/segment timing.
+- Capture GFX profile output with compare-runner when placement counters and
+  accuracy need to be tied to the same run.
+
+### Android / OpenCL
+
+- Use Perfetto or AGI for GPU busy/idle gaps and CPU blocking around queue
+  submits or waits.
+- Confirm the target has a working OpenCL GPU runtime before interpreting
+  backend performance.
+
+### Linux / Raspberry Pi OpenCL
+
+- Use `perf stat` and `perf record` for CPU-side attribution.
+- Keep backend, device key, model, runtime libraries, and calibration artifact
+  matched before comparing runs.
+
+## Output Expectations
+
+- State the most likely bottleneck category first.
+- Point to exact fields or tool outputs.
+- Recommend the next measurement step or fix target.