diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index e3abdf6..db43a56 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -19,7 +19,7 @@ -- [ ] The canonical proto and API documentation are synchronized +- [ ] The canonical schema and API documentation are synchronized - [ ] Presence, defaults, and invalid states are defined - [ ] Acceptance, streaming, terminal, and error behavior are defined - [ ] Discovery and capability impacts are documented diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 74417b5..85ece56 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -20,11 +20,12 @@ limitations under the License. Thank you for your interest in OpenEngine. We welcome bug reports, feedback on the API design, and pull requests. -OpenEngine is a pre-adoption API draft. `proto/openengine/v1/openengine.proto` is -the canonical wire contract and the single source of truth. Until an external -consumer adopts it, the schema may remove or renumber fields to stay minimal. -After external adoption, changes within `openengine.v1` will be additive. Please -open an issue to discuss protocol changes before sending a PR. +OpenEngine is a pre-adoption API draft. The files under +`proto/openengine/v1/` are the canonical wire contract and the single source of +truth. Until an external consumer adopts it, the schema may remove or renumber +fields to stay minimal. After external adoption, changes within `openengine.v1` +will be additive. Please open an issue to discuss protocol changes before +sending a PR. - **Bugs / feedback / design questions**: open a [GitHub issue](https://github.com/ai-dynamo/openengine/issues). - **Pull requests**: open against `main`. Keep changes focused (one logical change per PR). diff --git a/README.md b/README.md index f80a5ff..aadc6fc 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ SPDX-License-Identifier: Apache-2.0 Buf CI Apache 2.0 License Status: Experimental - API: openengine.v1 + API: openengine.v1 Transport: gRPC Schema: Protocol Buffers

@@ -27,7 +27,7 @@ SPDX-License-Identifier: Apache-2.0

Why OpenEngine? · API reference - · Canonical proto + · Canonical schema · Contributing

@@ -98,8 +98,9 @@ exist alongside OpenEngine. ## Capabilities -The complete service is defined in -[`proto/openengine/v1/openengine.proto`](proto/openengine/v1/openengine.proto). +The canonical schema is organized by domain under +[`proto/openengine/v1/`](proto/openengine/v1/), with the service definition in +[`openengine.proto`](proto/openengine/v1/openengine.proto). | Area | What the contract provides | | --------------------- | ----------------------------------------------------------------------------------------------------------------- | @@ -153,11 +154,11 @@ python -m grpc_tools.protoc \ -I "$PROTO_INCLUDE" \ --python_out="$OUT_DIR" \ --grpc_python_out="$OUT_DIR" \ - proto/openengine/v1/openengine.proto + proto/openengine/v1/*.proto ``` Other protobuf-supported languages can generate clients and servers from the -same canonical file. +same canonical package. ## Project status @@ -187,8 +188,8 @@ git commit --signoff -m "docs: describe the change" ``` Please validate protobuf changes with Buf and keep -[`proto/openengine/v1/openengine.proto`](proto/openengine/v1/openengine.proto) and -[`docs/api.md`](docs/api.md) synchronized. +[`proto/openengine/v1/`](proto/openengine/v1/) and [`docs/api.md`](docs/api.md) +synchronized. ## Security diff --git a/docs/api.md b/docs/api.md index fe8d3a3..8f5c3da 100644 --- a/docs/api.md +++ b/docs/api.md @@ -5,9 +5,9 @@ SPDX-License-Identifier: Apache-2.0 # OpenEngine API v1 -This is the human-readable reference for -[`openengine.v1`](../proto/openengine/v1/openengine.proto). -The proto is the source of truth. +This is the human-readable reference for the +[`openengine.v1` schema](../proto/openengine/v1/). The proto package is the +source of truth. --- diff --git a/docs/motivation.md b/docs/motivation.md index 667ccdc..c2b695d 100644 --- a/docs/motivation.md +++ b/docs/motivation.md @@ -67,7 +67,7 @@ request and control data needed across the process boundary. | KV routing | Event streams and native event-source discovery | The [API reference](api.md) defines the fields. The -[proto](../proto/openengine/v1/openengine.proto) is the source of truth. +[`openengine.v1` schema](../proto/openengine/v1/) is the source of truth. ## What stays engine-specific diff --git a/proto/openengine/v1/README.md b/proto/openengine/v1/README.md new file mode 100644 index 0000000..9b451ca --- /dev/null +++ b/proto/openengine/v1/README.md @@ -0,0 +1,25 @@ + + +# OpenEngine v1 schema + +This directory is the canonical `openengine.v1` wire contract. All proto files +share the same package and together define the API. + +| File | Area | +| --- | --- | +| [`openengine.proto`](openengine.proto) | `OpenEngine` service and RPC declarations | +| [`engine.proto`](engine.proto) | Engine identity, roles, and parallelism | +| [`model.proto`](model.proto) | Model metadata and generation capabilities | +| [`generation.proto`](generation.proto) | Generation requests, streamed events, and usage | +| [`generation_params.proto`](generation_params.proto) | Sampling, stopping, response, KV, and guided-decoding parameters | +| [`lora.proto`](lora.proto) | LoRA adapter lifecycle | +| [`kv.proto`](kv.proto) | KV sessions, connector discovery, and cache events | +| [`lifecycle.proto`](lifecycle.proto) | Health, abort, and drain operations | +| [`observability.proto`](observability.proto) | Load snapshots and runtime events | +| [`error.proto`](error.proto) | Terminal errors for accepted streaming requests | + +Generate bindings from every `.proto` file in this directory. Compiling only +`openengine.proto` does not generate bindings for its imported message files. diff --git a/proto/openengine/v1/engine.proto b/proto/openengine/v1/engine.proto new file mode 100644 index 0000000..6766624 --- /dev/null +++ b/proto/openengine/v1/engine.proto @@ -0,0 +1,40 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Engine identity, role, and parallelism metadata. + +syntax = "proto3"; + +package openengine.v1; + +import "openengine/v1/kv.proto"; + +enum EngineRole { + ENGINE_ROLE_UNSPECIFIED = 0; + ENGINE_ROLE_AGGREGATED = 1; + ENGINE_ROLE_PREFILL = 2; + ENGINE_ROLE_DECODE = 3; +} + +message GetEngineInfoRequest {} + +message EngineInfo { + string engine_name = 1; // sglang, vllm, tensorrt_llm, etc. + string engine_version = 2; + EngineRole role = 3; + string instance_id = 4; + repeated string supported_models = 5; + ParallelismInfo parallelism = 6; + KvConnectorInfo kv_connector = 7; + uint32 schema_revision = 8; // Monotonic wire contract revision; zero is invalid. + uint32 minimum_client_revision = 9; // Oldest compatible client revision. + string schema_release = 10; // Immutable release or source tag for this schema. +} + +message ParallelismInfo { + optional uint32 tensor_parallel_size = 1; + optional uint32 pipeline_parallel_size = 2; + optional uint32 data_parallel_size = 3; + optional uint32 data_parallel_rank = 4; + optional uint32 data_parallel_start_rank = 5; +} diff --git a/proto/openengine/v1/error.proto b/proto/openengine/v1/error.proto new file mode 100644 index 0000000..dfba86f --- /dev/null +++ b/proto/openengine/v1/error.proto @@ -0,0 +1,36 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Terminal errors for accepted streaming requests. + +syntax = "proto3"; + +package openengine.v1; + +import "google/protobuf/struct.proto"; + +// Accepted failures emit one terminal EngineError and close with OK. +// Validation and transport failures use non-OK gRPC status instead. +message EngineError { + ErrorCode code = 1; + string message = 2; + bool retryable = 3; // Retry may succeed without changing the request. + optional uint64 retry_after_ms = 4; // Zero permits immediate retry. + google.protobuf.Struct details = 5; // Machine-readable context. +} + +enum ErrorCode { + ERROR_CODE_UNSPECIFIED = 0; + ERROR_CODE_INVALID_ARGUMENT = 1; + ERROR_CODE_UNSUPPORTED_FEATURE = 2; + ERROR_CODE_ROLE_MISMATCH = 3; + ERROR_CODE_MODEL_NOT_FOUND = 4; + ERROR_CODE_OVERLOADED = 5; + ERROR_CODE_REQUEST_NOT_FOUND = 6; + ERROR_CODE_DUPLICATE_REQUEST = 7; + ERROR_CODE_KV_SESSION_NOT_FOUND = 8; + ERROR_CODE_KV_TRANSFER_FAILED = 9; + ERROR_CODE_CANCELLED = 10; + ERROR_CODE_DRAINING = 11; + ERROR_CODE_INTERNAL = 12; +} diff --git a/proto/openengine/v1/generation.proto b/proto/openengine/v1/generation.proto new file mode 100644 index 0000000..3d8b50b --- /dev/null +++ b/proto/openengine/v1/generation.proto @@ -0,0 +1,145 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Generation requests, streamed events, and usage. + +syntax = "proto3"; + +package openengine.v1; + +import "openengine/v1/error.proto"; +import "openengine/v1/generation_params.proto"; +import "openengine/v1/kv.proto"; + +message GenerateRequest { + string request_id = 1; + string model = 2; + + oneof input { + string prompt = 3; + TokenIds token_ids = 4; + } + + SamplingParams sampling = 5; + StoppingOptions stopping = 6; + ResponseOptions response = 7; + KvOptions kv = 8; + GuidedDecoding guided = 9; + + // Multimodal inputs. Order is significant: the i-th item aligns with the + // i-th (un-expanded) placeholder marker carried in the prompt/token_ids. + // The engine fetches/decodes and preprocesses each item, then expands the + // marker into the model's replacement run. Empty for text-only requests. + repeated MediaItem media = 10; + + // Loaded LoRA adapter name to apply to this request. Empty = base model. + // ModelInfo.supports_lora advertises whether lifecycle and selection are + // available through OpenEngine. + string lora_name = 11; + + // Higher values receive higher scheduling priority. + optional int32 priority = 12; + + // Optional request metadata for tracing/admission/routing. + map metadata = 13; +} + +// Multimodal modality discriminator. 0 is treated as image for forward +// compatibility with senders that omit the field. +enum Modality { + MODALITY_UNSPECIFIED = 0; + MODALITY_IMAGE = 1; + MODALITY_VIDEO = 2; + MODALITY_AUDIO = 3; +} + +// A single multimodal input. Exactly one `source` should be set. The engine +// owns fetch, decode, and preprocessing, so pre-decoded or RDMA media +// descriptors are not represented here. +message MediaItem { + Modality modality = 1; + oneof source { + string url = 2; // http(s):// -- engine fetches + string data_uri = 3; // data:;base64,<...> -- engine decodes + bytes raw_bytes = 4; // pre-fetched bytes -- engine still preprocesses + } + string mime_type = 5; // optional, hints raw_bytes decode + string uuid = 6; // optional caller id / mm_hash +} + +message GenerateResponse { + string request_id = 1; + + // PromptOutput, PrefillReady, and EngineError are request-scoped. + // TokenOutput and GenerationFinished are output-scoped. + oneof event { + PromptOutput prompt = 2; + TokenOutput token = 3; + PrefillReady prefill_ready = 4; + GenerationFinished finished = 5; + EngineError error = 6; + } + + Usage usage = 10; // Cumulative request usage; only set on the final response. +} + +// Request-scoped prompt token information, emitted at most once. +message PromptOutput { + repeated TokenInfo tokens = 1; +} + +// An incremental output delta. Tokens and text are never cumulative. +message TokenOutput { + optional uint32 output_index = 1; // Required, including for output zero. + repeated TokenInfo tokens = 2; + string text = 3; +} + +message TokenInfo { + uint32 token_id = 1; + string token = 2; + optional double logprob = 3; + optional uint32 rank = 4; + repeated LogProb candidates = 5; +} + +message LogProb { + uint32 token_id = 1; + double logprob = 2; + string token = 3; + optional uint32 rank = 4; +} + +message PrefillReady { + KvSessionRef kv_session = 1; +} + +message GenerationFinished { + optional uint32 output_index = 1; // Required, including for output zero. + FinishReason reason = 2; + string message = 3; + StopMatch stop_match = 4; +} + +message StopMatch { + oneof match { + uint32 stop_token_id = 1; + string stop_text = 2; + uint32 eos_token_id = 3; + } +} + +enum FinishReason { + FINISH_REASON_UNSPECIFIED = 0; + FINISH_REASON_STOP = 1; + FINISH_REASON_LENGTH = 2; + FINISH_REASON_CANCELLED = 3; +} + +message Usage { + uint32 prompt_tokens = 1; + uint32 completion_tokens = 2; + uint32 total_tokens = 3; + optional uint32 cached_prompt_tokens = 4; + optional uint32 reasoning_tokens = 5; +} diff --git a/proto/openengine/v1/generation_params.proto b/proto/openengine/v1/generation_params.proto new file mode 100644 index 0000000..48143c5 --- /dev/null +++ b/proto/openengine/v1/generation_params.proto @@ -0,0 +1,89 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Portable generation parameters and reusable selection types. + +syntax = "proto3"; + +package openengine.v1; + +import "openengine/v1/kv.proto"; + +message TokenIds { + repeated uint32 ids = 1; +} + +message SamplingParams { + optional double temperature = 1; + optional double top_p = 2; + optional int32 top_k = 3; + optional double min_p = 4; + optional double frequency_penalty = 5; + optional double presence_penalty = 6; + optional double repetition_penalty = 7; + optional uint64 seed = 8; + optional uint32 num_sequences = 9; +} + +message StoppingOptions { + optional uint32 max_tokens = 1; + optional uint32 min_tokens = 2; + repeated StopCondition conditions = 3; + optional bool ignore_eos = 4; + optional bool include_stop_in_output = 5; +} + +message ResponseOptions { + optional bool return_prompt_logprobs = 1; + CandidateTokenSelection prompt_candidates = 2; + optional bool return_output_logprobs = 3; + CandidateTokenSelection output_candidates = 4; + optional uint32 prompt_logprob_start = 5; +} + +message CandidateTokenSelection { + oneof selection { + uint32 top_n = 1; + TokenIds token_ids = 2; + AllCandidates all = 3; + } +} + +message AllCandidates {} + +message KvOptions { + KvSessionRef session = 1; + optional uint32 data_parallel_rank = 2; + optional bool bypass_prefix_cache = 3; + optional string cache_salt = 4; +} + +message StopCondition { + oneof condition { + string stop_text = 1; + uint32 stop_token_id = 2; + } +} + +// Constrained / guided decoding spec. At most one of `guide` should be set. +// The engine enforces the constraint during sampling via its grammar backend +// (xgrammar / outlines / llguidance); clients cannot apply it post-hoc. +message GuidedDecoding { + oneof guide { + string json_schema = 1; // output conforms to this JSON schema + string regex = 2; // output matches this regex + string ebnf_grammar = 3; // output follows this EBNF / context-free grammar + string structural_tag = 4; // xgrammar structural-tag constraint (JSON string) + ChoiceConstraint choice = 5; + JsonObjectConstraint json_object = 6; + } + // Optional grammar backend override (e.g. "xgrammar", "outlines", + // "llguidance"). Empty = engine default. + string backend = 7; +} + +message ChoiceConstraint { + repeated string choices = 1; +} + +message JsonObjectConstraint {} diff --git a/proto/openengine/v1/kv.proto b/proto/openengine/v1/kv.proto new file mode 100644 index 0000000..95a2295 --- /dev/null +++ b/proto/openengine/v1/kv.proto @@ -0,0 +1,138 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// KV sessions, connector discovery, and cache events. + +syntax = "proto3"; + +package openengine.v1; + +import "google/protobuf/struct.proto"; +import "openengine/v1/error.proto"; + +message KvSessionRef { + string session_id = 1; + string transfer_backend = 2; + repeated KvEndpoint endpoints = 3; + uint32 dp_rank = 4; + // Engine-specific KV-transfer parameters (e.g. NixlConnector + // remote_host / remote_port / tp_size / remote_block_ids / do_remote_*), + // carried as a Struct so numbers, booleans, and arrays survive the wire with + // their JSON type intact. Struct numbers are IEEE-754 doubles (exact only to + // 2^53); carry larger integer values as strings or use a dedicated field. + google.protobuf.Struct attributes_struct = 5; +} + +message KvEndpoint { + string host = 1; + uint32 port = 2; + string protocol = 3; // grpc, nixl, ucx, tcp, shm, etc. +} + +message GetKvConnectorInfoRequest {} + +message KvConnectorInfo { + optional bool enabled = 1; + string transfer_backend = 2; + repeated KvEndpoint local_endpoints = 3; + repeated string supported_protocols = 4; + optional bool supports_remote_prefill = 5; + optional bool supports_decode_pull = 6; + optional bool supports_abort_cleanup = 7; + optional bool supports_drain = 8; + optional uint32 schema_version = 9; +} + +message GetKvEventSourcesRequest { + repeated uint32 data_parallel_ranks = 1; +} + +message GetKvEventSourcesResponse { + repeated KvEventSource sources = 1; +} + +message KvEventSource { + string transport = 1; // grpc, zmq + // Connectable address. MUST use a routable host, never a bind wildcard. + KvEndpoint endpoint_addr = 2; + string topic = 3; + string replay_endpoint = 4; // optional, for ZMQ replay + optional uint32 data_parallel_rank = 5; + string encoding = 6; // protobuf, msgpack + optional uint32 schema_version = 7; + optional uint32 buffer_steps = 8; + optional uint32 hwm = 9; + optional uint32 max_queue_size = 10; +} + +message SubscribeKvEventsRequest { + repeated uint32 data_parallel_ranks = 1; + bool include_snapshot = 2; + uint64 start_sequence_number = 3; +} + +message SubscribeKvEventsResponse { + oneof event { + KvEventBatch batch = 1; + EngineError error = 2; // Terminal. + } +} + +message KvEventBatch { + uint64 sequence_number = 1; + uint64 timestamp_unix_nanos = 2; + uint32 data_parallel_rank = 3; + repeated KvEvent events = 4; +} + +message KvEvent { + string request_id = 1; + KvSessionRef kv_session = 2; + + oneof event { + BlockStored block_stored = 10; + BlockRemoved block_removed = 11; + AllBlocksCleared all_blocks_cleared = 12; + } +} + +message BlockStored { + repeated KvBlockHash block_hashes = 1; + KvBlockHash parent_block_hash = 2; + repeated uint32 token_ids = 3; + uint32 block_size = 4; + int64 lora_id = 5; + string lora_name = 6; + StorageMedium medium = 7; + + // vLLM-compatible optional metadata for reconstructing block keys. + repeated OpaqueKeyTuple extra_keys = 20; + uint32 group_idx = 21; + string kv_cache_spec_kind = 22; + uint32 kv_cache_spec_sliding_window = 23; +} + +message BlockRemoved { + repeated KvBlockHash block_hashes = 1; + StorageMedium medium = 2; + uint32 group_idx = 3; +} + +message AllBlocksCleared {} + +message KvBlockHash { + bytes value = 1; + string encoding = 2; // int64, string, bytes, engine_specific +} + +message OpaqueKeyTuple { + repeated string values = 1; +} + +enum StorageMedium { + STORAGE_MEDIUM_UNSPECIFIED = 0; + STORAGE_MEDIUM_GPU = 1; + STORAGE_MEDIUM_CPU_PINNED = 2; + STORAGE_MEDIUM_DISK = 3; + STORAGE_MEDIUM_EXTERNAL = 4; +} diff --git a/proto/openengine/v1/lifecycle.proto b/proto/openengine/v1/lifecycle.proto new file mode 100644 index 0000000..d183875 --- /dev/null +++ b/proto/openengine/v1/lifecycle.proto @@ -0,0 +1,89 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Health, abort, and drain operations. + +syntax = "proto3"; + +package openengine.v1; + +import "openengine/v1/engine.proto"; +import "openengine/v1/error.proto"; +import "openengine/v1/kv.proto"; + +message HealthRequest { + // False means a lightweight readiness/liveness check. True asks the engine to + // run a role-appropriate minimal inference probe and report it as a check. + bool include_inference_probe = 1; + + // Optional. Used when include_inference_probe is true. Empty means engine + // default served model. + string model = 2; + + // Optional expected role for role-specific inference probes. + EngineRole role = 3; +} + +message HealthResponse { + HealthState state = 1; + repeated HealthCheck checks = 2; +} + +enum HealthState { + HEALTH_STATE_UNSPECIFIED = 0; + HEALTH_STATE_STARTING = 1; + HEALTH_STATE_READY = 2; + HEALTH_STATE_DEGRADED = 3; + HEALTH_STATE_DRAINING = 4; + HEALTH_STATE_NOT_READY = 5; +} + +message HealthCheck { + string name = 1; // grpc, scheduler, model, kv_connector, role, inference_probe + HealthState state = 2; + string message = 3; +} + +message AbortRequest { + oneof target { + string request_id = 1; + KvSessionRef kv_session = 2; + AllRequests all_requests = 3; + } +} + +message AllRequests {} + +message AbortResponse { + AbortStatus status = 1; + string message = 2; +} + +enum AbortStatus { + ABORT_STATUS_UNSPECIFIED = 0; + ABORT_STATUS_ABORTED = 1; + ABORT_STATUS_ALREADY_FINISHED = 2; +} + +message DrainRequest { + bool stop_accepting_new_requests = 1; + optional uint32 deadline_ms = 2; // Absent means no deadline; zero is immediate. + bool abort_after_deadline = 3; +} + +message DrainResponse { + oneof event { + DrainState state = 1; // Progress or successful completion. + EngineError error = 5; // Terminal failure. + } + optional uint32 in_flight_requests = 2; + optional uint32 open_kv_sessions = 3; + string message = 4; +} + +enum DrainState { + DRAIN_STATE_UNSPECIFIED = 0; + DRAIN_STATE_STARTED = 1; + DRAIN_STATE_IN_PROGRESS = 2; + DRAIN_STATE_COMPLETE = 3; +} diff --git a/proto/openengine/v1/lora.proto b/proto/openengine/v1/lora.proto new file mode 100644 index 0000000..e0951ad --- /dev/null +++ b/proto/openengine/v1/lora.proto @@ -0,0 +1,37 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// LoRA adapter lifecycle messages. + +syntax = "proto3"; + +package openengine.v1; + +message LoraAdapter { + int64 lora_id = 1; + string lora_name = 2; + string source_path = 3; +} + +message LoadLoraRequest { + LoraAdapter adapter = 1; +} + +message LoadLoraResponse { + LoraAdapter adapter = 1; + bool already_loaded = 2; +} + +message UnloadLoraRequest { + string lora_name = 1; +} + +message UnloadLoraResponse { + LoraAdapter adapter = 1; +} + +message ListLorasRequest {} + +message ListLorasResponse { + repeated LoraAdapter adapters = 1; +} diff --git a/proto/openengine/v1/model.proto b/proto/openengine/v1/model.proto new file mode 100644 index 0000000..828aac6 --- /dev/null +++ b/proto/openengine/v1/model.proto @@ -0,0 +1,76 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Model metadata and portable generation capabilities. + +syntax = "proto3"; + +package openengine.v1; + +message GetModelInfoRequest { + string model = 1; +} + +message ModelInfo { + string model_id = 1; + string served_model_name = 2; + repeated string served_model_aliases = 3; + optional uint32 max_context_length = 4; + optional uint32 max_output_tokens = 5; + optional uint32 kv_block_size = 6; + optional uint64 total_kv_blocks = 7; + optional uint64 max_running_requests = 8; + optional uint64 max_batched_tokens = 9; + repeated string tokenizer_modes = 10; + + optional bool supports_text_input = 20; + optional bool supports_token_ids_input = 21; + GenerationCapabilities generation = 22; + optional bool supports_lora = 23; + optional bool supports_multimodal = 24; + + // Engine-advertised response parser names. Clients can apply these to a + // model's output stream for tool-call extraction or reasoning separation. + // Empty means no parser is configured for this model. + string reasoning_parser = 25; + string tool_call_parser = 26; +} + +message GenerationCapabilities { + LogprobCapabilities prompt_logprobs = 1; + LogprobCapabilities output_logprobs = 2; + GuidedDecodingCapabilities guided_decoding = 3; + optional uint32 max_num_sequences = 4; + optional bool supports_priority = 5; + optional bool supports_stop_in_output = 6; + optional bool supports_cache_salt = 7; + optional bool supports_prefix_cache_bypass = 8; +} + +message LogprobCapabilities { + optional bool supported = 1; + repeated CandidateTokenSelectionMode candidate_selection_modes = 2; + optional uint32 max_top_n = 3; +} + +enum CandidateTokenSelectionMode { + CANDIDATE_TOKEN_SELECTION_MODE_UNSPECIFIED = 0; + CANDIDATE_TOKEN_SELECTION_MODE_TOP_N = 1; + CANDIDATE_TOKEN_SELECTION_MODE_TOKEN_IDS = 2; + CANDIDATE_TOKEN_SELECTION_MODE_ALL = 3; +} + +message GuidedDecodingCapabilities { + optional bool supported = 1; + repeated GuidedDecodingMode modes = 2; +} + +enum GuidedDecodingMode { + GUIDED_DECODING_MODE_UNSPECIFIED = 0; + GUIDED_DECODING_MODE_JSON_SCHEMA = 1; + GUIDED_DECODING_MODE_REGEX = 2; + GUIDED_DECODING_MODE_EBNF_GRAMMAR = 3; + GUIDED_DECODING_MODE_STRUCTURAL_TAG = 4; + GUIDED_DECODING_MODE_CHOICE = 5; + GUIDED_DECODING_MODE_JSON_OBJECT = 6; +} diff --git a/proto/openengine/v1/observability.proto b/proto/openengine/v1/observability.proto new file mode 100644 index 0000000..e4e9690 --- /dev/null +++ b/proto/openengine/v1/observability.proto @@ -0,0 +1,66 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +// Load snapshots and structured runtime events. + +syntax = "proto3"; + +package openengine.v1; + +import "openengine/v1/error.proto"; + +message GetLoadRequest { + bool include_per_rank = 1; +} + +message LoadInfo { + string instance_id = 1; + optional uint64 timestamp_unix_nanos = 2; + optional uint32 running_requests = 3; + optional uint32 queued_requests = 4; + optional uint32 active_kv_sessions = 5; + optional uint64 used_kv_blocks = 6; + optional uint64 total_kv_blocks = 7; + optional uint64 running_tokens = 8; + optional uint64 waiting_tokens = 9; + optional uint32 prefill_batch_size = 10; + optional uint32 decode_batch_size = 11; + repeated RankLoadInfo ranks = 20; + map attributes = 30; +} + +message RankLoadInfo { + optional uint32 data_parallel_rank = 1; + optional uint32 running_requests = 2; + optional uint32 queued_requests = 3; + optional uint64 used_kv_blocks = 4; + optional uint64 total_kv_blocks = 5; + optional uint32 prefill_batch_size = 6; + optional uint32 decode_batch_size = 7; +} + +message SubscribeRuntimeEventsRequest { + repeated RuntimeEventType types = 1; +} + +message SubscribeRuntimeEventsResponse { + oneof event { + RuntimeEvent runtime_event = 1; + EngineError error = 2; // Terminal. + } +} + +enum RuntimeEventType { + RUNTIME_EVENT_TYPE_UNSPECIFIED = 0; + RUNTIME_EVENT_TYPE_FORWARD_PASS = 1; + RUNTIME_EVENT_TYPE_BATCH = 2; + RUNTIME_EVENT_TYPE_QUEUE = 3; + RUNTIME_EVENT_TYPE_TRANSFER = 4; +} + +message RuntimeEvent { + string event_id = 1; + uint64 timestamp_unix_nanos = 2; + RuntimeEventType type = 3; + map attributes = 4; +} diff --git a/proto/openengine/v1/openengine.proto b/proto/openengine/v1/openengine.proto index 3ccb5d0..71f26b8 100644 --- a/proto/openengine/v1/openengine.proto +++ b/proto/openengine/v1/openengine.proto @@ -5,9 +5,9 @@ // vLLM, SGLang, TensorRT-LLM) and clients, including direct applications and // distributed frameworks (e.g. Dynamo). // -// This file is the CANONICAL contract and single source of truth for the wire -// shape. Implementations should generate client and server bindings from this -// proto or from an exact copy pinned to a known OpenEngine revision. +// The files in this package are the canonical contract and single source of +// truth for the wire shape. Implementations should generate client and server +// bindings from the complete openengine/v1 package at a known revision. // // Human-readable reference: // https://github.com/ai-dynamo/openengine/blob/main/docs/api.md @@ -16,7 +16,13 @@ syntax = "proto3"; package openengine.v1; -import "google/protobuf/struct.proto"; +import "openengine/v1/engine.proto"; +import "openengine/v1/generation.proto"; +import "openengine/v1/kv.proto"; +import "openengine/v1/lifecycle.proto"; +import "openengine/v1/lora.proto"; +import "openengine/v1/model.proto"; +import "openengine/v1/observability.proto"; service OpenEngine { // Core inference path. @@ -45,664 +51,3 @@ service OpenEngine { // Structured runtime events for planners/controllers. rpc SubscribeRuntimeEvents(SubscribeRuntimeEventsRequest) returns (stream SubscribeRuntimeEventsResponse); } - -// --------------------------------------------------------------------------- -// Core identity and roles -// --------------------------------------------------------------------------- - -enum EngineRole { - ENGINE_ROLE_UNSPECIFIED = 0; - ENGINE_ROLE_AGGREGATED = 1; - ENGINE_ROLE_PREFILL = 2; - ENGINE_ROLE_DECODE = 3; -} - -message GetEngineInfoRequest {} - -message EngineInfo { - string engine_name = 1; // sglang, vllm, tensorrt_llm, etc. - string engine_version = 2; - EngineRole role = 3; - string instance_id = 4; - repeated string supported_models = 5; - ParallelismInfo parallelism = 6; - KvConnectorInfo kv_connector = 7; - uint32 schema_revision = 8; // Monotonic wire contract revision; zero is invalid. - uint32 minimum_client_revision = 9; // Oldest compatible client revision. - string schema_release = 10; // Immutable release or source tag for this schema. -} - -message ParallelismInfo { - optional uint32 tensor_parallel_size = 1; - optional uint32 pipeline_parallel_size = 2; - optional uint32 data_parallel_size = 3; - optional uint32 data_parallel_rank = 4; - optional uint32 data_parallel_start_rank = 5; -} - -// --------------------------------------------------------------------------- -// Model and capacity metadata -// --------------------------------------------------------------------------- - -message GetModelInfoRequest { - string model = 1; -} - -message ModelInfo { - string model_id = 1; - string served_model_name = 2; - repeated string served_model_aliases = 3; - optional uint32 max_context_length = 4; - optional uint32 max_output_tokens = 5; - optional uint32 kv_block_size = 6; - optional uint64 total_kv_blocks = 7; - optional uint64 max_running_requests = 8; - optional uint64 max_batched_tokens = 9; - repeated string tokenizer_modes = 10; - - optional bool supports_text_input = 20; - optional bool supports_token_ids_input = 21; - GenerationCapabilities generation = 22; - optional bool supports_lora = 23; - optional bool supports_multimodal = 24; - - // Engine-advertised response parser names. Clients can apply these to a - // model's output stream for tool-call extraction or reasoning separation. - // Empty means no parser is configured for this model. - string reasoning_parser = 25; - string tool_call_parser = 26; -} - -message GenerationCapabilities { - LogprobCapabilities prompt_logprobs = 1; - LogprobCapabilities output_logprobs = 2; - GuidedDecodingCapabilities guided_decoding = 3; - optional uint32 max_num_sequences = 4; - optional bool supports_priority = 5; - optional bool supports_stop_in_output = 6; - optional bool supports_cache_salt = 7; - optional bool supports_prefix_cache_bypass = 8; -} - -message LogprobCapabilities { - optional bool supported = 1; - repeated CandidateTokenSelectionMode candidate_selection_modes = 2; - optional uint32 max_top_n = 3; -} - -enum CandidateTokenSelectionMode { - CANDIDATE_TOKEN_SELECTION_MODE_UNSPECIFIED = 0; - CANDIDATE_TOKEN_SELECTION_MODE_TOP_N = 1; - CANDIDATE_TOKEN_SELECTION_MODE_TOKEN_IDS = 2; - CANDIDATE_TOKEN_SELECTION_MODE_ALL = 3; -} - -message GuidedDecodingCapabilities { - optional bool supported = 1; - repeated GuidedDecodingMode modes = 2; -} - -enum GuidedDecodingMode { - GUIDED_DECODING_MODE_UNSPECIFIED = 0; - GUIDED_DECODING_MODE_JSON_SCHEMA = 1; - GUIDED_DECODING_MODE_REGEX = 2; - GUIDED_DECODING_MODE_EBNF_GRAMMAR = 3; - GUIDED_DECODING_MODE_STRUCTURAL_TAG = 4; - GUIDED_DECODING_MODE_CHOICE = 5; - GUIDED_DECODING_MODE_JSON_OBJECT = 6; -} - -// --------------------------------------------------------------------------- -// LoRA lifecycle -// --------------------------------------------------------------------------- - -message LoraAdapter { - int64 lora_id = 1; - string lora_name = 2; - string source_path = 3; -} - -message LoadLoraRequest { - LoraAdapter adapter = 1; -} - -message LoadLoraResponse { - LoraAdapter adapter = 1; - bool already_loaded = 2; -} - -message UnloadLoraRequest { - string lora_name = 1; -} - -message UnloadLoraResponse { - LoraAdapter adapter = 1; -} - -message ListLorasRequest {} - -message ListLorasResponse { - repeated LoraAdapter adapters = 1; -} - -// --------------------------------------------------------------------------- -// Generation API -// --------------------------------------------------------------------------- - -message GenerateRequest { - string request_id = 1; - string model = 2; - - oneof input { - string prompt = 3; - TokenIds token_ids = 4; - } - - SamplingParams sampling = 5; - StoppingOptions stopping = 6; - ResponseOptions response = 7; - KvOptions kv = 8; - GuidedDecoding guided = 9; - - // Multimodal inputs. Order is significant: the i-th item aligns with the - // i-th (un-expanded) placeholder marker carried in the prompt/token_ids. - // The engine fetches/decodes and preprocesses each item, then expands the - // marker into the model's replacement run. Empty for text-only requests. - repeated MediaItem media = 10; - - // Loaded LoRA adapter name to apply to this request. Empty = base model. - // ModelInfo.supports_lora advertises whether lifecycle and selection are - // available through OpenEngine. - string lora_name = 11; - - // Higher values receive higher scheduling priority. - optional int32 priority = 12; - - // Optional request metadata for tracing/admission/routing. - map metadata = 13; -} - -// Multimodal modality discriminator. 0 is treated as image for forward -// compatibility with senders that omit the field. -enum Modality { - MODALITY_UNSPECIFIED = 0; - MODALITY_IMAGE = 1; - MODALITY_VIDEO = 2; - MODALITY_AUDIO = 3; -} - -// A single multimodal input. Exactly one `source` should be set. The engine -// owns fetch, decode, and preprocessing, so pre-decoded or RDMA media -// descriptors are not represented here. -message MediaItem { - Modality modality = 1; - oneof source { - string url = 2; // http(s):// -- engine fetches - string data_uri = 3; // data:;base64,<...> -- engine decodes - bytes raw_bytes = 4; // pre-fetched bytes -- engine still preprocesses - } - string mime_type = 5; // optional, hints raw_bytes decode - string uuid = 6; // optional caller id / mm_hash -} - -message TokenIds { - repeated uint32 ids = 1; -} - -message SamplingParams { - optional double temperature = 1; - optional double top_p = 2; - optional int32 top_k = 3; - optional double min_p = 4; - optional double frequency_penalty = 5; - optional double presence_penalty = 6; - optional double repetition_penalty = 7; - optional uint64 seed = 8; - optional uint32 num_sequences = 9; -} - -message StoppingOptions { - optional uint32 max_tokens = 1; - optional uint32 min_tokens = 2; - repeated StopCondition conditions = 3; - optional bool ignore_eos = 4; - optional bool include_stop_in_output = 5; -} - -message ResponseOptions { - optional bool return_prompt_logprobs = 1; - CandidateTokenSelection prompt_candidates = 2; - optional bool return_output_logprobs = 3; - CandidateTokenSelection output_candidates = 4; - optional uint32 prompt_logprob_start = 5; -} - -message CandidateTokenSelection { - oneof selection { - uint32 top_n = 1; - TokenIds token_ids = 2; - AllCandidates all = 3; - } -} - -message AllCandidates {} - -message KvOptions { - KvSessionRef session = 1; - optional uint32 data_parallel_rank = 2; - optional bool bypass_prefix_cache = 3; - optional string cache_salt = 4; -} - -message StopCondition { - oneof condition { - string stop_text = 1; - uint32 stop_token_id = 2; - } -} - -// Constrained / guided decoding spec. At most one of `guide` should be set. -// The engine enforces the constraint during sampling via its grammar backend -// (xgrammar / outlines / llguidance); clients cannot apply it post-hoc. -message GuidedDecoding { - oneof guide { - string json_schema = 1; // output conforms to this JSON schema - string regex = 2; // output matches this regex - string ebnf_grammar = 3; // output follows this EBNF / context-free grammar - string structural_tag = 4; // xgrammar structural-tag constraint (JSON string) - ChoiceConstraint choice = 5; - JsonObjectConstraint json_object = 6; - } - // Optional grammar backend override (e.g. "xgrammar", "outlines", - // "llguidance"). Empty = engine default. - string backend = 7; -} - -message ChoiceConstraint { - repeated string choices = 1; -} - -message JsonObjectConstraint {} - -message GenerateResponse { - string request_id = 1; - - // PromptOutput, PrefillReady, and EngineError are request-scoped. - // TokenOutput and GenerationFinished are output-scoped. - oneof event { - PromptOutput prompt = 2; - TokenOutput token = 3; - PrefillReady prefill_ready = 4; - GenerationFinished finished = 5; - EngineError error = 6; - } - - Usage usage = 10; // Cumulative request usage; only set on the final response. -} - -// Request-scoped prompt token information, emitted at most once. -message PromptOutput { - repeated TokenInfo tokens = 1; -} - -// An incremental output delta. Tokens and text are never cumulative. -message TokenOutput { - optional uint32 output_index = 1; // Required, including for output zero. - repeated TokenInfo tokens = 2; - string text = 3; -} - -message TokenInfo { - uint32 token_id = 1; - string token = 2; - optional double logprob = 3; - optional uint32 rank = 4; - repeated LogProb candidates = 5; -} - -message LogProb { - uint32 token_id = 1; - double logprob = 2; - string token = 3; - optional uint32 rank = 4; -} - -message PrefillReady { - KvSessionRef kv_session = 1; -} - -message GenerationFinished { - optional uint32 output_index = 1; // Required, including for output zero. - FinishReason reason = 2; - string message = 3; - StopMatch stop_match = 4; -} - -message StopMatch { - oneof match { - uint32 stop_token_id = 1; - string stop_text = 2; - uint32 eos_token_id = 3; - } -} - -enum FinishReason { - FINISH_REASON_UNSPECIFIED = 0; - FINISH_REASON_STOP = 1; - FINISH_REASON_LENGTH = 2; - FINISH_REASON_CANCELLED = 3; -} - -message Usage { - uint32 prompt_tokens = 1; - uint32 completion_tokens = 2; - uint32 total_tokens = 3; - optional uint32 cached_prompt_tokens = 4; - optional uint32 reasoning_tokens = 5; -} - -// --------------------------------------------------------------------------- -// Disaggregated serving and KV API -// --------------------------------------------------------------------------- - -message KvSessionRef { - string session_id = 1; - string transfer_backend = 2; - repeated KvEndpoint endpoints = 3; - uint32 dp_rank = 4; - // Engine-specific KV-transfer parameters (e.g. NixlConnector - // remote_host / remote_port / tp_size / remote_block_ids / do_remote_*), - // carried as a Struct so numbers, booleans, and arrays survive the wire with - // their JSON type intact. Struct numbers are IEEE-754 doubles (exact only to - // 2^53); carry larger integer values as strings or use a dedicated field. - google.protobuf.Struct attributes_struct = 5; -} - -message KvEndpoint { - string host = 1; - uint32 port = 2; - string protocol = 3; // grpc, nixl, ucx, tcp, shm, etc. -} - -// --------------------------------------------------------------------------- -// KV connector and KV events -// --------------------------------------------------------------------------- - -message GetKvConnectorInfoRequest {} - -message KvConnectorInfo { - optional bool enabled = 1; - string transfer_backend = 2; - repeated KvEndpoint local_endpoints = 3; - repeated string supported_protocols = 4; - optional bool supports_remote_prefill = 5; - optional bool supports_decode_pull = 6; - optional bool supports_abort_cleanup = 7; - optional bool supports_drain = 8; - optional uint32 schema_version = 9; -} - -message GetKvEventSourcesRequest { - repeated uint32 data_parallel_ranks = 1; -} - -message GetKvEventSourcesResponse { - repeated KvEventSource sources = 1; -} - -message KvEventSource { - string transport = 1; // grpc, zmq - // Connectable address. MUST use a routable host, never a bind wildcard. - KvEndpoint endpoint_addr = 2; - string topic = 3; - string replay_endpoint = 4; // optional, for ZMQ replay - optional uint32 data_parallel_rank = 5; - string encoding = 6; // protobuf, msgpack - optional uint32 schema_version = 7; - optional uint32 buffer_steps = 8; - optional uint32 hwm = 9; - optional uint32 max_queue_size = 10; -} - -message SubscribeKvEventsRequest { - repeated uint32 data_parallel_ranks = 1; - bool include_snapshot = 2; - uint64 start_sequence_number = 3; -} - -message SubscribeKvEventsResponse { - oneof event { - KvEventBatch batch = 1; - EngineError error = 2; // Terminal. - } -} - -message KvEventBatch { - uint64 sequence_number = 1; - uint64 timestamp_unix_nanos = 2; - uint32 data_parallel_rank = 3; - repeated KvEvent events = 4; -} - -message KvEvent { - string request_id = 1; - KvSessionRef kv_session = 2; - - oneof event { - BlockStored block_stored = 10; - BlockRemoved block_removed = 11; - AllBlocksCleared all_blocks_cleared = 12; - } -} - -message BlockStored { - repeated KvBlockHash block_hashes = 1; - KvBlockHash parent_block_hash = 2; - repeated uint32 token_ids = 3; - uint32 block_size = 4; - int64 lora_id = 5; - string lora_name = 6; - StorageMedium medium = 7; - - // vLLM-compatible optional metadata for reconstructing block keys. - repeated OpaqueKeyTuple extra_keys = 20; - uint32 group_idx = 21; - string kv_cache_spec_kind = 22; - uint32 kv_cache_spec_sliding_window = 23; -} - -message BlockRemoved { - repeated KvBlockHash block_hashes = 1; - StorageMedium medium = 2; - uint32 group_idx = 3; -} - -message AllBlocksCleared {} - -message KvBlockHash { - bytes value = 1; - string encoding = 2; // int64, string, bytes, engine_specific -} - -message OpaqueKeyTuple { - repeated string values = 1; -} - -enum StorageMedium { - STORAGE_MEDIUM_UNSPECIFIED = 0; - STORAGE_MEDIUM_GPU = 1; - STORAGE_MEDIUM_CPU_PINNED = 2; - STORAGE_MEDIUM_DISK = 3; - STORAGE_MEDIUM_EXTERNAL = 4; -} - -// --------------------------------------------------------------------------- -// Health, abort, and drain -// --------------------------------------------------------------------------- - -message HealthRequest { - // False means a lightweight readiness/liveness check. True asks the engine to - // run a role-appropriate minimal inference probe and report it as a check. - bool include_inference_probe = 1; - - // Optional. Used when include_inference_probe is true. Empty means engine - // default served model. - string model = 2; - - // Optional expected role for role-specific inference probes. - EngineRole role = 3; -} - -message HealthResponse { - HealthState state = 1; - repeated HealthCheck checks = 2; -} - -enum HealthState { - HEALTH_STATE_UNSPECIFIED = 0; - HEALTH_STATE_STARTING = 1; - HEALTH_STATE_READY = 2; - HEALTH_STATE_DEGRADED = 3; - HEALTH_STATE_DRAINING = 4; - HEALTH_STATE_NOT_READY = 5; -} - -message HealthCheck { - string name = 1; // grpc, scheduler, model, kv_connector, role, inference_probe - HealthState state = 2; - string message = 3; -} - -message AbortRequest { - oneof target { - string request_id = 1; - KvSessionRef kv_session = 2; - AllRequests all_requests = 3; - } -} - -message AllRequests {} - -message AbortResponse { - AbortStatus status = 1; - string message = 2; -} - -enum AbortStatus { - ABORT_STATUS_UNSPECIFIED = 0; - ABORT_STATUS_ABORTED = 1; - ABORT_STATUS_ALREADY_FINISHED = 2; -} - -message DrainRequest { - bool stop_accepting_new_requests = 1; - optional uint32 deadline_ms = 2; // Absent means no deadline; zero is immediate. - bool abort_after_deadline = 3; -} - -message DrainResponse { - oneof event { - DrainState state = 1; // Progress or successful completion. - EngineError error = 5; // Terminal failure. - } - optional uint32 in_flight_requests = 2; - optional uint32 open_kv_sessions = 3; - string message = 4; -} - -enum DrainState { - DRAIN_STATE_UNSPECIFIED = 0; - DRAIN_STATE_STARTED = 1; - DRAIN_STATE_IN_PROGRESS = 2; - DRAIN_STATE_COMPLETE = 3; -} - -// --------------------------------------------------------------------------- -// Runtime observability -// --------------------------------------------------------------------------- - -message GetLoadRequest { - bool include_per_rank = 1; -} - -message LoadInfo { - string instance_id = 1; - optional uint64 timestamp_unix_nanos = 2; - optional uint32 running_requests = 3; - optional uint32 queued_requests = 4; - optional uint32 active_kv_sessions = 5; - optional uint64 used_kv_blocks = 6; - optional uint64 total_kv_blocks = 7; - optional uint64 running_tokens = 8; - optional uint64 waiting_tokens = 9; - optional uint32 prefill_batch_size = 10; - optional uint32 decode_batch_size = 11; - repeated RankLoadInfo ranks = 20; - map attributes = 30; -} - -message RankLoadInfo { - optional uint32 data_parallel_rank = 1; - optional uint32 running_requests = 2; - optional uint32 queued_requests = 3; - optional uint64 used_kv_blocks = 4; - optional uint64 total_kv_blocks = 5; - optional uint32 prefill_batch_size = 6; - optional uint32 decode_batch_size = 7; -} - -message SubscribeRuntimeEventsRequest { - repeated RuntimeEventType types = 1; -} - -message SubscribeRuntimeEventsResponse { - oneof event { - RuntimeEvent runtime_event = 1; - EngineError error = 2; // Terminal. - } -} - -enum RuntimeEventType { - RUNTIME_EVENT_TYPE_UNSPECIFIED = 0; - RUNTIME_EVENT_TYPE_FORWARD_PASS = 1; - RUNTIME_EVENT_TYPE_BATCH = 2; - RUNTIME_EVENT_TYPE_QUEUE = 3; - RUNTIME_EVENT_TYPE_TRANSFER = 4; -} - -message RuntimeEvent { - string event_id = 1; - uint64 timestamp_unix_nanos = 2; - RuntimeEventType type = 3; - map attributes = 4; -} - -// --------------------------------------------------------------------------- -// Standard errors -// --------------------------------------------------------------------------- - -// Accepted failures emit one terminal EngineError and close with OK. -// Validation and transport failures use non-OK gRPC status instead. -message EngineError { - ErrorCode code = 1; - string message = 2; - bool retryable = 3; // Retry may succeed without changing the request. - optional uint64 retry_after_ms = 4; // Zero permits immediate retry. - google.protobuf.Struct details = 5; // Machine-readable context. -} - -enum ErrorCode { - ERROR_CODE_UNSPECIFIED = 0; - ERROR_CODE_INVALID_ARGUMENT = 1; - ERROR_CODE_UNSUPPORTED_FEATURE = 2; - ERROR_CODE_ROLE_MISMATCH = 3; - ERROR_CODE_MODEL_NOT_FOUND = 4; - ERROR_CODE_OVERLOADED = 5; - ERROR_CODE_REQUEST_NOT_FOUND = 6; - ERROR_CODE_DUPLICATE_REQUEST = 7; - ERROR_CODE_KV_SESSION_NOT_FOUND = 8; - ERROR_CODE_KV_TRANSFER_FAILED = 9; - ERROR_CODE_CANCELLED = 10; - ERROR_CODE_DRAINING = 11; - ERROR_CODE_INTERNAL = 12; -}