Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions ci/markdown-link-check-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@
},
{
"pattern": "^https://docs\\.cursor\\.com"
},
{
"pattern": "^https://docs\\.docker\\.com"
}
]
}
43 changes: 43 additions & 0 deletions docs/source/build-workflows/llms/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ NVIDIA NeMo Agent Toolkit supports the following LLM providers:
| [OpenAI](https://openai.com) | `openai` | OpenAI API |
| [AWS Bedrock](https://aws.amazon.com/bedrock/) | `aws_bedrock` | AWS Bedrock API |
| [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/quickstart) | `azure_openai` | Azure OpenAI API |
| [OCI Generative AI](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm) | `oci` | OCI Generative AI |
| [LiteLLM](https://github.com/BerriAI/litellm) | `litellm` | LiteLLM API |
| [Hugging Face](https://huggingface.co) | `huggingface` | Hugging Face API |
| [Hugging Face Inference](https://huggingface.co/docs/api-inference) | `huggingface_inference` | Hugging Face Inference API, Endpoints, and TGI |
Expand All @@ -52,6 +53,15 @@ llms:
azure_openai_llm:
_type: azure_openai
azure_deployment: gpt-4o-mini
oci_llm:
_type: oci
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
region: us-chicago-1
compartment_id: ocid1.compartment.oc1..example
auth_type: API_KEY
auth_profile: DEFAULT
auth_file_location: ~/.oci/config
provider: meta
litellm_llm:
_type: litellm
model_name: gpt-4o
Expand Down Expand Up @@ -118,6 +128,39 @@ The AWS Bedrock LLM provider is defined by the {py:class}`~nat.llm.aws_bedrock_l
* `credentials_profile_name` - The credentials profile name to use for the model
* `max_retries` - The maximum number of retries for the request

### OCI Generative AI

You can use the following fields to configure the OCI Generative AI LLM provider:

* `region` - OCI region for the Generative AI service (defaults to `us-chicago-1`). The service endpoint is derived automatically.
* `endpoint` - Optional explicit endpoint URL. Overrides the region-derived endpoint when set.
* `compartment_id` - The OCI compartment OCID used for inference requests
* `auth_type` - OCI SDK auth mode such as `API_KEY`, `SECURITY_TOKEN`, `INSTANCE_PRINCIPAL`, or `RESOURCE_PRINCIPAL`
* `auth_profile` - OCI config profile name for file-backed auth
* `auth_file_location` - Path to the OCI config file
* `provider` - Optional provider override such as `meta`, `google`, `cohere`, or `openai`

The OCI Generative AI LLM provider is defined by the {py:class}`~nat.llm.oci_llm.OCIModelConfig` class.

* `model_name` - The name of the model to use
* `region` - OCI region (defaults to `us-chicago-1`). The endpoint is derived from `https://inference.generativeai.{region}.oci.oraclecloud.com`.
* `endpoint` - Optional explicit endpoint URL. Overrides the region-derived endpoint.
* `compartment_id` - OCI compartment OCID
* `auth_type` - OCI SDK auth type
* `auth_profile` - OCI profile name for file-backed auth
* `auth_file_location` - Path to the OCI config file
* `provider` - Optional OCI provider override such as `meta`, `google`, `cohere`, or `openai`
* `temperature` - The temperature to use for the model
* `top_p` - The top-p value to use for the model
* `max_tokens` - The maximum number of tokens to generate
* `seed` - The seed to use for the model
* `max_retries` - The maximum number of retries for the request
* `request_timeout` - HTTP request timeout in seconds

:::{note}
This provider targets OCI Generative AI through the OCI SDK-backed `langchain-oci` path and does not enable the Responses API.
:::

### Azure OpenAI

You can use the following environment variables to configure the Azure OpenAI LLM provider:
Expand Down
3 changes: 2 additions & 1 deletion docs/source/components/integrations/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,5 @@ limitations under the License.
./frameworks.md
./a2a.md
AWS Bedrock <./integrating-aws-bedrock-models.md>
```
OCI Generative AI <./integrating-oci-generative-ai-models.md>
```
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
<!--
SPDX-FileCopyrightText: Copyright (c) 2025-2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
SPDX-License-Identifier: Apache-2.0

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

# NVIDIA NeMo Agent Toolkit OCI Integration

The NeMo Agent Toolkit supports integration with multiple [LLM](../../build-workflows/llms/index.md) providers, including OCI Generative AI. The `oci` provider uses OCI SDK authentication and is designed for OCI Generative AI model and endpoint access. For workflow parity with the AWS Bedrock path, the toolkit also includes a LangChain wrapper built on `langchain-oci`.

To view the full list of supported LLM providers, run `nat info components -t llm_provider`.

## Configuration

### Prerequisites
Before integrating OCI, ensure you have:

- access to OCI Generative AI in the target region
- a valid OCI auth method such as `API_KEY`, `SECURITY_TOKEN`, `INSTANCE_PRINCIPAL`, or `RESOURCE_PRINCIPAL`
- the target compartment OCID
- the target OCI region (defaults to `us-chicago-1`) or a custom endpoint URL

Common deployment patterns include:

- OCI Generative AI regional endpoints
- custom OCI Generative AI endpoints
- OCI-hosted inference for NVIDIA Nemotron used as a live integration target

### Example Configuration
Add the OCI LLM configuration to your workflow config file:

```yaml
llms:
oci_llm:
_type: oci
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
region: us-chicago-1
compartment_id: ocid1.compartment.oc1..example
auth_type: API_KEY
auth_profile: DEFAULT
temperature: 0.0
Comment thread
coderabbitai[bot] marked this conversation as resolved.
max_tokens: 1024
top_p: 1.0
request_timeout: 60
```

### Configurable Options
* `model_name`: The name of the OCI-hosted model to use (required)
* `region`: OCI region for the Generative AI service (defaults to `us-chicago-1`). The service endpoint is derived automatically.
* `endpoint`: Optional explicit service endpoint URL. Overrides the region-derived endpoint when set.
* `compartment_id`: OCI compartment OCID
* `auth_type`: OCI SDK auth type
* `auth_profile`: OCI profile name for file-backed auth
* `auth_file_location`: Path to the OCI config file
* `provider`: Optional OCI provider override such as `meta`, `google`, `cohere`, or `openai`
* `temperature`: Controls randomness in the output (0.0 to 1.0)
* `max_tokens`: Maximum number of tokens to generate
* `top_p`: Top-p sampling parameter (0.0 to 1.0)
* `seed`: Optional random seed
* `max_retries`: Maximum number of retries for the request
* `request_timeout`: HTTP request timeout in seconds

### Limitations
* This provider targets OCI Generative AI through the OCI SDK-backed `langchain-oci` path.
* The Responses API is not enabled for this provider in the current release.

## Nemotron On OCI

One strong OCI deployment pattern is NVIDIA Nemotron hosted on OCI and exposed through an OpenAI-compatible route. In that setup, the toolkit can validate live integration behavior against the OCI-hosted Nemotron endpoint while the official provider and LangChain wrapper cover the OCI Generative AI path.

## Usage
Reference the OCI LLM in your configuration:

```yaml
llms:
oci_llm:
_type: oci
model_name: nvidia/Llama-3.1-Nemotron-Nano-8B-v1
region: us-chicago-1
compartment_id: ocid1.compartment.oc1..example
auth_profile: DEFAULT
```
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Troubleshooting
* `401 Unauthorized`: verify the OCI profile, signer, and IAM permissions for Generative AI.
* `404 Not Found`: confirm the regional endpoint or custom endpoint URL is correct.
* `Connection errors`: verify OCI networking and that the regional endpoint is reachable.
* `Tool calling issues`: verify the served model supports tool calling and that the serving stack is configured for it.
1 change: 1 addition & 0 deletions docs/source/get-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ The following [LLM](../build-workflows/llms/index.md) API providers are supporte
- OpenAI
- AWS Bedrock
- Azure OpenAI
- OCI Generative AI

## Packages

Expand Down
77 changes: 77 additions & 0 deletions examples/A2A/math_assistant_a2a/uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading