Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions Popular_Models_Guide/HSTU/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
<!--
# Copyright 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The copyright notice here is missing the (c) symbol and covers only 2026, whereas the root README.md added in the same PR uses Copyright (c) 2023-2026. Other files in the repo consistently include (c). Consider aligning the format.

Suggested change
# Copyright 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct it's a legal requirement.

#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-->

# HSTU Generative Recommenders on Triton

[Hierarchical Sequential Transduction Units (HSTU)](https://arxiv.org/abs/2402.17152)
power **Generative Recommenders (GRs)**: recommendation workloads reformulated as
generative modeling over high-cardinality, non-stationary event streams. HSTU
supports both retrieval and ranking style tasks.

Triton Inference Server can serve HSTU models through the
[PyTorch backend](https://github.com/triton-inference-server/pytorch_backend)
using ahead-of-time (AOT) Inductor packages (`platform: "torch_aoti"`). Training,
export, KV-cache runtime, and end-to-end examples live in NVIDIA's
[recsys-examples](https://github.com/NVIDIA/recsys-examples) repository rather
than in this tutorials tree.

## Where to go next

| Resource | Description |
| -------- | ----------- |
| [HSTU overview](https://github.com/NVIDIA/recsys-examples/blob/main/examples/hstu/README.md) | Architecture, training, and inference entry points |
| [HSTU inference](https://github.com/NVIDIA/recsys-examples/blob/main/examples/hstu/inference/README.md) | Inference features, KV-cache, AOTInductor export, and KuaiRand examples |
| [PyTorch AOTI on Triton](https://github.com/triton-inference-server/pytorch_backend#aot-inductor-support-beta) | `torch_aoti` model repository layout and configuration |

> [!NOTE]
> Use the recsys-examples guides for building, exporting, and validating HSTU
> models. This page only points Triton users at that workflow and the Torch AOTI
> serving path.
32 changes: 32 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,30 @@
<!--
# Copyright (c) 2023-2026, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-->
# Triton Tutorials

For users experiencing the "Tensor in" & "Tensor out" approach to Deep Learning Inference, getting started with Triton can lead to many questions. The goal of this repository is to familiarize users with Triton's features and provide guides and examples to ease migration. For a feature by feature explanation, refer to the [Triton Inference Server documentation](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html).
Expand All @@ -23,6 +50,11 @@ The table below contains some popular models that are supported in our tutorials
[Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) |[HuggingFace Transformers Tutorial](https://github.com/triton-inference-server/tutorials/tree/main/Quick_Deploy/HuggingFaceTransformers) |
[LLaVA-v1.5-7B](https://huggingface.co/llava-hf/llava-1.5-7b-hf) | [TensorRT-LLM Tutorial](Popular_Models_Guide/Llava1.5/llava_trtllm_guide.md)

## Generative Recommenders
| Example Models | Tutorial Link |
| :-------------: | :------------------------------: |
| [HSTU](https://github.com/NVIDIA/recsys-examples/tree/main/examples/hstu) | [HSTU on Triton (Torch AOTI)](Popular_Models_Guide/HSTU/README.md) |

**Note:**
This is not an exhausitive list of what Triton supports, just what is included in the tutorials.

Expand Down
Loading