From 872f11a9df1a72f51611ba82e98dd3f9e4e1f47e Mon Sep 17 00:00:00 2001 From: Yingge He Date: Tue, 21 Jul 2026 21:06:40 -0700 Subject: [PATCH] docs: add HSTU (Generative Recommenders) tutorial page Add Popular_Models_Guide/HSTU/README.md introducing HSTU/GRs and pointing to the recsys-examples model/training/export/inference guides and the PyTorch backend AOT Inductor (torch_aoti) docs. Add a Generative Recommenders row to the top-level tutorials table. TRI-1616 Signed-off-by: Yingge He --- Popular_Models_Guide/HSTU/README.md | 54 +++++++++++++++++++++++++++++ README.md | 32 +++++++++++++++++ 2 files changed, 86 insertions(+) create mode 100644 Popular_Models_Guide/HSTU/README.md diff --git a/Popular_Models_Guide/HSTU/README.md b/Popular_Models_Guide/HSTU/README.md new file mode 100644 index 00000000..8862620c --- /dev/null +++ b/Popular_Models_Guide/HSTU/README.md @@ -0,0 +1,54 @@ + + +# HSTU Generative Recommenders on Triton + +[Hierarchical Sequential Transduction Units (HSTU)](https://arxiv.org/abs/2402.17152) +power **Generative Recommenders (GRs)**: recommendation workloads reformulated as +generative modeling over high-cardinality, non-stationary event streams. HSTU +supports both retrieval and ranking style tasks. + +Triton Inference Server can serve HSTU models through the +[PyTorch backend](https://github.com/triton-inference-server/pytorch_backend) +using ahead-of-time (AOT) Inductor packages (`platform: "torch_aoti"`). Training, +export, KV-cache runtime, and end-to-end examples live in NVIDIA's +[recsys-examples](https://github.com/NVIDIA/recsys-examples) repository rather +than in this tutorials tree. + +## Where to go next + +| Resource | Description | +| -------- | ----------- | +| [HSTU overview](https://github.com/NVIDIA/recsys-examples/blob/main/examples/hstu/README.md) | Architecture, training, and inference entry points | +| [HSTU inference](https://github.com/NVIDIA/recsys-examples/blob/main/examples/hstu/inference/README.md) | Inference features, KV-cache, AOTInductor export, and KuaiRand examples | +| [PyTorch AOTI on Triton](https://github.com/triton-inference-server/pytorch_backend#aot-inductor-support-beta) | `torch_aoti` model repository layout and configuration | + +> [!NOTE] +> Use the recsys-examples guides for building, exporting, and validating HSTU +> models. This page only points Triton users at that workflow and the Torch AOTI +> serving path. diff --git a/README.md b/README.md index 6cad9fdd..9dbf7fde 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,30 @@ + # Triton Tutorials For users experiencing the "Tensor in" & "Tensor out" approach to Deep Learning Inference, getting started with Triton can lead to many questions. The goal of this repository is to familiarize users with Triton's features and provide guides and examples to ease migration. For a feature by feature explanation, refer to the [Triton Inference Server documentation](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/index.html). @@ -23,6 +50,11 @@ The table below contains some popular models that are supported in our tutorials [Falcon-7B](https://huggingface.co/tiiuae/falcon-7b) |[HuggingFace Transformers Tutorial](https://github.com/triton-inference-server/tutorials/tree/main/Quick_Deploy/HuggingFaceTransformers) | [LLaVA-v1.5-7B](https://huggingface.co/llava-hf/llava-1.5-7b-hf) | [TensorRT-LLM Tutorial](Popular_Models_Guide/Llava1.5/llava_trtllm_guide.md) +## Generative Recommenders +| Example Models | Tutorial Link | +| :-------------: | :------------------------------: | +| [HSTU](https://github.com/NVIDIA/recsys-examples/tree/main/examples/hstu) | [HSTU on Triton (Torch AOTI)](Popular_Models_Guide/HSTU/README.md) | + **Note:** This is not an exhausitive list of what Triton supports, just what is included in the tutorials.