-
Notifications
You must be signed in to change notification settings - Fork 324
Deploy Featureform #2931
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
kaitlynmichael
wants to merge
1
commit into
main
Choose a base branch
from
DOC-6297
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Deploy Featureform #2931
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| --- | ||
| Title: Featureform | ||
| alwaysopen: false | ||
| categories: | ||
| - docs | ||
| - operate | ||
| - featureform | ||
| description: Deploy and operate Featureform on Kubernetes. | ||
| linkTitle: Featureform | ||
| hideListLinks: true | ||
| weight: 50 | ||
| bannerText: Featureform is currently in preview and subject to change. To request access to the Featureform Docker image, contact your Redis account team. | ||
| bannerChildren: true | ||
| --- | ||
|
|
||
| Featureform runs as a Kubernetes-based platform for feature engineering and online feature serving. Use this section to prepare your environment, choose a deployment method, install Featureform, and verify that the platform is ready for application teams. | ||
|
|
||
| These docs are cloud-neutral in framing. Where Terraform examples are useful, request them from your Redis account team. | ||
|
|
||
| ## Get started | ||
|
|
||
| - [Architecture]({{< relref "/operate/featureform/architecture" >}}) | ||
| - [Prerequisites]({{< relref "/operate/featureform/prerequisites" >}}) | ||
| - [Deployment methods]({{< relref "/operate/featureform/deployment-methods" >}}) | ||
| - [Deploy with Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}) | ||
| - [Request Terraform examples for AWS]({{< relref "/operate/featureform/deploy-with-terraform-aws" >}}) | ||
| - [Verify your installation]({{< relref "/operate/featureform/verify-your-installation" >}}) | ||
|
|
||
| Use [Deploy with Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}) if you already manage your own Kubernetes platform. If you need Terraform examples for AWS environments, request them from your Redis account team. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| --- | ||
| title: Featureform architecture | ||
| description: Understand the main components and external dependencies of a Featureform deployment. | ||
| linkTitle: Architecture | ||
| weight: 10 | ||
| --- | ||
|
|
||
| Featureform is deployed on Kubernetes and integrates with external systems for metadata storage, certificate management, ingress, and offline feature computation. | ||
|
|
||
| ## Main components | ||
|
|
||
| A typical deployment includes the following Featureform services: | ||
|
|
||
| - Metadata server | ||
| - API server | ||
| - Feature server | ||
| - Dashboard | ||
| - Coordinator | ||
|
|
||
| These services run on Kubernetes and are installed through the Featureform Helm chart. | ||
|
|
||
| ## External dependencies | ||
|
|
||
| Featureform depends on several external systems: | ||
|
|
||
| - PostgreSQL for metadata storage | ||
| - Redis for online feature serving | ||
| - an offline data system such as Snowflake, BigQuery, or Databricks or Spark | ||
| - Kubernetes ingress and load balancing for external access | ||
| - TLS certificate management for secure traffic | ||
|
|
||
| ## Kubernetes model | ||
|
|
||
| Featureform is designed to run as an application on an existing Kubernetes cluster. The platform team is responsible for cluster networking, storage, ingress, image pull credentials, and secure access to required external systems. | ||
|
|
||
| The current deployment material includes: | ||
|
|
||
| - a Helm chart for teams that already have Kubernetes infrastructure | ||
| - request-only Terraform examples for teams that need AWS-focused deployment guidance | ||
|
|
||
| ## Deployment choices | ||
|
|
||
| - Use [Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}) when you already have a Kubernetes cluster and supporting infrastructure. | ||
| - If you need Terraform examples for AWS deployments, see [Request Terraform examples for AWS]({{< relref "/operate/featureform/deploy-with-terraform-aws" >}}). | ||
|
|
||
| ## Related information | ||
|
|
||
| - [Prerequisites]({{< relref "/operate/featureform/prerequisites" >}}) | ||
| - [Deployment methods]({{< relref "/operate/featureform/deployment-methods" >}}) | ||
| - [User-facing Featureform docs]({{< relref "/develop/ai/featureform/" >}}) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| --- | ||
| title: Deploy Featureform with Helm | ||
| description: Install Featureform on an existing Kubernetes cluster with Helm. | ||
| linkTitle: Deploy with Helm | ||
| weight: 40 | ||
| --- | ||
|
|
||
| Use this path when you already have a Kubernetes cluster and want to install Featureform directly with the Helm chart. | ||
|
|
||
| ## Before you begin | ||
|
|
||
| Confirm that you already have: | ||
|
|
||
| - a Kubernetes cluster running 1.27 or later | ||
| - PostgreSQL available for Featureform metadata | ||
| - ingress or load-balancer capacity for external access | ||
| - TLS plan and certificate approach | ||
| - image pull credentials for the Featureform registry | ||
|
|
||
| See [Prerequisites]({{< relref "/operate/featureform/prerequisites" >}}) for the full checklist. | ||
|
|
||
| ## Installation flow | ||
|
|
||
| The Helm-based deployment generally follows this sequence: | ||
|
|
||
| 1. Create the target namespace. | ||
| 2. Create any required image pull secrets. | ||
| 3. Install or confirm supporting components such as `cert-manager`. | ||
| 4. Prepare a values file with your hostname, PostgreSQL settings, certificate settings, and ingress configuration. | ||
| 5. Install the Featureform chart. | ||
| 6. Verify pods, certificates, and external access. | ||
|
|
||
| ## Example sequence | ||
|
|
||
| ```bash | ||
| kubectl create namespace featureform | ||
|
|
||
| kubectl create secret docker-registry regcred \ | ||
| --docker-server=https://index.docker.io/v1/ \ | ||
| --docker-username=<registry-username> \ | ||
| --docker-password=<registry-password> \ | ||
| --docker-email=<registry-email> \ | ||
| --namespace=featureform | ||
| ``` | ||
|
|
||
| Install `cert-manager` if your environment uses it for certificate automation, then install the Featureform chart with your values file. | ||
|
|
||
| ## Values to prepare | ||
|
|
||
| At minimum, prepare values for: | ||
|
|
||
| - external hostname | ||
| - certificate mode | ||
| - PostgreSQL connection | ||
| - ingress behavior | ||
| - registry secret usage | ||
|
|
||
| Review the chart values in the source material and align them with your environment standards before installation. | ||
|
|
||
| ## After installation | ||
|
|
||
| Continue with [Verify your installation]({{< relref "/operate/featureform/verify-your-installation" >}}). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| --- | ||
| title: Request Terraform examples for AWS | ||
| description: Learn when to request AWS Terraform examples for Featureform from your Redis account team. | ||
| linkTitle: Request Terraform examples for AWS | ||
| weight: 50 | ||
| --- | ||
|
|
||
| Use this page if you need AWS-focused Terraform examples for a Featureform deployment. | ||
|
|
||
| Terraform examples are not published in the docs. If you need them, contact your Redis account team. | ||
|
|
||
| ## When to request Terraform examples | ||
|
|
||
| Terraform examples are useful when you need guidance for an AWS deployment that includes: | ||
|
|
||
| - EKS cluster infrastructure | ||
| - networking | ||
| - PostgreSQL provisioning or connection settings | ||
| - DNS and certificate-related settings | ||
| - Featureform Helm installation | ||
|
|
||
| ## Before you begin | ||
|
|
||
| Gather the following inputs: | ||
|
|
||
| - AWS region and environment name | ||
| - whether you will create a new PostgreSQL instance or use an existing one | ||
| - whether you will use a raw load balancer URL or create a DNS record | ||
| - Docker registry credentials for Featureform images | ||
| - certificate approach for the environment | ||
|
|
||
| ## What to prepare for the conversation | ||
|
|
||
| Be ready to describe the main environment decisions: | ||
|
|
||
| - `create_rds`: create a PostgreSQL instance or use an existing database | ||
| - `create_dns_record`: use a custom Route53 record or rely on the load balancer URL | ||
| - `enable_argo`: install optional GitOps-related components | ||
|
|
||
| Use the simplest path for first deployments unless your organization already has stricter platform requirements. | ||
|
|
||
| ## What to do now | ||
|
|
||
| 1. Review the [prerequisites]({{< relref "/operate/featureform/prerequisites" >}}) for Kubernetes, PostgreSQL, networking, TLS, and registry access. | ||
| 2. Decide whether you need a self-serve Helm deployment or AWS Terraform examples. | ||
| 3. If you need Terraform examples, contact your Redis account team with your AWS requirements and the environment decisions listed above. | ||
|
|
||
| ## AWS-specific notes | ||
|
|
||
| AWS Terraform examples typically assume: | ||
|
|
||
| - EKS as the Kubernetes platform | ||
| - Route53 for optional DNS | ||
| - RDS for optional PostgreSQL creation | ||
| - AWS networking patterns with public ingress and private worker nodes | ||
|
|
||
| Treat those as the concrete example, not as a requirement that every Featureform deployment use AWS. | ||
|
|
||
| {{< note >}} | ||
| If you already have a Kubernetes platform and PostgreSQL in place, [Deploy with Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}) is usually the simpler path. | ||
| {{< /note >}} | ||
|
|
||
| ## After installation | ||
|
|
||
| Continue with [Verify your installation]({{< relref "/operate/featureform/verify-your-installation" >}}). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| --- | ||
| title: Featureform deployment methods | ||
| description: Choose between the self-serve Helm path and request-only Terraform examples for Featureform. | ||
| linkTitle: Deployment methods | ||
| weight: 30 | ||
| --- | ||
|
|
||
| Choose the deployment method that fits your environment and the amount of infrastructure you want the workflow to manage. | ||
|
|
||
| ## Deploy with Helm | ||
|
|
||
| Use Helm when: | ||
|
|
||
| - you already have a Kubernetes cluster | ||
| - you already manage PostgreSQL, ingress, certificates, and network access | ||
| - you want direct control over chart values and Kubernetes objects | ||
|
|
||
| This option fits teams that already have an established Kubernetes platform. | ||
|
|
||
| See [Deploy with Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}). | ||
|
|
||
| ## Request Terraform examples for AWS | ||
|
|
||
| Request Terraform examples when: | ||
|
|
||
| - you want AWS-focused example infrastructure | ||
| - your target environment is AWS | ||
| - you want example guidance for cluster, networking, database, and application deployment together | ||
|
|
||
| Terraform examples are not published as part of the docs. If you need them, contact your Redis account team. | ||
|
|
||
| See [Request Terraform examples for AWS]({{< relref "/operate/featureform/deploy-with-terraform-aws" >}}). | ||
|
|
||
| ## Summary | ||
|
|
||
| - Choose Helm if your organization already has Kubernetes platform standards. | ||
| - Contact your Redis account team if you need AWS Terraform examples. | ||
| - In both cases, confirm prerequisites before installation and complete the verification steps after deployment. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| --- | ||
| title: Featureform prerequisites | ||
| description: Review the platform, networking, certificate, and registry prerequisites for Featureform. | ||
| linkTitle: Prerequisites | ||
| weight: 20 | ||
| --- | ||
|
|
||
| Before you deploy Featureform, verify that your platform can satisfy the infrastructure, access, and operational requirements in this checklist. | ||
|
|
||
| ## Kubernetes | ||
|
|
||
| You need: | ||
|
|
||
| - a Kubernetes cluster running version 1.27 or later | ||
| - sufficient worker-node capacity for the Featureform services | ||
| - `kubectl` access for the team performing the installation | ||
|
|
||
| Supported environments can include managed Kubernetes offerings or self-managed clusters, as long as the cluster can support ingress, storage, and the required networking. | ||
|
|
||
| ## Networking | ||
|
|
||
| Your deployment should provide: | ||
|
|
||
| - ingress and load balancing for external HTTPS and gRPC access | ||
| - private connectivity from Featureform to PostgreSQL and any offline data systems | ||
| - outbound access for image pulls and any required external services | ||
| - HTTP/2-capable load balancing where gRPC communication depends on it | ||
|
|
||
| If you are deploying on AWS, the example deployment uses public subnets for ingress and private subnets for worker nodes. | ||
|
|
||
| ## PostgreSQL | ||
|
|
||
| Featureform requires PostgreSQL for metadata storage. Before installation, decide whether you will: | ||
|
|
||
| - provision a new PostgreSQL instance as part of your infrastructure workflow | ||
| - connect Featureform to an existing PostgreSQL deployment | ||
|
|
||
| Record the host, port, database name, and credentials before installation. | ||
|
|
||
| ## TLS | ||
|
|
||
| Decide which certificate approach you will use: | ||
|
|
||
| - public certificates, such as Let's Encrypt, for production-facing environments | ||
| - self-signed certificates for testing and isolated environments | ||
| - customer-managed certificates if your organization has an existing PKI process | ||
|
|
||
| ## Registry credentials | ||
|
|
||
| Featureform images require registry access. Make sure the Kubernetes namespace that will run Featureform has valid image pull credentials before installation. | ||
|
|
||
| {{< note >}} | ||
| If your pods cannot pull images, verify the registry secret before troubleshooting the application itself. | ||
| {{< /note >}} | ||
|
|
||
| ## Required tools | ||
|
|
||
| Install the following tools based on your deployment method: | ||
|
|
||
| - `kubectl` | ||
| - `helm` version 3 or later | ||
| - `terraform` if you are using the Terraform path | ||
| - your cloud provider CLI if your cluster access depends on it | ||
|
|
||
| ## Optional platform dependencies | ||
|
|
||
| Some deployments also add: | ||
|
|
||
| - `cert-manager` for certificate automation | ||
| - Prometheus or another metrics system for monitoring | ||
| - ArgoCD or other GitOps tooling if your organization standardizes on declarative delivery | ||
|
|
||
| ## What to read next | ||
|
|
||
| - [Deployment methods]({{< relref "/operate/featureform/deployment-methods" >}}) | ||
| - [Deploy with Helm]({{< relref "/operate/featureform/deploy-with-helm" >}}) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| --- | ||
| title: Verify your Featureform installation | ||
| description: Validate that Featureform is reachable and its core components are healthy after deployment. | ||
| linkTitle: Verify your installation | ||
| weight: 60 | ||
| --- | ||
|
|
||
| After installation, verify that Featureform is running correctly before you hand it to application teams. | ||
|
|
||
| ## Check application access | ||
|
|
||
| Retrieve the application URL from your deployment workflow, then confirm that you can reach the dashboard or external endpoint. | ||
|
|
||
| For Helm-based deployments, retrieve it from your ingress or load balancer objects. If you are working from Terraform examples provided by Redis, use the outputs generated by your deployment workflow. | ||
|
|
||
| ## Check pod health | ||
|
|
||
| All Featureform pods should reach a healthy running state: | ||
|
|
||
| ```bash | ||
| kubectl get pods -n featureform | ||
| ``` | ||
|
|
||
| Investigate any pod that remains pending, crash-looping, or otherwise unhealthy. | ||
|
|
||
| ## Check certificate status | ||
|
|
||
| If your environment uses Kubernetes-managed certificates, confirm they are ready: | ||
|
|
||
| ```bash | ||
| kubectl get certificate -n featureform | ||
| ``` | ||
|
|
||
| ## Validate registry and image pulls | ||
|
|
||
| If pods fail to start, check image pull events and confirm that the namespace has valid registry credentials. | ||
|
|
||
| ## Validate PostgreSQL connectivity | ||
|
|
||
| Featureform depends on PostgreSQL for metadata. If the application starts but registration or UI operations fail, check database connectivity, credentials, and network access. | ||
|
|
||
| ## Run a user-level smoke test | ||
|
|
||
| After the platform is healthy, validate the user workflow: | ||
|
|
||
| 1. connect a client with `ff.Client(...)` | ||
| 2. register a simple provider and dataset | ||
| 3. run `client.apply()` | ||
| 4. materialize a small feature view to Redis | ||
| 5. serve a feature value | ||
|
|
||
| Use the [Featureform quickstart]({{< relref "/develop/ai/featureform/quickstart" >}}) as the smoke test path. | ||
|
|
||
| ## Troubleshooting focus areas | ||
|
|
||
| If verification fails, start with: | ||
|
|
||
| - namespace events and pod logs | ||
| - ingress and DNS configuration | ||
| - certificate readiness | ||
| - PostgreSQL connectivity | ||
| - registry credentials | ||
| - Redis connectivity for online serving workflows |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Broken relref links to nonexistent featureform pages
High Severity
The
relreflinks to"/develop/ai/featureform/"and"/develop/ai/featureform/quickstart"reference pages that don't exist in the repository. There is nofeatureformdirectory undercontent/develop/ai/. Hugo'srelrefshortcode triggers a build error when the target page is missing, which will prevent the entire site from building.Additional Locations (1)
content/operate/featureform/verify-your-installation.md#L51-L52