Create a Helm Chart and local Kind setup

[albal]

This pull request introduces a full-featured Helm chart for deploying the Edit Mind application and its supporting services on Kubernetes, with special support for local development using Kind. The changes add all necessary Kubernetes manifests (deployments, statefulsets, services, config, secrets, and persistent storage), a comprehensive default configuration, and updated documentation for Kubernetes-based workflows.

The most important changes are:

Helm Chart and Kubernetes Manifests:

• Added a new Helm chart at charts/edit-mind with all required templates for deploying Edit Mind and its dependencies (web, background-jobs, ML, Chroma, Redis, Postgres) as Kubernetes resources, including deployments, statefulsets, services, configmaps, secrets, and persistent volume claims. [1] [2] [3] [4] [5] [6] [7]

Configuration and Customization:

• Introduced a comprehensive values.yaml file for the Helm chart, allowing users to configure images, ports, resource requests, GPU support, secrets, persistence, and environment variables for all components.

Documentation Updates:

• Updated README.md to document the new Kubernetes deployment workflow, including prerequisites, usage of the provided kind-deploy.sh script, and instructions for manual Helm installation and configuration overrides.
• Added installation notes in charts/edit-mind/templates/NOTES.txt to guide users on accessing the application after deploying with Helm.

These changes enable users to deploy Edit Mind on any Kubernetes cluster, with easy local development using Kind and Helm, and provide flexibility for production deployments.

[IliasHad]

First of all, thank you, @albal, for your contribution. I never used Helm or Kind before. What will be the best way to test your setup locally?

[albal]

I use Ubuntu as my daily driver but it should also work from a WSL Instance and on Mac. There are some scripts in the scripts folder that you can try out. For e2e testing you might want to look at this action though I think some of the images may be too big for GHA.

Testing on Kind is meant for local development when you want to target real Kubernetes infrastructure as the end goal. I know not everyone will be running Kubernetes but if they were they could scale out parts of application to multiple node and GPUs.

[IliasHad]

I use Ubuntu as my daily driver but it should also work from a WSL Instance and on Mac. There are some scripts in the scripts folder that you can try out. For e2e testing you might want to look at this action though I think some of the images may be too big for GHA.

Awesome, thank you will try that soon. Yes, the ML image is too big and working on getting it optimized.

Testing on Kind is meant for local development when you want to target real Kubernetes infrastructure as the end goal. I know not everyone will be running Kubernetes but if they were they could scale out parts of application to multiple node and GPUs.

Prefect timing, I was looking to work on making the edit-mind-ml and edit-mind-background-jobs scale out to multiple node and GPUs to process more videos at the same time because some users already have mutliple GPUs and wanna process videos across their GPUs. Thank you so much for your contribution