LocalAI – One Command Setup for llama.cpp and llama-swap

[hossbit]

Hi everyone,

I created a small open-source project that simplifies deploying llama.cpp and llama-swap on Ubuntu with Vulkan support, automatic model discovery, systemd integration, and OpenAI-compatible APIs.

The goal is to make it easier for new users to get a local LLM server running quickly without manually configuring all the components.

Features:

• One-command installation
• Vulkan GPU acceleration
• Automatic GGUF model discovery
• llama-swap integration
• OpenAI-compatible API
• Systemd service management
• Compatible with Continue, Open WebUI, LibreChat, AnythingLLM, and OpenAI SDKs

Project:

https://github.com/hossbit/localai

Feedback, suggestions, and contributions are welcome. If you find the project useful, please consider giving it a ⭐ on GitHub to help others discover it.

Thanks!