Description
LocalAI is an open-source, MIT-licensed runtime whose site describes its goal as making AI run on every machine. It packages a single binary that exposes an API compatible with OpenAI, Anthropic, Ollama and ElevenLabs, so most existing clients only need a URL change to point at models running on a user's own hardware instead of a cloud provider. The project has passed 48,900 GitHub stars and is currently at version 4.8.0.
Read More
Underneath that API, LocalAI is a thin core that pulls in a separate inference backend for whichever engine a model needs, among them llama.cpp, vLLM, SGLang and MLX, switched with one line in a model's configuration. A model gallery lists more than 1,255 models that install in one click. Every feature is built to run on CPU first and is tested that way in continuous integration, with GPU acceleration available across CUDA, ROCm, SYCL, Metal and Vulkan on x86_64 and ARM64 hardware.
LocalAI organizes its capabilities into six areas: Reason, for language models, tool calling and structured output; Listen, for real-time transcription and speaker diarization; Speak, for speech synthesis and voice cloning; See, for vision, detection, recognition and depth estimation; Create, for image, video, music and sound generation; and Act, for agents, MCP connections, skills and retrieval-augmented generation.
A realtime mode streams speech in over WebRTC, runs tool calls mid-conversation, and streams speech back out, aiming to feel like a live conversation rather than a chat window. A separate set of features it calls Senses gives a running system live transcription with speaker labels, detection of roughly 527 kinds of sound event, and voice or face recognition, all designed to run locally and mostly on CPU.
For larger deployments, LocalAI adds distributed routing, VRAM-aware placement of models, prefix-cache affinity and failover across machines using NATS-based federation, so a cluster can grow by adding hardware rather than reconfiguring software. It is built by Ettore Di Giacinto and has been highlighted publicly by figures in the open-source AI community, including llama.cpp author Georgi Gerganov. It targets developers and businesses who want to run AI workloads on their own infrastructure, from a single laptop up to a room of GPUs, while keeping their data on hardware they control.
- CategoryMachine Learning Platforms
- Pricing ModelOpen source
- DeploymentSelf-hosted, Cloud
- PlatformsLinux, Windows, macOS
LocalAI's Core Features
Single binary exposing OpenAI-, Anthropic-, Ollama- and ElevenLabs-compatible APIs
Swappable inference backends: llama.cpp, vLLM, SGLang and MLX
CPU-first design tested in CI, with optional CUDA, ROCm, SYCL, Metal and Vulkan acceleration
Model gallery with 1,255+ one-click installable models
Realtime voice mode over WebRTC with mid-conversation tool calls
Senses features: live transcription, sound-event detection and voice/face recognition
Text, vision, image, video, music and sound generation in one runtime
Agent support via MCP, skills and retrieval-augmented generation
Distributed clustering with VRAM-aware routing and failover across machines
How to use LocalAI?
Install LocalAI: Download the single binary or container for your operating system and architecture.
Pick a model: Choose a model from the built-in gallery of 1,255+ models to install in one click.
Point your client at it: Repoint an existing OpenAI, Anthropic, Ollama or ElevenLabs client at LocalAI's API instead of the cloud.
Choose an engine: Set which backend, llama.cpp, vLLM, SGLang or MLX, a model should run on in its config.
Add capabilities: Turn on voice, vision, image or agent features as needed.
Scale out: Add more machines and let LocalAI's routing and failover handle distribution.
LocalAI's Use Cases
- Self-hosted chat API
- Local voice assistant
- Multi-modal local runtime
- Multi-machine AI cluster
