Best MLOps & Model Deployment AI GitHub Repos

Ollama

AI GitHub Repos

#10

Open-source tool for running large language models on your own computer. It lets developers install models like Llama, Gemma, Qwen and DeepSeek with a single command,…Ollama is an open-source tool for running large language models on your own computer. It lets developers install models like Llama, Gemma, Qwen and DeepSeek with a single command,…

Langfuse

AI GitHub Repos

#19

Open-source AI engineering platform offering LLM evaluations, observability, metrics, prompt management, and more. It integrates with OpenTelemetry, LangChain, and…Langfuse is an open-source AI engineering platform offering LLM evaluations, observability, metrics, prompt management, and more. It integrates with OpenTelemetry, LangChain, and…

vLLM

AI GitHub Repos

#23

High-throughput and memory-efficient inference and serving engine designed for large language models (LLMs). It optimizes performance while minimizing…vllm-project/vllm is a high-throughput and memory-efficient inference and serving engine designed for large language models (LLMs). It optimizes performance while minimizing…

Unsloth

AI GitHub Repos

#33

Local user interface designed to run and train large language models (LLMs) and diffusion models. It supports various models including Qwen3.8, Kimi K3,…unslothai/unsloth is a local user interface designed to run and train large language models (LLMs) and diffusion models. It supports various models including Qwen3.8, Kimi K3,…

SGLang

AI GitHub Repos

#53

High-performance serving framework designed for large language models and multimodal models. It provides efficient serving capabilities that enhance the performance of…SGLang is a high-performance serving framework designed for large language models and multimodal models. It provides efficient serving capabilities that enhance the performance of…

Netron Visualizer

AI GitHub Repos

#69

Visualizer for neural network, deep learning, and machine learning models. It provides an intuitive interface to explore model architectures and parameters, making it…Netron is a visualizer for neural network, deep learning, and machine learning models. It provides an intuitive interface to explore model architectures and parameters, making it…

LMDeploy

AI GitHub Repos

#73

Toolkit designed for compressing, deploying, and serving large language models (LLMs). It provides developers with the necessary tools to efficiently manage LLMs,…LMDeploy is a toolkit designed for compressing, deploying, and serving large language models (LLMs). It provides developers with the necessary tools to efficiently manage LLMs,…

DeepEval

AI GitHub Repos

#80

Open-source framework designed for evaluating large language models (LLMs). It allows developers to contribute and improve the evaluation process, enhancing the…DeepEval is an open-source framework designed for evaluating large language models (LLMs). It allows developers to contribute and improve the evaluation process, enhancing the…

llmware

AI GitHub Repos

#81

Unified framework designed for building enterprise RAG pipelines using small, specialized models. It simplifies the integration of various AI models into cohesive…llmware is a unified framework designed for building enterprise RAG pipelines using small, specialized models. It simplifies the integration of various AI models into cohesive…

Phoenix

AI GitHub Repos

#85

Arize Phoenix is an AI observability and evaluation tool designed to help developers monitor and assess AI models. It provides insights into model performance, enabling better…

Feast Feature Store

AI GitHub Repos

#90

Open-source feature store designed for AI and machine learning applications. It enables data scientists to manage and serve features to models in production, enhancing…Feast is an open-source feature store designed for AI and machine learning applications. It enables data scientists to manage and serve features to models in production, enhancing…

BentoML

AI GitHub Repos

#93

Open-source platform designed to simplify the deployment of AI models and applications. It enables developers to build model inference APIs, manage job queues, and…BentoML is an open-source platform designed to simplify the deployment of AI models and applications. It enables developers to build model inference APIs, manage job queues, and…

NannyML

AI GitHub Repos

#106

Python library focused on post-deployment data science. It helps data scientists monitor and analyze model performance after deployment, ensuring reliability and…NannyML is a Python library focused on post-deployment data science. It helps data scientists monitor and analyze model performance after deployment, ensuring reliability and…

llamafile

AI GitHub Repos

#136

Tool designed to simplify the distribution and execution of large language models (LLMs) using a single file. It facilitates collaboration and development within…Llamafile is a tool designed to simplify the distribution and execution of large language models (LLMs) using a single file. It facilitates collaboration and development within…

SGLang-Omni

AI GitHub Repos

#155This week

High-performance serving framework designed for audio models like TTS and ASR, as well as unified multimodal models. It provides robust support for deploying and…SGLang-Omni is a high-performance serving framework designed for audio models like TTS and ASR, as well as unified multimodal models. It provides robust support for deploying and…

Caveman

AI GitHub Repos

#180

GitHub project that optimizes token usage for coding agents by mimicking caveman speech. It reduces token consumption by 65%, enhancing efficiency for developers…Caveman is a GitHub project that optimizes token usage for coding agents by mimicking caveman speech. It reduces token consumption by 65%, enhancing efficiency for developers…

FailproofAI

AI GitHub Repos

#181

Provides observability and enforcement for AI agent harnesses, ensuring every run is captured and runtime reliability is maintained with policy enforcement. It is…FailproofAI provides observability and enforcement for AI agent harnesses, ensuring every run is captured and runtime reliability is maintained with policy enforcement. It is…

RagaAI Catalyst

AI GitHub Repos

#200

Python SDK designed for AI observability, monitoring, and evaluation. It offers features like agent and tool tracing, debugging multi-agent systems, and…RagaAI Catalyst is a Python SDK designed for AI observability, monitoring, and evaluation. It offers features like agent and tool tracing, debugging multi-agent systems, and…

PowerInfer

AI GitHub Repos

#201

High-speed large language model serving tool designed for local deployment. It enables efficient and rapid inference of large language models, making it suitable…PowerInfer is a high-speed large language model serving tool designed for local deployment. It enables efficient and rapid inference of large language models, making it suitable…

AirLLM

AI GitHub Repos

#236

GitHub project that enables 70 billion parameter inference using a single 4GB GPU. It is designed to optimize large language model performance on limited hardware,…AirLLM is a GitHub project that enables 70 billion parameter inference using a single 4GB GPU. It is designed to optimize large language model performance on limited hardware,…

Rocketride Server

AI GitHub Repos

#237

High-performance AI pipeline engine with a C++ core. It supports over 50 Python-extensible nodes, enabling the building, debugging, and scaling of LLM…Rocketride Server is a high-performance AI pipeline engine with a C++ core. It supports over 50 Python-extensible nodes, enabling the building, debugging, and scaling of LLM…

GPUStack

AI GitHub Repos

#116

GPU cluster manager designed for high-performance AI model serving, including vLLM and SGLang, and provides on-demand SSH-accessible GPU instances. It facilitates…GPUStack is a GPU cluster manager designed for high-performance AI model serving, including vLLM and SGLang, and provides on-demand SSH-accessible GPU instances. It facilitates…

Strata

AI GitHub Repos

#23

Inference engine that enables the deployment of Qwen3.8-Flash-Next on consumer hardware. It supports one-click installation on Windows and Linux, and offers…Strata is an inference engine that enables the deployment of Qwen3.8-Flash-Next on consumer hardware. It supports one-click installation on Windows and Linux, and offers…

omlx

AI GitHub Repos

#138

LLM inference server designed for Apple Silicon, featuring continuous batching and SSD caching. It is managed directly from the macOS menu bar, providing an efficient…omlx is an LLM inference server designed for Apple Silicon, featuring continuous batching and SSD caching. It is managed directly from the macOS menu bar, providing an efficient…

Seldon Core

AI GitHub Repos

MLOps and LLMOps framework for deploying, managing, and scaling AI systems on Kubernetes. It enables standardized deployment of various model types across…Seldon Core is an MLOps and LLMOps framework for deploying, managing, and scaling AI systems on Kubernetes. It enables standardized deployment of various model types across…

KServe AI Inference Platform

AI GitHub Repos

Standardized distributed AI inference platform designed for scalable, multi-framework deployment on Kubernetes. It supports generative and predictive AI models,…KServe is a standardized distributed AI inference platform designed for scalable, multi-framework deployment on Kubernetes. It supports generative and predictive AI models,…

TraceML

AI GitHub Repos

Polyaxon TraceML is an engine designed for AI/ML data tracking, visualization, explainability, and drift detection. It provides comprehensive dashboards for monitoring and…

Radicalbit AI Monitoring

AI GitHub Repos

Comprehensive solution for overseeing AI models in production. It provides tools to ensure model performance and reliability, making it essential for…Radicalbit AI Monitoring is a comprehensive solution for overseeing AI models in production. It provides tools to ensure model performance and reliability, making it essential for…

TruLens Evaluation Tool

AI GitHub Repos

GitHub project designed for evaluating and tracking experiments with large language models (LLMs) and AI agents. It provides tools to assess performance and manage AI…TruLens is a GitHub project designed for evaluating and tracking experiments with large language models (LLMs) and AI agents. It provides tools to assess performance and manage AI…

Whylogs Data Logging Library

AI GitHub Repos

Open-source data logging library designed for machine learning models and data pipelines. It provides visibility into data quality and model performance over time,…Whylogs is an open-source data logging library designed for machine learning models and data pipelines. It provides visibility into data quality and model performance over time,…

Cua

AI GitHub Repos

Open-source project designed to enhance computer usage with cross-OS fleets, drivers, and benchmarks for training and evaluation. It aims to streamline data generation…Cua is an open-source project designed to enhance computer usage with cross-OS fleets, drivers, and benchmarks for training and evaluation. It aims to streamline data generation…

mlc-llm

AI GitHub Repos

Universal LLM deployment engine that utilizes machine learning compilation to streamline the deployment process. It is designed to enhance the efficiency and…mlc-llm is a universal LLM deployment engine that utilizes machine learning compilation to streamline the deployment process. It is designed to enhance the efficiency and…

MNN

AI GitHub Repos

High-performance, lightweight inference engine developed by Alibaba. It is designed to optimize on-device LLMs and Edge AI applications, providing fast and efficient…MNN is a high-performance, lightweight inference engine developed by Alibaba. It is designed to optimize on-device LLMs and Edge AI applications, providing fast and efficient…

ms-swift

AI GitHub Repos

GitHub repository that enables the use of PEFT or full-parameter tuning for over 600 large language models (LLMs) and 300 multi-modal language models (MLLMs). It…ms-swift is a GitHub repository that enables the use of PEFT or full-parameter tuning for over 600 large language models (LLMs) and 300 multi-modal language models (MLLMs). It…

LMCache

AI GitHub Repos

High-performance key-value cache layer designed to enhance the efficiency of large language models (LLMs). It optimizes data retrieval speeds, ensuring faster…LMCache is a high-performance key-value cache layer designed to enhance the efficiency of large language models (LLMs). It optimizes data retrieval speeds, ensuring faster…

Soup

AI GitHub Repos

GitHub repository that enables the fine-tuning of large language models (LLMs) using a single YAML file. It utilizes layer streaming to train an 8 billion parameter…Soup is a GitHub repository that enables the fine-tuning of large language models (LLMs) using a single YAML file. It utilizes layer streaming to train an 8 billion parameter…

GenieX

AI GitHub Repos

Enables the execution of frontier LLMs and VLMs locally on Qualcomm devices, utilizing NPU, GPU, and CPU. It simplifies the process with minimal code, offering efficient AI…GenieX enables the execution of frontier LLMs and VLMs locally on Qualcomm devices, utilizing NPU, GPU, and CPU. It simplifies the process with minimal code, offering efficient AI…

GPTCache

AI GitHub Repos

Semantic cache designed for large language models (LLMs), fully integrated with LangChain and llama_index. It optimizes the performance of LLMs by caching semantic…GPTCache is a semantic cache designed for large language models (LLMs), fully integrated with LangChain and llama_index. It optimizes the performance of LLMs by caching semantic…

FlashInfer

AI GitHub Repos

Kernel library designed for serving large language models (LLMs). It facilitates efficient deployment and management of LLMs, enabling developers to contribute and…FlashInfer is a kernel library designed for serving large language models (LLMs). It facilitates efficient deployment and management of LLMs, enabling developers to contribute and…

AI Manager

AI GitHub Repos

Desktop application designed to streamline the use of AI coding tools. It provides developers with an integrated environment to manage and deploy AI models…AI Manager is a desktop application designed to streamline the use of AI coding tools. It provides developers with an integrated environment to manage and deploy AI models…