GPTCache

LMCache

AI GitHub Repos

High-performance key-value cache layer designed to enhance the efficiency of large language models (LLMs). It optimizes data retrieval speeds, ensuring faster…LMCache is a high-performance key-value cache layer designed to enhance the efficiency of large language models (LLMs). It optimizes data retrieval speeds, ensuring faster…

omlx

AI GitHub Repos

#135

LLM inference server designed for Apple Silicon, featuring continuous batching and SSD caching. It is managed directly from the macOS menu bar, providing an efficient…omlx is an LLM inference server designed for Apple Silicon, featuring continuous batching and SSD caching. It is managed directly from the macOS menu bar, providing an efficient…

LMDeploy

AI GitHub Repos

#72

Toolkit designed for compressing, deploying, and serving large language models (LLMs). It provides developers with the necessary tools to efficiently manage LLMs,…LMDeploy is a toolkit designed for compressing, deploying, and serving large language models (LLMs). It provides developers with the necessary tools to efficiently manage LLMs,…

vLLM

AI GitHub Repos

#23

High-throughput and memory-efficient inference and serving engine designed for large language models (LLMs). It optimizes performance while minimizing…vllm-project/vllm is a high-throughput and memory-efficient inference and serving engine designed for large language models (LLMs). It optimizes performance while minimizing…

Soup

AI GitHub Repos

GitHub repository that enables the fine-tuning of large language models (LLMs) using a single YAML file. It utilizes layer streaming to train an 8 billion parameter…Soup is a GitHub repository that enables the fine-tuning of large language models (LLMs) using a single YAML file. It utilizes layer streaming to train an 8 billion parameter…

FlagEmbedding

AI GitHub Repos

GitHub repository focused on retrieval and retrieval-augmented large language models (LLMs). It provides tools and resources for developers to contribute to the…FlagEmbedding is a GitHub repository focused on retrieval and retrieval-augmented large language models (LLMs). It provides tools and resources for developers to contribute to the…

openvibe

AI GitHub Repos

Python-based, open-source framework designed for AI coding agents. It provides a modular structure for developing and deploying Auto-GPT applications. Key features…Openvibe is a Python-based, open-source framework designed for AI coding agents. It provides a modular structure for developing and deploying Auto-GPT applications. Key features…

EleutherAI GPT-NeoX

AI GitHub Repos

Open-source implementation of model parallel autoregressive transformers on GPUs. It leverages the Megatron and DeepSpeed libraries to facilitate…EleutherAI GPT-NeoX is an open-source implementation of model parallel autoregressive transformers on GPUs. It leverages the Megatron and DeepSpeed libraries to facilitate…