Project Maya

LocalAI

AI GitHub Repos

#44

Open-source AI engine that allows users to run various models, including LLMs, vision, voice, image, and video, on any hardware without requiring a GPU. This…LocalAI is an open-source AI engine that allows users to run various models, including LLMs, vision, voice, image, and video, on any hardware without requiring a GPU. This…

GPUStack

AI GitHub Repos

#114

GPU cluster manager designed for high-performance AI model serving, including vLLM and SGLang, and provides on-demand SSH-accessible GPU instances. It facilitates…GPUStack is a GPU cluster manager designed for high-performance AI model serving, including vLLM and SGLang, and provides on-demand SSH-accessible GPU instances. It facilitates…

Shimmy

AI GitHub Repos

Pure-Rust WebGPU inference engine that is OpenAI-API compatible and GGUF native. It runs on any GPU without the need for Python or llama.cpp, offering a single binary…Shimmy is a pure-Rust WebGPU inference engine that is OpenAI-API compatible and GGUF native. It runs on any GPU without the need for Python or llama.cpp, offering a single binary…

openTPU

AI GitHub Repos

Open-source AI accelerator developed by FeSens. It combines RTL, ISA, simulator, compiler, and profiler in one repository, enabling the execution of AI models like…openTPU is an open-source AI accelerator developed by FeSens. It combines RTL, ISA, simulator, compiler, and profiler in one repository, enabling the execution of AI models like…

A free tool that detects your GPU or Mac and shows which open AI models, including LLMs, coding, image, and video models, you can run locally, with VRAM requirements and speed…

AirLLM

AI GitHub Repos

#134

GitHub project that enables 70 billion parameter inference using a single 4GB GPU. It is designed to optimize large language model performance on limited hardware,…AirLLM is a GitHub project that enables 70 billion parameter inference using a single 4GB GPU. It is designed to optimize large language model performance on limited hardware,…

TensorRT-LLM

AI GitHub Repos

#64

Provides a Python API for defining Large Language Models, optimizing inference on NVIDIA GPUs. It includes components for creating Python and C++ runtimes to…TensorRT-LLM provides a Python API for defining Large Language Models, optimizing inference on NVIDIA GPUs. It includes components for creating Python and C++ runtimes to…

GenieX

AI GitHub Repos

Enables the execution of frontier LLMs and VLMs locally on Qualcomm devices, utilizing NPU, GPU, and CPU. It simplifies the process with minimal code, offering efficient AI…GenieX enables the execution of frontier LLMs and VLMs locally on Qualcomm devices, utilizing NPU, GPU, and CPU. It simplifies the process with minimal code, offering efficient AI…