Description
smolmaxxing is an open-source GitHub repository dedicated to pushing the boundaries of running large language models (LLMs) on the smallest possible devices. The project's core mission is to maximize the ability to deploy and execute different LLMs efficiently on hardware typically considered underpowered for such tasks, including smartphones, tablets, and smart TV boxes.
Read More
The repository contains a comprehensive collection of resources, including benchmarks, data, documentation, and tools. It showcases performance metrics for various LLMs, such as Qwen3-Next-80B-A3B and Qwen3.6-35B-A3B, running on specific devices like a OnePlus phone or a Xiaomi TV Box S. The project meticulously tracks token per second (tok/s) performance, demonstrating the feasibility of running models ranging from 260K to 9B parameters on these compact devices.
smolmaxxing provides a structured approach to experimentation, with dedicated directories for assets, benchmarks, data, and documentation. It includes patches and tools to facilitate the optimization process, along with screenshots illustrating the project's progress. The `README.md` file serves as a central hub, featuring a scoreboard of the best-performing models per size class and links to models hosted on Hugging Face.
This project is ideal for developers, researchers, and enthusiasts interested in edge AI, on-device machine learning, and optimizing LLM inference for low-power environments. It fosters collaboration, with contributions from multiple developers, and includes a `CONTRIBUTING.md` guide for those looking to submit their device benchmarks or code improvements. The `LICENSE` file indicates an MIT license, promoting open usage and modification.
- CategoryMLOps & Model Deployment
- Pricing ModelOpen source
- DeploymentCloud, Self-hosted
- PlatformsWeb
smolmaxxing's Core Features
Optimized LLM inference on small devices
Benchmarks for various LLMs on diverse hardware
Support for models from 260K to 9B parameters
Performance tracking (tokens per second)
Documentation for device-specific optimizations
Tools and patches for LLM deployment
Scoreboard of best-performing models
Links to Hugging Face models
Community contributions and issue forms
MIT License for open-source use
Examples for phones, iPads, and TV boxes
Repository for ongoing research and development
Getting Started with smolmaxxing
Clone Repository: Use `git clone` to get the smolmaxxing project onto your local machine.
Explore Documentation: Review the `docs/` directory and `README.md` for project overview, device specifics, and setup instructions.
Review Benchmarks: Examine the `bench/` and `data/` directories to understand existing LLM performance on various small devices.
Install Dependencies: Identify and install any necessary libraries or tools required for running LLMs on your target device.
Apply Patches/Tools: Utilize provided patches and tools from the `patches/` and `tools/` directories to optimize LLM execution.
Run Experiments: Follow the guidelines in `EXPERIMENT.md` to conduct your own LLM inference tests on small devices.
Contribute Findings: Use the `Submit your device` issue form or `CONTRIBUTING.md` to share your results or code improvements.
smolmaxxing's Use Cases
- Edge AI Development
- On-Device LLM Inference
- Performance Benchmarking
- Resource-Constrained AI
- AI for IoT Devices
- Mobile AI Application








