Description
TAHO is software from TAHO Labs that targets a specific, well-documented problem: most AI and HPC clusters run at a fraction of their theoretical capacity, commonly cited at around 5% utilization in production, because workloads are large and irregular and don't fit neatly onto the processors running them. TAHO describes itself as an execution fabric that sits directly above the silicon and below orchestration, serving frameworks and caching layers, working underneath whatever a team already runs (Kubernetes, SLURM, vLLM, SGLang, TensorRT-LLM) rather than replacing it.
Read More
Instead of placing a whole workload on one machine and leaving part of the processor idle, TAHO decomposes the workload into small, thread-level tasks and routes each one to the resource that best fits it, whether that's a CPU, GPU, accelerator, or edge device. The company reports this can produce up to 10x faster execution and up to 90% lower compute cost on hardware a team already owns, without requiring new hardware, a migration, or code rewrites. It supports NVIDIA, AMD, Intel and Qualcomm hardware and hyperscaler ASICs, and works with CUDA, ROCm and oneAPI.
TAHO is built for teams running high-throughput, always-on compute such as AI training and inference, multi-threaded workloads, large language models and simulation pipelines; the company says it is not intended for simple web apps, basic APIs or light, intermittent compute. It runs in the cloud, on-premises, at the edge or across hybrid deployments, supports models over 100GB distributed across multiple nodes, and can hot-reload workload updates without downtime. Node-to-node communication uses encrypted peer-to-peer networking, and the company says TAHO adopts a customer's existing security posture with sandboxed execution and runtime isolation.
TAHO is available through cloud marketplaces including Microsoft, with Google Cloud listed as coming soon. Pricing is request-based for business and partner plans; free access is offered to individual AI and infrastructure engineers for a limited time, and to students through an education program. TAHO Labs was founded by Todd Smith (CEO, previously infrastructure roles at Facebook, Snap and Docker), Michal Ashby (CTO, previously at Meta, Snap, Google and Disney) and Justin Gelinas (CBO, co-founder of Britelite Immersive).
- CategoryAI DevOps & Cloud Tools
- Pricing ModelRequest Access
- DeploymentCloud
TAHO's Core Features
Decomposes AI/HPC workloads into thread-level tasks that fit available processors
Works underneath existing orchestrators (Kubernetes, SLURM) and serving frameworks (vLLM, SGLang, TensorRT-LLM) without replacing them
Supports NVIDIA, AMD, Intel and Qualcomm hardware, plus hyperscaler ASICs, via CUDA, ROCm and oneAPI
No code, model or pipeline changes required to adopt it
Automatic failure detection and workload redistribution across healthy hardware
Hot-reload updates to running workloads with no downtime
Encrypted peer-to-peer node-to-node networking and runtime sandboxing
Runs across cloud, on-premises, edge and hybrid deployments, including models over 100GB across multiple nodes
How to use TAHO?
Check the fit: Confirm your workload is high-throughput, always-on compute, such as AI training, inference or HPC simulation.
Get access: Request access as a business, apply as an engineer for free access, or apply through the education program.
Deploy alongside your stack: Install TAHO below your existing orchestrator and serving framework, without changing your code.
Let TAHO decompose the work: TAHO breaks workloads into thread-level tasks and routes them to the best-fitting processor.
Monitor gains: Track utilization, execution speed and compute cost against your existing hardware baseline.
TAHO's Use Cases
- AI training and inference
- LLM serving
- HPC simulation pipelines
- GPU fleet cost reduction
