OpenLLM alternatives
Tools that could take the place of OpenLLM, ranked by the stacks builders put them in rather than by stars alone.
Other runtimes in the runchr registry that do the work of OpenLLM — a tool somebody pairs with it ranks above a more popular stranger. Read the OpenLLM page or browse every runtimes.
Python framework for serving AI models and LLM apps as production APIs.
A high-throughput inference and serving engine for GPUs.
CLI proxy that compresses shell command output before it reaches an LLM.
Extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS.
Distributed AI compute engine for scaling ML training, tuning, and serving.
Optimized inference runtime for LLMs and visual gen models on NVIDIA GPUs.
Run and manage AI workloads across clouds, Kubernetes, and Slurm.
Datacenter-scale distributed inference serving framework for LLMs and diffusion
Toolkit for compressing, deploying, and serving LLMs with TurboMind and PyTorch
AI-native proxy server and data plane for agentic apps with LLM routing.
Kubernetes platform for deploying and serving generative and predictive AI
GPU cluster manager that orchestrates vLLM, SGLang, and TensorRT-LLM for model