vLLM alternatives
Tools that could take the place of vLLM, ranked by the stacks builders put them in rather than by stars alone.
Other runtimes in the runchr registry that do the work of vLLM — a tool somebody pairs with it ranks above a more popular stranger. Read the vLLM page or browse every runtimes.
Optimized inference runtime for LLMs and visual gen models on NVIDIA GPUs.
Toolkit for compressing, deploying, and serving LLMs with TurboMind and PyTorch
High-performance LLM inference engine for multi-vendor GPUs and clusters.
Run quantised models locally, with an OpenAI-compatible server.
Self-host open-source LLMs as OpenAI-compatible API endpoints.
Python framework for serving AI models and LLM apps as production APIs.
GPU kernel library for fast LLM inference: attention, GEMM, MoE, sampling.
PaddlePaddle inference and serving toolkit for LLMs and VLMs.
Hardware plugin enabling vLLM inference serving on Huawei Ascend NPUs.
Run open models locally and serve an OpenAI-compatible API.
CLI proxy that compresses shell command output before it reaches an LLM.
Extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS.