rtp-llm alternatives
Tools that could take the place of rtp-llm, ranked by the stacks builders put them in rather than by stars alone.
Other runtimes in the runchr registry that do the work of rtp-llm — a tool somebody pairs with it ranks above a more popular stranger. Read the rtp-llm page or browse every runtimes.
Run open models locally and serve an OpenAI-compatible API.
Run quantised models locally, with an OpenAI-compatible server.
A high-throughput inference and serving engine for GPUs.
CLI proxy that compresses shell command output before it reaches an LLM.
Extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS.
Distributed AI compute engine for scaling ML training, tuning, and serving.
Single-file executables that run LLMs and speech-to-text locally.
Optimized inference runtime for LLMs and visual gen models on NVIDIA GPUs.
On-device inference engine for phones, wearables, and robots.
Self-host open-source LLMs as OpenAI-compatible API endpoints.
OpenVINO toolkit optimizes and deploys deep learning inference across Intel
Portable C99 inference engine running Kimi K3 on a single CPU in 8 GB RAM.