CLI proxy that compresses shell command output before it reaches an LLM.
Runtimes for DevOps
The 18 runtimes in the runchr registry that do devops work. Deploy and operate infrastructure.
18 of 18 tools · Metrics as of 2026-11-06
Extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS.
Distributed AI compute engine for scaling ML training, tuning, and serving.
Optimized inference runtime for LLMs and visual gen models on NVIDIA GPUs.
Self-host open-source LLMs as OpenAI-compatible API endpoints.
Run and manage AI workloads across clouds, Kubernetes, and Slurm.
Python framework for serving AI models and LLM apps as production APIs.
Datacenter-scale distributed inference serving framework for LLMs and diffusion
Toolkit for compressing, deploying, and serving LLMs with TurboMind and PyTorch
AI-native proxy server and data plane for agentic apps with LLM routing.
Kubernetes platform for deploying and serving generative and predictive AI
GPU cluster manager that orchestrates vLLM, SGLang, and TensorRT-LLM for model
PaddlePaddle inference and serving toolkit for LLMs and VLMs.
Hardware plugin enabling vLLM inference serving on Huawei Ascend NPUs.
Serve large Qwen models fast on consumer GPUs with patched vLLM.
KV cache library enabling elastic GPU memory sharing for LLM serving engines.
Distributed model serving framework for building your own AI cluster.
High-performance ML model serving framework with dynamic batching and CPU/GPU