vllm-ascend alternatives
Tools that could take the place of vllm-ascend, ranked by the stacks builders put them in rather than by stars alone.
Other runtimes in the runchr registry that do the work of vllm-ascend — a tool somebody pairs with it ranks above a more popular stranger. Read the vllm-ascend page or browse every runtimes.
A high-throughput inference and serving engine for GPUs.
CLI proxy that compresses shell command output before it reaches an LLM.
Extensible multi-platform HTTP/1-2-3 web server with automatic HTTPS.
Distributed AI compute engine for scaling ML training, tuning, and serving.
Optimized inference runtime for LLMs and visual gen models on NVIDIA GPUs.
Self-host open-source LLMs as OpenAI-compatible API endpoints.
Run and manage AI workloads across clouds, Kubernetes, and Slurm.
Python framework for serving AI models and LLM apps as production APIs.
Datacenter-scale distributed inference serving framework for LLMs and diffusion
Toolkit for compressing, deploying, and serving LLMs with TurboMind and PyTorch
AI-native proxy server and data plane for agentic apps with LLM routing.
Kubernetes platform for deploying and serving generative and predictive AI