Skip to content

@vllm-project vLLM

Change the repository type filter

All

    Repositories list

    37 repositories

    • tpu-inference

      Public
      TPU inference for vLLM, with unified JAX and PyTorch support.
      Python
      •
      Apache License 2.0
      •186•328•56•199•Updated May 11, 2026May 11, 2026

    • vllm

      Public
      A high-throughput and memory-efficient inference and serving engine for LLMs
      amdcudainference
      amdcudainferencepytorchtransformeropenaimoellamagptmodel-serving
      Python
      •
      Apache License 2.0
      •17k•80k•2k•2.9k•Updated May 11, 2026May 11, 2026

    • vllm-dashboard

      Public
      TypeScript
      •2•0•1•0•Updated May 11, 2026May 11, 2026

    • perf-eval

      Public
      Performance benchmark & accuracy evaluation for vLLM
      Python
      •2•0•0•1•Updated May 11, 2026May 11, 2026

    • vllm-project.github.io

      Public
      HTML
      •91•39•1•4•Updated May 11, 2026May 11, 2026

    • compressed-tensors

      Public
      A safetensors extension to efficiently store sparse quantized tensors on disk
      Python
      •
      Apache License 2.0
      •86•276•13•26•Updated May 11, 2026May 11, 2026

    • llm-compressor

      Public
      Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM
      compressionquantization
      compressionquantization
      Python
      •
      Apache License 2.0
      •504•3.2k•66•54•Updated May 11, 2026May 11, 2026

    • semantic-router

      Public
      System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge
      kubernetesrustgolang
      kubernetesrustgolangmcpfine-tuningpii-detectionmixture-of-modelshuggingface-transformersbert-classificationllm
      Go
      •
      Apache License 2.0
      •665•4.2k•104•78•Updated May 11, 2026May 11, 2026

    • guidellm

      Public
      Evaluate and Enhance Your LLM Deployments for Real-World Inference Needs
      Python
      •
      Apache License 2.0
      •150•1.1k•64•28•Updated May 11, 2026May 11, 2026

    • speculators

      Public
      A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
      Python
      •
      Apache License 2.0
      •80•420•17•31•Updated May 11, 2026May 11, 2026

    • vllm-gguf-plugin

      Public
      vLLM Quantization plugin for GGUF
      Cuda
      •
      Apache License 2.0
      •1•2•2•2•Updated May 11, 2026May 11, 2026

    • vllm-omni

      Public
      A framework for efficient model inference with omni-modality models
      inferencepytorchtransformer
      inferencepytorchtransformerimage-generationdiffusionmodel-servingmultimodalvideo-generationaudio-generation
      Python
      •
      Apache License 2.0
      •903•4.7k•412•373•Updated May 11, 2026May 11, 2026

    • vllm-gaudi

      Public
      Community maintained hardware plugin for vLLM on Intel Gaudi
      Python
      •
      Apache License 2.0
      •129•39•4•39•Updated May 11, 2026May 11, 2026

    • vllm-ascend

      Public
      Community maintained hardware plugin for vLLM on Ascend
      inferencetransformermodel-serving
      inferencetransformermodel-servingmlopsascendllmllmopsllm-servingvllm
      Python
      •
      Apache License 2.0
      •1.2k•2.1k•1.4k•466•Updated May 11, 2026May 11, 2026

    • vllm-metal

      Public
      Community maintained hardware plugin for vLLM on Apple Silicon
      Python
      •
      Apache License 2.0
      •128•1.1k•11•8•Updated May 11, 2026May 11, 2026

    • recipes

      Public
      Common recipes to run vLLM
      JavaScript
      •
      Apache License 2.0
      •265•787•26•58•Updated May 11, 2026May 11, 2026

    • agentic-api

      Public
      Stateful API logic for agentic applications using vLLM
      Makefile
      •
      Apache License 2.0
      •9•25•1•2•Updated May 11, 2026May 11, 2026

    • aibrix

      Public
      Cost-efficient and pluggable Infrastructure components for GenAI inference
      Go
      •
      Apache License 2.0
      •579•4.8k•276•40•Updated May 11, 2026May 11, 2026

    • vllm-xpu-kernels

      Public
      The vLLM XPU kernels for Intel GPU
      C++
      •
      Apache License 2.0
      •57•42•14•33•Updated May 11, 2026May 11, 2026

    • vllm-daily

      Public
      vLLM Daily Summarization of Merged PRs
      4•50•0•0•Updated May 10, 2026May 10, 2026

    • dllm-plugin

      Public
      vLLM plugin for block-based diffusion language model (dLLM) support
      Python
      •
      Apache License 2.0
      •5•15•5•0•Updated May 9, 2026May 9, 2026

    • ci-infra

      Public
      This repo hosts code for vLLM CI & Performance Benchmark infrastructure.
      HCL
      •
      Apache License 2.0
      •68•38•0•42•Updated May 8, 2026May 8, 2026

    • production-stack

      Public
      vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization
      Python
      •
      Apache License 2.0
      •402•2.3k•98•68•Updated May 7, 2026May 7, 2026

    • bart-plugin

      Public
      vLLM Model plugin for the encoder-decoder BART model
      Python
      •
      Apache License 2.0
      •7•11•2•6•Updated May 7, 2026May 7, 2026

    • router

      Public
      A high-performance and light-weight router for vLLM large scale deployment
      Rust
      •
      Apache License 2.0
      •77•223•13•19•Updated May 6, 2026May 6, 2026

    • flash-attention

      Public
      Fast and memory-efficient exact attention
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •2.7k•122•0•26•Updated May 5, 2026May 5, 2026

    • FlashMLA

      Public
      C++
      •
      MIT License
      •1k•13•0•3•Updated Apr 20, 2026Apr 20, 2026

    • vllm-skills

      Public
      Agent skills for vLLM
      Shell
      •
      Apache License 2.0
      •20•67•3•2•Updated Apr 3, 2026Apr 3, 2026

    • vllm-neuron

      Public
      Community maintained hardware plugin for vLLM on AWS Neuron
      awsinferentiaaws-neuron
      awsinferentiaaws-neurontrainium
      Python
      •
      Apache License 2.0
      •11•29•6•1•Updated Mar 20, 2026Mar 20, 2026

    • perf-dashboard

      Public
      Performance dashboard for vLLM
      Python
      •2•1•0•1•Updated Mar 10, 2026Mar 10, 2026
    ProTip! When viewing an organization's repositories, you can use the props. filter to filter by custom property.