[Roadmap] vLLM Roadmap Q1 2025

[simon-mo]

This page is accessible via roadmap.vllm.ai

This is a living document! For each item here, we intend to link the RFC as well as discussion Slack channel in the vLLM Slack

vLLM Core

These projects will deliver performance enhancements to majority of workloads running on vLLM, and the core team has assigned priorities to signal what must get done. Help is also wanted here, especially for people want to get more involved in the core of vLLM.

Ship a performant and modular V1 architecture (#8779, #sig-v1)

• (P0) Optimized default path that is on by default
• (P0) Speculative decoding (n-gram on by default)
• (P0) Efficient memory manager for different shapes of KV cache ([RFC]: Hybrid Memory Allocator #11382)
• (P1) Efficient structured decoding & Jump decoding in V1 ([RFC]: Implement Structured Output support for V1 engine #11908)
• (P1) Full multi-modal support in V1
• (P1) LoRA ([V1] LoRA Support #10957)
• (P2) Hardware support
• (P2) Pipeline parallelism
• (P2) Extension system

Support large and long context models

• (P0) MoE optimizations: Data Parallel for Attention + Expert Parallel for MoE
• (P1) Productionize Prefill Disaggregation
• (P1) Productionize KV Cache offloading to CPU and disk
• (Help Wanted) Investigate context parallelism

Improved performance in batch mode

• (P0) Optimized vLLM in post training workflow (#sig-post-training)
• (P1) Efficiency in batch inference and long generations

Others

• (P0) Blackwell Support
• (P1) Track vLLM Performance
• (Help Wanted) Extensible sampler

Model Support

• Arbitrary HF model ([Model]: Add transformers backend support #11330)
• Alternative or private checkpoint format

Hardware Support

• PagedAttention and Chunked Prefill on Trainium and Inferentia
• Productionize and support large scale deployment of vLLM on TPU
• Progress in Gaudi Support
• Out of tree support for IBM Spyre, Ascend, and Tenstorrent ([RFC]: Hardware pluggable #11162)

Optimizations

• AsyncTP
• Flash Attention 3 (FA3) Support #12429
• Design for sparse KV cache framework

CI and Developer Productivity

• Wheel server
• Multi-platform wheels and docker
• Better performance tracker
• Easier installation (optional dependencies, separate kernel packages)

Ecosystem Projects

These are independent projects that we love to have native collaboration and integration with!

• Distributed batch inference
• Large scale serving
• Production Stack ([Roadmap] vLLM production stack roadmap for 2025 Q1 production-stack#26)
• Multi-modality output
• Collaboration with HuggingFace
• Collaboration with Ollama

---

If any of the items you wanted is not on the roadmap, your suggestion and contribution is strongly welcomed! Please feel free to comment in this thread, open feature request, or create an RFC.

Historical Roadmap: #9006, #5805, #3861, #2681, #244

[Zachary-ai-engineer]

Will vLLM consider optimizing communication operations such as all gather/reduce through 4bit or 8bit quantization?

[JaheimLee]

When will V1 support fp8 kv cache?

[youyc22]

Will vllm consider supporting sparse attention like streamingLLM and h2o?

[new-TonyWang]

Does vllm have plans to optimize host operations? Currently, both scheduling and sampling are handled on the host, which reduces GPU utilization. Is it possible to pipeline the scheduling, model execution, and post-processing operations to improve efficiency?

Three-Stage Pipeline Timeline

Time Step	Scheduling	Model Execution	Post-processing
Step 1	Batch N	—	—
Step 2	Batch N+1	Batch N	—
Step 3	Batch N+2	Batch N+1	Batch N
Step 4	Batch N+3	Batch N+2	Batch N+1
Step 5	Batch N+4	Batch N+3	Batch N+2

---