vLLM
An example of using vLLM with LWS
Examples
This section contains examples of using LWS with or without specific inference runtime.
TensorRT-LLM
An example of using TensorRT-LLM with LWS
llama.cpp
An example of using llama.cpp with LWS
SGLang
An example of using SGLang with LWS
Topology Aware Scheduling with Kueue
An example on using topology aware scheduling with LWS and Kueue, using vLLM
Feedback
Was this page helpful?
Glad to hear it! Please tell us how we can improve.
Sorry to hear that. Please tell us how we can improve.