Concepts

Core concepts of the LeaderWorkerSet and DisaggregatedSet APIs.

This project provides two complementary Kubernetes APIs for distributed AI/ML workloads: LeaderWorkerSet (LWS) (leaderworkerset.x-k8s.io/v1) and DisaggregatedSet (DS) (disaggregatedset.x-k8s.io/v1).

Architecture: Two Complementary APIs

LeaderWorkerSet and DisaggregatedSet work together in a layered architecture:

  • LeaderWorkerSet (LWS) (leaderworkerset.x-k8s.io/v1): A foundational API for deploying a group of pods as a single unit of replication. LWS addresses multi-node model-parallel inference where pods within a replica share fate, require tight co-location, and communicate via high-speed interconnects.
  • DisaggregatedSet (DS) (disaggregatedset.x-k8s.io/v1): A higher-level orchestration API designed for disaggregated inference architectures (e.g., separating prefill and decode phases). DisaggregatedSet manages and coordinates multiple underlying LeaderWorkerSets as distinct roles within a unified logical workload. DisaggregatedSet uses LeaderWorkerSet as a core primitive, it composes multiple LeaderWorkerSets into a complete serving topology, handling cross-role coordination that individual pod groups cannot manage alone.
DisaggregatedSet coordinates roles; LeaderWorkerSet manages pod groupsA DisaggregatedSet creates and coordinates child LeaderWorkerSets for prefill and decode. In this example, prefill processes the prompt and builds the initial KV cache; decode generates output tokens and extends that cache. DisaggregatedSet manages ratio-preserving rollouts, coordinated drain, service discovery, and slices. Both LeaderWorkerSets manage pod group lifecycle, restart policies, leader and worker templates, topology, and subgroups. One slice and one revision are shown.DisaggregatedSetCoordinates roles across the workloadRatio-preserving rollouts · Coordinated drainService discovery · Slice managementCreates and coordinatesLeaderWorkerSetPrefill roleProcesses the promptBuilds the initial KV cacheLeaderWorkerSetDecode roleGenerates output tokensReads and extends the KV cacheBoth LeaderWorkerSets manage their own pod groupsPod group lifecycle · Restart policiesLeader and worker templatesExclusive topology · Subgroup scheduling
One slice and one revision shown. Each role can have children across multiple slices and revisions.

Comparison Matrix

Feature / DimensionLeaderWorkerSet (LWS)DisaggregatedSet (DS)
Primary PurposeDeploying a group of pods as a unit of replicationOrchestrating multi-role disaggregated serving topologies
Unit of ReplicationReplica = 1 leader Pod + N worker PodsSet = Multiple roles, each mapped to a child LWS
Workload TypeHomogeneous multi-node inferenceHeterogeneous multi-role inference (prefill, decode, encode)
CRDleaderworkerset.x-k8s.io/v1disaggregatedset.x-k8s.io/v1
Rollout OwnershipLWS controller (maxUnavailable, maxSurge)DisaggregatedSet controller (lockstep, ratio-preserving)
ScalingHorizontal Pod Autoscaler (HPA) via scale subresourceIndependent per-role scaling & full topology slice scaling
Service DiscoveryHeadless service per replica (UniquePerReplica subdomain policy)Headless service per role with revision-aware routing
Placement & TopologyExclusive topology placement & subgroups per replicaSlice-level placement policy and topology spread
Failure Handling & LifecyclePer-replica restart policies (RecreateGroupOnPodRestart, None, RecreateGroupAfterStart), isolating failure recovery to the affected pod groupPer-role LWS failure handling; coordinated rollout and drain across all roles

When to Use Which API

Use LeaderWorkerSet when:

  • All inference pods are homogeneous (same model sharding, same hardware requirements across all nodes).
  • You do not need to separate prefill from decode (e.g., running standard tensor-parallel inference with vLLM, SGLang, or TensorRT-LLM).
  • You need fine-grained control over pod subgroup placement or group restart policies within a single pod group.

Use DisaggregatedSet when:

  • You are deploying disaggregated LLM inference (e.g., vLLM with P/D disaggregation, SGLang, or llm-d) where distinct phases (such as prefill and decode) require different GPU types, different container images, or different pod group sizes.
  • You want to scale prefill and decode replicas independently based on traffic patterns (e.g., prompt length vs. generation length).
  • You require coordinated, lockstep rollouts across multiple roles without disrupting serving ratios or dropping requests.
  • You want declarative management of a complex multi-role topology in a single Kubernetes manifest.
  • You are evaluating or adopting disaggregated serving architectures with first-class Kubernetes support.

LeaderWorkerSet

Core concepts of LeaderWorkerSet (LWS) — unit of replication, relationship with StatefulSet, architecture, and design rationale.

DisaggregatedSet

Understanding DisaggregatedSet — purpose, relationship to LeaderWorkerSet, and when to use it.

Accelerators

Accelerator-specific integrations across LeaderWorkerSet and DisaggregatedSet.

Feedback

Was this page helpful?