Concepts

This project provides two complementary Kubernetes APIs for distributed AI/ML workloads: LeaderWorkerSet (LWS) (leaderworkerset.x-k8s.io/v1) and DisaggregatedSet (DS) (disaggregatedset.x-k8s.io/v1).

Architecture: Two Complementary APIs

LeaderWorkerSet and DisaggregatedSet work together in a layered architecture:

• LeaderWorkerSet (LWS) (leaderworkerset.x-k8s.io/v1): A foundational API for deploying a group of pods as a single unit of replication. LWS addresses multi-node model-parallel inference where pods within a replica share fate, require tight co-location, and communicate via high-speed interconnects.
• DisaggregatedSet (DS) (disaggregatedset.x-k8s.io/v1): A higher-level orchestration API designed for disaggregated inference architectures (e.g., separating prefill and decode phases). DisaggregatedSet manages and coordinates multiple underlying LeaderWorkerSets as distinct roles within a unified logical workload. DisaggregatedSet uses LeaderWorkerSet as a core primitive, it composes multiple LeaderWorkerSets into a complete serving topology, handling cross-role coordination that individual pod groups cannot manage alone.

┌─────────────────────────────────────────────────────────────┐
│                    DisaggregatedSet                         │
│  (Multi-role orchestration, ratio-preserving rollouts,      │
│   service discovery, slice management, coordinated drain)   │
└──────────────┬───────────────────────────────┬──────────────┘
               │                               │
               ▼                               ▼
  ┌─────────────────────────┐     ┌─────────────────────────┐
  │ LeaderWorkerSet (Role 1)│     │ LeaderWorkerSet (Role 2)│
  │     e.g., Prefill       │     │      e.g., Decode       │
  ├─────────────────────────┤     ├─────────────────────────┤
  │ • Pod group lifecycle   │     │ • Pod group lifecycle   │
  │ • Leader/worker template│     │ • Leader/worker template│
  │ • Exclusive topology    │     │ • Exclusive topology    │
  │ • Subgroup scheduling   │     │ • Subgroup scheduling   │
  │ • Failure restart policy│     │ • Failure restart policy│
  └─────────────────────────┘     └─────────────────────────┘

Comparison Matrix

When to Use Which API

Use LeaderWorkerSet when:

• All inference pods are homogeneous (same model sharding, same hardware requirements across all nodes).
• You do not need to separate prefill from decode (e.g., running standard tensor-parallel inference with vLLM, SGLang, or TensorRT-LLM).
• You need fine-grained control over pod subgroup placement or group restart policies within a single pod group.

Use DisaggregatedSet when:

• You are deploying disaggregated LLM inference (e.g., vLLM with P/D disaggregation, SGLang, or llm-d) where distinct phases (such as prefill and decode) require different GPU types, different container images, or different pod group sizes.
• You want to scale prefill and decode replicas independently based on traffic patterns (e.g., prompt length vs. generation length).
• You require coordinated, lockstep rollouts across multiple roles without disrupting serving ratios or dropping requests.
• You want declarative management of a complex multi-role topology in a single Kubernetes manifest.
• You are evaluating or adopting disaggregated serving architectures with first-class Kubernetes support.