The Impact of KV-Transfer Transport on PD-Disaggregation Performance: A Case Study of MiniMax-M3 Inference

This Habr article examines the impact of various network protocols on the performance of disaggregated inference (PD-disaggregation) for large language models. The study uses the MiniMax-M3 model on a cluster of two nodes, each equipped with eight NVIDIA B200 GPUs. The focus is on comparing the efficiency of RDMA/InfiniBand with GPUDirect support against the standard TCP/Ethernet stack when transferring KV-cache between nodes during prefill and decode stages. The experimental methodology covers four levels of analysis, ranging from physical fabric characteristics to end-to-end performance metrics of the sglang engine. A total of 268 test runs were analyzed, identifying critical bottlenecks in distributed inference architecture. The findings provide valuable data for infrastructure optimization when scaling LLM services, emphasizing the importance of selecting the appropriate network transport to minimize latency during context transfer between compute nodes.