PARD: Enhancing Goodput for Inference Pipeline via Proactive Request Dropping

Zhixin Zhao, Yitao Hu, Simin Chen, Mingfang Ji, Wei Yang, Yuhao Zhang, Laiping Zhao, Wenxin Li, Xiulong Liu, Wenyu Qu, Hao Wang
December, 2025
Type
Conference paper
Publication
Proceedings of the 21st European Conference on Computer Systems
Machine Learning System Inference