Collection by
Jun 4, 2024
Jun 4, 2024
-
S^{3}: Increasing GPU Utilization during Generative Inference for Higher Throughput
Paper ā¢ 2306.06000 ā¢ Published ā¢ 1 -
Fast Distributed Inference Serving for Large Language Models
Paper ā¢ 2305.05920 ā¢ Published ā¢ 1 -
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline
Paper ā¢ 2305.13144 ā¢ Published ā¢ 1 -
Towards MoE Deployment: Mitigating Inefficiencies in Mixture-of-Expert (MoE) Inference
Paper ā¢ 2303.06182 ā¢ Published ā¢ 1
