@inproceedings{murahari-etal-2023-mux,
    title = "{MUX}-{PLM}s: Pre-training Language Models with Data Multiplexing",
    author = "Murahari, Vishvak  and
      Deshpande, Ameet  and
      Jimenez, Carlos  and
      Shafran, Izhak  and
      Wang, Mingqiu  and
      Cao, Yuan  and
      Narasimhan, Karthik",
    editor = "Can, Burcu  and
      Mozes, Maximilian  and
      Cahyawijaya, Samuel  and
      Saphra, Naomi  and
      Kassner, Nora  and
      Ravfogel, Shauli  and
      Ravichander, Abhilasha  and
      Zhao, Chen  and
      Augenstein, Isabelle  and
      Rogers, Anna  and
      Cho, Kyunghyun  and
      Grefenstette, Edward  and
      Voita, Lena",
    booktitle = "Proceedings of the 8th Workshop on Representation Learning for NLP (RepL4NLP 2023)",
    month = jul,
    year = "2023",
    address = "Toronto, Canada",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.repl4nlp-1.17",
    doi = "10.18653/v1/2023.repl4nlp-1.17",
    pages = "196--211",
    abstract = "The widespread adoption of large language models such as ChatGPT and Bard has led to unprecedented demand for these technologies. The burgeoning cost of inference for ever-increasing model sizes coupled with hardware shortages has limited affordable access and poses a pressing need for efficiency approaches geared towards high throughput and performance. Multi-input multi-output (MIMO) algorithms such as data multiplexing, offer a promising solution with a many-fold increase in throughput by performing inference for multiple inputs at the cost of a single input. Yet these approaches are not currently performant enough to be deployed in modern systems. We change that by developing MUX-PLMs, a class of high throughput pre-trained language models (PLMs) trained with data multiplexing, that can be fine-tuned for any downstream task to yield high-throughput high-performance. Our novel multiplexing and demultiplexing modules proficiently entangle and disentangle inputs, and enable high-performance high throughput that are competitive with vanilla PLMs while achieving 2x/5x inference speedup with only a 1â4{\%} drop on a broad suite of tasks.",
}

The widespread adoption of large language models such as ChatGPT and Bard has led to unprecedented demand for these technologies. The burgeoning cost of inference for ever-increasing model sizes coupled with hardware shortages has limited affordable access and poses a pressing need for efficiency approaches geared towards high throughput and performance. Multi-input multi-output (MIMO) algorithms such as data multiplexing, offer a promising solution with a many-fold increase in throughput by performing inference for multiple inputs at the cost of a single input. Yet these approaches are not currently performant enough to be deployed in modern systems. We change that by developing MUX-PLMs, a class of high throughput pre-trained language models (PLMs) trained with data multiplexing, that can be fine-tuned for any downstream task to yield high-throughput high-performance. Our novel multiplexing and demultiplexing modules proficiently entangle and disentangle inputs, and enable high-performance high throughput that are competitive with vanilla PLMs while achieving 2x/5x inference speedup with only a 1â4{\%} drop on a broad suite of tasks.
Murahari, Vishvak Deshp e, Ameet Jimenez, Carlos Shafran, Izhak Wang, Mingqiu Cao, Yuan Narasimhan, Karthik
MUX-PLMs: Pre-training Language Models with Data Multiplexing