Hydragen: High-Throughput LLM Inference with Shared Prefixes
Paper
•
2402.05099
•
Published
•
18
efficient inference
-
-
Ouroboros: Speculative Decoding with Large Model Enhanced Drafting
Paper • 2402.13720 • Published • 6 -
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention
Paper • 2405.12981 • Published • 28 -
Your Transformer is Secretly Linear
Paper • 2405.12250 • Published • 150 -
Not All Language Model Features Are Linear
Paper • 2405.14860 • Published • 39 -
Addition is All You Need for Energy-efficient Language Models
Paper • 2410.00907 • Published • 144