View a PDF of the paper titled Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation, by Luxi Lin and 7 other authors
Abstract:Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft models for every target model, which is costly and inefficient. To address this, we introduce a practical parameter- and data-efficient framework named Efficient Draft Adaptation, abbreviated as EDA, for efficiently adapting draft models. EDA introduces three innovations: (1) a decoupled architecture that utilizes shared and private components to model the shared and target-specific output distributions separately, enabling parameter-efficient adaptation by updating only the lightweight private component; (2) a data regeneration strategy that utilizes the fine-tuned target model to regenerate training data, thereby improving the alignment between training and speculative decoding, leading to higher average acceptance length; (3) a sample selection mechanism that prioritizes high-value data for efficient adaptation. Our experiments show that EDA effectively restores speculative performance on fine-tuned models, achieving superior average acceptance lengths with significantly reduced training costs compared to full retraining. Code is available at this https URL.
Submission history
From: Luxi Lin [view email]
[v1]
Tue, 10 Mar 2026 11:35:58 UTC (1,339 KB)
[v2]
Fri, 13 Nov 2026 09:41:09 UTC (1,356 KB)
![[2603.09527] Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation [2603.09527] Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation](https://techwavedispatch.com/wp-content/uploads/2026/11/260309527-Efficiently-Aligning-Draft-Models-via-Parameter-and-Data-Efficient-Adaptation-1024x597.png)