arXiv:2611.17772v1 Announce Type: new
Abstract: Knowledge distillation (KD) is a highly effective paradigm for transferring capabilities from large language models (LLMs) to smaller, more specialized student models. In commercial deployment where frontier models are restricted behind black-box APIs, sequence-level distillation remains a potent vector for competitors to clone proprietary capabilities at a fraction of the development cost. Existing defenses mostly rely on surrogate models to manipulate next-token sampling, which increases computational overhead and real-time inference latency. In this work, we propose a lightweight defense that dynamically injects style-steering vectors into the teacher model’s activations upon malicious queries. This framework offers exceptional flexibility, allowing hosts to customize target defensive styles and dynamically blend them to maximize stylistic variance. Crucially, this inference-time intervention preserves semantic utility and fluency during malicious queries, misleading adversaries into believing that they are extracting pristine teacher outputs, while standard consumers continue to receive original, unmodified responses. Our experiments demonstrate that this framework successfully disrupts student learning, achieving substantial anti-distillation performance while incurring negligible overhead and preserving text naturalness. Our code is available at https://anonymous.4open.science/r/distill-resist-38C6/.
Trending
- Sleep Vs. Shutdown – Which Should Xbox Series X Owners Actually Use?
- Can You Completely Remove One UI From A Samsung Phone?
- The Purpose Of The SC Port On Your Old Laptop
- Perplexity’s new Monitor API turns web search from a one-off query into a nonstop watchdog
- 10 Rare Game Consoles That Almost No One Owns
- Common Reasons Why Some USB Ports Are Red
- Turning Thermal Noise Into Ultra-Low-Power Signals
- Does Old Computer Hardware Ever Become Truly Useless?
