arXiv:2611.18643v1 Announce Type: new
Abstract: Cross-device Federated Learning (FL) is frequently bottlenecked by the prohibitive memory and communication costs of training deep neural networks on resource-constrained edge hardware. While federated dynamic sparse training aims to alleviate these costs by adjusting sparse structures during training, existing methods rely on magnitude-based heuristics that are fundamentally ill-suited for the non-convergent, heterogeneous environments inherent to FL. To address this challenge, we propose FedFit, a federated dynamic sparse training framework that replaces simple heuristics with optimization-centric criteria for structure adjustment. By leveraging a second-order approximation of the loss landscape via the Fisher Information Matrix, FedFit enables precise and efficient structure adjustment without the overhead of explicit Hessian computation. Empirical evaluations across computer vision and natural language processing benchmarks demonstrate that FedFit significantly narrows the sparse-to-dense accuracy gap, outperforming state-of-the-art methods while maintaining high communication efficiency. Our code is available at https://github.com/Serena-28/Fedfit.git .
Trending
- 5 Gadgets You Should Never Buy Used
- [2511.22613] Variational analysis of determinantal varieties
- Why So Many Desktop PC Cases Have Glass Walls (And Why You Might Not Want That)
- [2611.25942] Gradient Boosting with Kolmogorov-Arnold Units for Interpretable Regression beyond Pairwise Interactions
- How To Use Gemini Notebook To Turn Your Research Into An Interactive Site
- moderated flow matching with data balancing for trajectory planning
- What Data Storage Looked Like In The Early 2000s Vs. Today
- DTS: Delta-Based Tile Scoring for Efficient Long-Context Sparse Attention
