arXiv:2611.18649v1 Announce Type: new
Abstract: We introduce an algebraic foundation for neural architecture design in which convolution, attention, state-space models, and equivariant layers are expressed through compositions of bilinear product operators over two structures: a position algebra governing positional routing and a feature algebra governing feature interactions. This decomposition makes routing, feature algebra, and operator composition explicit design variables.We focus on the feature-algebra axis through an Algebraic Recipe: fix the routing and operator composition of an existing architecture while replacing its feature algebra. This preserves the information-flow structure while allowing computational and symmetry properties to be modified through algebraic specification.We instantiate the recipe in linear attention, Mamba, and equivariant networks. Structure-constant and Clifford algebras improve accuracy at comparable parameter count in linear attention; structure-constant algebras yield up to $3\times$ faster training and inference in Vision Mamba at competitive accuracy; and an $\mathrm{SO}(3)$-representation algebra provides rotational equivariance by construction. These results demonstrate that feature algebra alone can control accuracy, efficiency, and symmetry without redesigning positional routing.This turns algebraic structure into an explicit design variable and reduces architecture design, within our framework, to algebraic specification.
Trending
- [2611.25385] Drifting, Discriminating, and Directioning for Action Coordination in Higher-dimensional Action Space
- Trump threatens ‘tremendous tax’ for moving AI data centers outside the US
- Teenage Engineering’s First CD Player Kind Of Looks Like A Turntable
- Linear RNN Scaling Laws in Stationary Prediction
- Google’s Gemini can now run Mac apps and finish desktop chores in the background while you keep working
- [2611.25342] Class-Conditional Conformal Triage for ICAO Face-Image Compliance under Class Imbalance
- Is It Worth $8 A Month?
- [2606.20932] Hierarchical Pooling for Sheaf Neural Networks
