Close Menu
techwavedispatch.com

    Subscribe to Updates

    Get the latest creative news from FooBar about art, design and business.

    What's Hot

    Building Your Own Mini PC Is Harder Than You Would Think

    November 21, 2026

    [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures

    November 21, 2026

    [2611.26235] Physics-Informed Feature Representations and Pretrained Tabular Foundation Models for Generalizable Melt Pool Geometry Prediction

    November 21, 2026
    Facebook X (Twitter) Instagram
    Trending
    • Building Your Own Mini PC Is Harder Than You Would Think
    • [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures
    • [2611.26235] Physics-Informed Feature Representations and Pretrained Tabular Foundation Models for Generalizable Melt Pool Geometry Prediction
    • Invertible Transport Between Sample-Only Distributions via Projected Characteristic Functions
    • Six Products And Services Google Discontinued In 2026
    • Reward misalignment as an opportunity for materials discovery
    • The Features You’re Actually Paying $450 Extra For
    • Thermodynamically Consistent Neural Networks for a Universal Fluid Gibbs Free Energy Equation of State
    Facebook X (Twitter) Instagram
    techwavedispatch.com
    Subscribe
    Saturday, November 21
    • Home
    • AI Tools & Software
    • Machine Learning
    • Generative AI
    • Robotics & Automation
    • Future Tech & Hardware
    • AI in Business
    • Ethics & Regulations
    • More
      • Gadgets & Smart Devices
      • Cybersecurity & AI
      • Tech Breakthroughs
    techwavedispatch.com
    Home»Machine Learning»[2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures
    Machine Learning

    [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures

    adminBy adminNovember 21, 2026No Comments2 Mins Read
    Facebook Twitter Pinterest LinkedIn Tumblr Email
    [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures
    Share
    Facebook Twitter LinkedIn Pinterest Email

    [Submitted on 19 Nov 2026]

    View a PDF of the paper titled Learn, Freeze, and Transfer an Activation Function Across Architectures, by Arun Prakash A

    View PDF
    HTML (experimental)

    Abstract:Transfer learning typically reuses knowledge acquired by a source network through its learned weights. We investigate whether a learned activation function can also be transferred and remain useful in networks trained from scratch. We study this complementary form of transfer through a learn-freeze-transfer protocol, in which the activation is optimized jointly with a source network in a single training run, and is then frozen and reused in target networks trained from scratch. A pointwise activation with globally shared parameters is independent of layer dimensions and can therefore be incorporated into networks of different architectures trained from scratch. We instantiate this protocol with FAct, a five-parameter activation represented by a two-harmonic truncated Fourier series, initialized from a Fourier approximation of GELU and learned in a Vision Transformer (ViT) on ImageNet-1K. When reused in a ViT of the same architecture trained from scratch on ImageNet-1K, the frozen activation (Frozen FAct) improves top-1 accuracy by 1.46 percentage points over GELU. Across four smaller target architectures (standard ViT, Swin Transformer, linear-attention ViT, and MLP-Mixer) and four image-classification datasets, it achieves the highest mean accuracy among eleven activation functions in 15 of 16 settings under a common training configuration, with improvements of up to 1.71 percentage points over the best conventional activation. When the learning rate and weight decay are tuned separately for each activation under the same search budget, Frozen FAct still ranks first in 11 of the 16 settings, second in four, and fourth in one. These results provide empirical evidence that a learned activation function can retain its usefulness when transferred across vision architectures, independently of the weights with which it was learned.

    Submission history

    From: Arun Prakash A [view email]
    [v1]
    Thu, 19 Nov 2026 14:44:43 UTC (142 KB)

    Activation Architectures Freeze Function Learn Transfer
    Share. Facebook Twitter Pinterest LinkedIn Tumblr Email
    admin
    • Website

    Related Posts

    [2611.26235] Physics-Informed Feature Representations and Pretrained Tabular Foundation Models for Generalizable Melt Pool Geometry Prediction

    November 21, 2026

    Invertible Transport Between Sample-Only Distributions via Projected Characteristic Functions

    November 21, 2026

    Reward misalignment as an opportunity for materials discovery

    November 21, 2026
    Leave A Reply Cancel Reply

    Top Posts

    NVIDIA Is Investigating Reports That CEO Jensen Huang Was The Target Of Chinese Spy Efforts

    November 8, 202619 Views

    ADIndex: Layer-Adaptive Sparse Attention Indexing via Group-Level Distillation

    November 11, 20266 Views

    One Policy for 100 Household Tasks: Our Solution for the 2026 BEHAVIOR Challenge

    November 11, 20265 Views

    C$^2$-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference

    November 11, 20265 Views
    Don't Miss
    Gadgets & Smart Devices

    Building Your Own Mini PC Is Harder Than You Would Think

    By adminNovember 21, 20260

    Smaller size can mean greater effort. Capix Denan/Shutterstock Building a custom…

    [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures

    November 21, 2026

    [2611.26235] Physics-Informed Feature Representations and Pretrained Tabular Foundation Models for Generalizable Melt Pool Geometry Prediction

    November 21, 2026

    Invertible Transport Between Sample-Only Distributions via Projected Characteristic Functions

    November 21, 2026
    About Us

    Welcome to TechWave Dispatch

    At TechWave Dispatch, we bring you the latest news, insights, and developments from the ever-evolving world of technology. Our mission is to keep readers informed about the innovations shaping our digital future.

    From artificial intelligence and machine learning to cybersecurity, robotics, smart devices, and groundbreaking technologies, we cover the topics that matter in today's fast-moving tech landscape.

    Our Picks

    Building Your Own Mini PC Is Harder Than You Would Think

    November 21, 2026

    [2611.26201] Learn, Freeze, and Transfer an Activation Function Across Architectures

    November 21, 2026

    [2611.26235] Physics-Informed Feature Representations and Pretrained Tabular Foundation Models for Generalizable Melt Pool Geometry Prediction

    November 21, 2026
    Most Popular

    After blocking Muse, Amazon now lets users text Alexa to help them shop

    November 6, 20260 Views

    A controlled world for robot learning and evaluation

    November 6, 20260 Views

    Egyptian and Nigerian English join the Open ASR Leaderboard

    November 6, 20260 Views
    Facebook X (Twitter) Instagram Pinterest
    • Terms and Conditions
    • Privacy Policy
    • Contact Us
    • About Us
    © 2026 TechWave Dispatch. All Rights Reserved.

    Type above and press Enter to search. Press Esc to cancel.