[2605.05871] Retain-Neutral Surrogates for Min-Max Unlearning

View a PDF of the paper titled Retain-Neutral Surrogates for Min-Max Unlearning, by Junhao Cai and 6 other authors

View PDF
HTML (experimental)

Abstract:Machine unlearning aims to make a model forget designated training data while retaining performance on the rest. In min-max unlearning, an inner adversary perturbs the model to maximize the forget loss, and the model is updated to minimize the retain loss at the resulting surrogate point. When the forget and retain gradients are positively coupled, the standard perturbation also moves toward higher retain loss, and the retain step begins by recovering that loss. We propose Retain-Orthogonal Surrogate Unlearning (ROSU), which redefines the adversary to admit only perturbations orthogonal to the retain gradient. The one that raises the forget loss fastest has a closed form, and ROSU uses it to set the surrogate point and amplifies it into the forgetting step. Under positive coupling the ROSU surrogate avoids the retain cost that coupling induces in the standard surrogate, and the difference shrinks as the gradients approach orthogonality. ROSU has the smallest accuracy gap to retraining in every vision setting (CIFAR-10, CIFAR-100, Tiny-ImageNet) and is competitive with the baselines on language benchmarks (TOFU, WMDP). The redefinition helps most under high positive coupling. Code is available at this https URL.