arXiv:2611.26359v1 Announce Type: new
Abstract: Continuous diffusion models generate discrete data through stochastic dynamics in an embedding space. We consider jump diffusions that additionally reset coordinates to embeddings of categorical values. A coordinate’s own corrupted state can make prediction easy, potentially reducing the pressure to learn dependencies between coordinates. We study leave-one-out (LOO) prediction, in which each coordinate is predicted from the corrupted states of the other coordinates. We show that, under stated assumptions, jump diffusions with complete coordinate resets and state-independent clocks preserve the prescribed marginals when reset destinations use exact LOO probabilities and the drift uses the full endpoint posterior. We further show that this drift can be constructed from LOO probabilities by analytic likelihood completion without additional network evaluations. We evaluate unrestricted, LOO, and completed predictions on Text8 and LM1B text generation and ZINC250k molecular graph generation. LOO-based variants improve key generation metrics over matched unrestricted controls on all three datasets. Controlled interventions assess reliance on local inputs and context, and we analyze the effect of analytic likelihood completion.
Trending
- Realized versus Population Reliability: Anytime-Valid Self-Consistency Certification under Correlated Reasoning
- Learning from Context: Leave-One-Out Prediction in Jump Diffusions
- Represented but misread: Post-training changes how language models read out what they represent
- Mamba Flow Matching Neural Processes: Linear-Time Inference for Irregularly Observed Spatial Fields
- Cost-Aware Agentic Feature Optimization System for Fleet-Scale Ranking Models
- TRCF: Temporal-Reference Complementary Forecasting for Irregular Multivariate Time Series
- [2611.26515] Simple Private Noisy SGD is Minimax-Optimal under Population Strong Convexity
- OceanCoder: A Verification-Guided Large Language Model for Automated Simulation Modelling in Ocean Engineering
