Ax Renjini R. Nair (Microsoft), Damian K. Kowalczyk (Microsoft), Marco Gaudesi (Microsoft), Chhaya Methani (Microsoft) 9d ago

SLM Finetuning for Natural Language to Domain Specific Code Generation in Production

Fine-tuning small language models for domain-specific code generation in production environments with strict latency requirements.

Ax James Nguyen 9d ago

From Recency Bias to Stable Convergence Block Kaczmarz Methods for Online Preference Learning in Matchmaking Applications

Kaczmarz-based preference learning algorithms for real-time matchmaking with stable convergence replacing recency-biased normalization.

Ax Ziyue Liu, Ruijie Zhang, Zhengyang Wang, Yequan Zhao, Yupeng Su, Zi Yang, Zheng Zhang 9d ago

Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning

Extension of Muon optimizer reducing computational overhead in foundation model pre-training through adaptive second-moment preconditioning.

Ax Wei Liu, Anweshit Panda, Ujwal Pandey, Haven Cook, George M. Slota, Naigang Wang, Jie Chen, Yangyang Xu 9d ago

LoDAdaC: a unified local training-based decentralized framework with adaptive gradients and compressed communication

Decentralized learning framework combining adaptive gradients and compressed communication for federated settings with multiple local training steps.

Ax Kening Wang, Di Wen, Yufan Chen, Ruiping Liu, Junwei Zheng, Jiale Wei, Kailun Yang, Rainer Stiefelhagen, Kunyu Peng 9d ago

Towards Multi-Source Domain Generalization for Sleep Staging with Noisy Labels

First benchmark for multi-source domain generalization in automatic sleep staging with noisy labels across institutions and devices.

Ax Chi Zhang, Jingpu Cheng, Zhixian Wang, Ping Liu 9d ago

Closed-Form Concept Erasure via Double Projections

Closed-form method for concept erasure in diffusion models using double projections without iterative optimization.

Ax Prakash Suman, Yanzhen Qu 9d ago

Cross-Validated Cross-Channel Self-Attention and Denoising for Automatic Modulation Classification

Cross-validated self-attention with denoising for automatic modulation classification under low signal-to-noise conditions.

Ax Jose Efraim Aguilar Escamilla, Haoyang Hong, Jiawei Li, Haoyu Zhao, Xuezhou Zhang, Sanghyun Hong, Huazheng Wang 9d ago

When Can You Poison Rewards? A Tight Characterization of Reward Poisoning in Linear MDPs

Characterizes necessity and sufficiency conditions for reward poisoning attacks in reinforcement learning with linear MDPs.

Ax Yujie Li, Jiuniu Wang, Mugen Peng, Guangzuo Li, Wenjia Xu 9d ago

Graph-RHO: Critical-path-aware Heterogeneous Graph Network for Long-Horizon Flexible Job-Shop Scheduling

Heterogeneous graph network with critical-path awareness for long-horizon flexible job-shop scheduling using rolling horizon optimization.

Ax Hongkang Li, Hancheng Min, Rene Vidal 9d ago

Transformers Learn the Optimal DDPM Denoiser for Multi-Token GMMs

Theoretical analysis of why transformers learn optimal DDPM denoiser for multi-token Gaussian mixture models.

Ax Zunhai Su, Hengyuan Zhang, Wei Wu, Yifan Zhang, Yaxiu Liu, He Xiao, Qingyao Yang, Yuxuan Sun, Rui Yang, Chao Zhang, Keyu Fan, Weihao Ye, Jing Xiong, Hui Shen, Chaofan Tao, Taiqiang Wu, Zhongwei Wan, Yulei Qian, Yuchen Xie, Ngai Wong 9d ago

Attention Sink in Transformers: A Survey on Utilization, Interpretation, and Mitigation

Survey on attention sink phenomenon in transformers, covering utilization, interpretation, and mitigation strategies.

Ax Francesco Carlucci, Giovanni Pollo, Xiaying Wang, Massimo Poncino, Enrico Macii, Luca Benini, Sara Vinco, Alessio Burrello, Daniele Jahier Pagliari 9d ago

End-to-end Automated Deep Neural Network Optimization for PPG-based Blood Pressure Estimation on Wearables

Automated DNN optimization for PPG-based blood pressure estimation on resource-constrained wearable devices.

Ax Yogesh Prasanna Kumar Rao, Tamas Keviczky, Raj Thilak Rajan 9d ago

Consensus-based Recursive Multi-Output Gaussian Process

Distributed consensus-based framework for recursive multi-output Gaussian processes in large-scale streaming settings.

Ax Ami Chopra, Supriya Bordoloi, Shyamanta M. Hazarika 9d ago

A Temporally Augmented Graph Attention Network for Affordance Classification

Temporally augmented graph attention network for affordance classification from EEG sequential data.

Ax Rui Lin, Zhenyu Jin, Guancheng Zhou, Xuyang Ge, Wentao Shu, Jiaxing Wu, Junxuan Wang, Zhengfu He, Junping Zhang, Xipeng Qiu 9d ago

Tracing the Thought of a Grandmaster-level Chess-Playing Transformer

Interprets internal computation of Leela Chess Zero transformer using sparse decomposition to explain grandmaster-level reasoning.

Ax Keivan Faghih Niresi, Christian M{\o}ller Jensen, Carsten Skovmose Kalles{\o}e, Rafael Wisniewski, Olga Fink 9d ago

Virtual Smart Metering in District Heating Networks via Heterogeneous Spatial-Temporal Graph Neural Networks

Spatial-temporal graph neural networks for virtual metering in sparsely instrumented district heating networks.

Ax Yuto Omae, Kazuki Sakai, Yohei Kakimoto, Makoto Sasaki, Yusuke Sakai, Hirotaka Takahashi 9d ago

Wolkowicz-Styan Upper Bound on the Hessian Eigenspectrum for Cross-Entropy Loss in Nonlinear Smooth Neural Networks

Theoretical bounds on Hessian eigenspectrum for cross-entropy loss in nonlinear neural networks.

Ax Shihong Ding, Weicheng Lin, Cong Fang 9d ago

Mild Over-Parameterization Benefits Asymmetric Tensor PCA

Theoretical analysis of asymmetric tensor PCA showing gradient descent benefits from mild over-parameterization.

Ax Joana Sim\~oes, Jo\~ao Correia 9d ago

Exploring the impact of fairness-aware criteria in AutoML

Studies fairness-aware criteria in automated machine learning frameworks to mitigate bias and discriminatory outcomes.

Ax Yuqi Su, Xiaolei Fang 9d ago

A Multi-head Attention Fusion Network for Industrial Prognostics under Discrete Operational Conditions

Multi-head attention fusion network for predicting degradation of industrial machinery operating under changing conditions.

Ax Mani Rash Ahmadi 9d ago

The Phase Is the Gradient: Equilibrium Propagation for Frequency Learning in Kuramoto Networks

Theoretical analysis proving phase displacement in Kuramoto oscillator networks equals gradient of loss for frequency learning.

Ax Jie Shi, Siamak Mehrkanoon 9d ago

A Diffusion-Contrastive Graph Neural Network with Virtual Nodes for Wind Nowcasting in Unobserved Regions

Graph neural network with diffusion-contrastive learning for wind nowcasting in regions lacking dense observation networks.

Ax Adil Derrazi, Javad Pourmostafa Roshan Sharami 9d ago

Integrating SAINT with Tree-Based Models: A Case Study in Employee Attrition Prediction

Combines SAINT attention mechanism with tree-based models like XGBoost for improved employee attrition prediction on tabular HR data.

Ax Jiaqi Wen, Pingbo Tang, Shaolei Ren, Jianyi Yang 9d ago

WaterAdmin: Orchestrating Community Water Distribution Optimization via AI Agents

AI agents for optimizing community water distribution systems by scheduling pumps and valves to meet demands while minimizing energy in dynamic real-world environments.

Ax Muhammad Imran Hossain, Md Fazley Rafy, Sarika Khushlani Solanki, Anurag K. Srivastava 9d ago

Battery health prognosis using Physics-informed neural network with Quantum Feature mapping

Combines physics-informed neural networks with quantum feature mapping for battery state-of-health estimation across chemistries.

Ax Rui Chen, Jinsong Wu 9d ago

Structural Gating and Effect-aligned Lag-resolved Temporal Causal Discovery Framework with Application to Heat-Pollution Extremes

Proposes SGED-TCD framework for lag-resolved causal discovery in multivariate time series with applications to environmental data.

Ax Zhe Ye, Aidan Z. H. Yang, Huangyuan Su, Zhenyu Liao, Samuel Tenka, Zhizhen Qin, Udaya Ghai, Dawn Song, Soonho Kong 9d ago

Intent-aligned Formal Specification Synthesis via Traceable Refinement

Presents VeriSpecGen for automatic formal specification synthesis from natural language using LLMs with traceability for code verification.

Ax Eric Easley, Sebastian Farquhar 9d ago

Latent Instruction Representation Alignment: defending against jailbreaks, backdoors and undesired knowledge in LLMs

Introduces LIRA method to defend LLMs against jailbreaks, backdoors, and unlearning by training models to align instruction representation.

Ax Elahe Khatibi, Ziyu Wang, Ankita Sharma, Krishnendu Chakrabarty, Sanaz Rahimi Moosavi, Farshad Firouzi, Amir Rahmani 9d ago

CARE-ECG: Causal Agent-based Reasoning for Explainable and Counterfactual ECG Interpretation

Proposes CARE-ECG, causal agent-based reasoning framework for explainable ECG interpretation combining LLMs with physiological structure.

Ax Ziyu Wang, Elahe Khatibi, Ankita Sharma, Krishnendu Chakrabarty, Sanaz Rahimi Moosavi, Farshad Firouzi, Amir Rahmani 9d ago

Membership Inference Attacks Expose Participation Privacy in ECG Foundation Encoders

Demonstrates membership inference attacks on ECG foundation encoders, exposing participation privacy risks in self-supervised pretraining.

Ax Naichuan Zheng, Hailun Xia, Zepeng Sun, Weiyi Li, Yinze Zhou 9d ago

Towards Green Wearable Computing: A Physics-Aware Spiking Neural Network for Energy-Efficient IMU-based Human Activity Recognition

Proposes physics-aware spiking neural networks for energy-efficient wearable IMU-based human activity recognition on edge devices.

Ax Candi Zheng, Yuan Lan 9d ago

Rethinking the Diffusion Model from a Langevin Perspective

Organizes diffusion model fundamentals from Langevin perspective, offering simplified mathematical framework for beginners.

Ax Ye Su, Mingrui Ye, Yining Wang, Jipeng Guo, Yong Liu 9d ago

Exact Finite-Sample Variance Decomposition of Subagging: A Spectral Filtering Perspective

Derives exact finite-sample variance decomposition for subagging ensembles, providing mathematical characterization of resampling ratios.

Ax Xiangyang Yin, Xingyu Liu, Tianhua Xia, Bo Bao, Vithursan Thangarasa, Valavan Manohararajah, Eric Sather, Sai Qian Zhang 9d ago

CodeQuant: Unified Clustering and Quantization for Enhanced Outlier Smoothing in Low-Precision Mixture-of-Experts

Proposes CodeQuant for quantizing mixture-of-experts models by combining clustering and quantization to handle outlier-induced errors.

Ax Jiahui Zhang, Rouyi Wang, Kuangqi Zhou, Tianshu Xiao, Lingyan Zhu, Yaosen Min, Yang Wang 9d ago

PepBenchmark: A Standardized Benchmark for Peptide Machine Learning

Introduces PepBenchmark, standardized benchmark with datasets and protocols for peptide drug discovery machine learning.

Ax Yuzhen Mao, Qitong Wang, Martin Ester, Ke Li 9d ago

IceCache: Memory-efficient KV-cache Management for Long-Sequence LLMs

Presents IceCache for memory-efficient KV-cache management in long-sequence LLMs via CPU offloading and selective GPU retention.

Ax Shunyu Wu, Jiawei Huang, Weibin Feng, Boxin Li, Xiao Zhang, Erli Meng, Dan Li, Jian Lou, See-Kiong Ng 9d ago

WaveMoE: A Wavelet-Enhanced Mixture-of-Experts Foundation Model for Time Series Forecasting

Proposes WaveMoE, a mixture-of-experts foundation model for time series forecasting using wavelet-enhanced frequency-domain information.

Ax Nikodem Tomczak 9d ago

Heterogeneous Connectivity in Sparse Networks: Fan-in Profiles, Gradient Hierarchy, and Topological Equilibria

Proposes Profiled Sparse Networks with heterogeneous connectivity patterns, benchmarked on vision and tabular classification tasks.

Ax Kewei Zhu, Cameron Wilson, Bartosz Mazur, Yi Li, Ashleigh M. Chester, Peyman Z. Moghadam 9d ago

ReadMOF: Structure-Free Semantic Embeddings from Systematic MOF Nomenclature for Machine Learning

Introduces ReadMOF framework using chemical nomenclature and pretrained language models for metal-organic framework property prediction.

Ax Subramanyam Sahoo 9d ago

Calibration Collapse Under Sycophancy Fine-Tuning: How Reward Hacking Breaks Uncertainty Quantification in LLMs

Studies how reward hacking during RLHF fine-tuning degrades LLM calibration and uncertainty quantification despite improving helpfulness.

Ax Giacomo Cignoni, Simone Magistri, Andrew D. Bagdanov, Antonio Carta 9d ago

Preventing Latent Rehearsal Decay in Online Continual SSL with SOLAR

Explores online continual self-supervised learning with focus on stability-plasticity trade-off in models learning from unlabeled streaming data.

Ax Luis Balderas, Miguel Lastra, Jos\'e M. Ben\'itez 9d ago

MoEITS: A Green AI approach for simplifying MoE-LLMs

MoEITS: green AI approach for reducing computational burden of Mixture-of-Experts LLMs through simplification.

Ax Aviraj Newatia, Michael Cooper, Viet Nguyen, Rahul G. Krishnan 9d ago

Mitigating Privacy Risk via Forget Set-Free Unlearning

Machine unlearning method for removing training data influence without direct access to forget sets.

Ax Rajveer Singh 9d ago

SpectralLoRA: Is Low-Frequency Structure Sufficient for LoRA Adaptation? A Spectral Analysis of Weight Updates

Spectral analysis of LoRA weight updates showing low-frequency dominance enables efficient parameter-efficient fine-tuning.

Ax Haihui Xie, Wenkun Wen, Shuwu Chen, Zhaogang Shu, Minghua Xia 9d ago

Energy-Efficient Federated Edge Learning For Small-Scale Datasets in Large IoT Networks

Federated learning framework for IoT networks with energy efficiency optimization for small-scale datasets.

Ax Hao Wang, Guozhi Wang, Han Xiao, Yufeng Zhou, Yue Pan, Jichao Wang, Ke Xu, Yafei Wen, Xiaohu Ruan, Xiaoxin Chen, Honggang Qi 9d ago

Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents

Self-distillation method for multi-turn LLM agents using skill-conditioning to improve sample efficiency in reinforcement learning.

Ax Binbin Zheng, Xing Ma, Yiheng Liang, Jingqing Ruan, Xiaoliang Fu, Kepeng Lin, Benchang Zhu, Ke Zeng, Xunliang Cai 9d ago

SCOPE: Signal-Calibrated On-Policy Distillation Enhancement with Dual-Path Adaptive Weighting

On-policy distillation method for LLM alignment with adaptive weighting based on signal quality and credit assignment.

Ax Xun Qian, Alexander Gaponov, Grigory Malinovsky, Peter Richt\'arik 9d ago

Communication-Efficient Gluon in Federated Learning

Communication-efficient optimization method extending Muon for federated learning of large language models.

Ax Zikang Shan, Han Zhong, Liwei Wang, Li Zhao 9d ago

Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning

Revisits value modeling in LLM reinforcement learning using generative critics for improved credit assignment.

Ax Giansalvo Cirrincione 9d ago

INCRT: An Incremental Transformer That Determines Its Own Architecture

Transformer architecture that dynamically determines its own depth and width during training by pruning redundant heads.