Modern GPU Programming for MLSys Book
Educational resource on GPU programming for ML systems, covering attention kernels, LLM kernels, and optimization techniques.
Educational resource on GPU programming for ML systems, covering attention kernels, LLM kernels, and optimization techniques.
Gartner analysis predicting AI coding costs will exceed developer salaries by 2028 due to token consumption and billing models.
HN show: Appaca, an AI workspace for operators using LLMs to generate code. Third iteration of no-code/LLM idea.
Astryx is an open-source design system built on React and StyleX, originally from Meta's internal tools. Explicitly agent-ready and customizable.
Ludwig is a specification-driven development framework where prose-first markdown specs drive LLM code generation and verification. Targets Rust with cargo-based deterministic checks.
Method addressing visual hallucinations in MLLMs by mitigating visual laziness through counterfactual alignment.
MCTS approach with geometry awareness for solving extremal combinatorial geometry problems efficiently.
Defense mechanisms for tool-using LLM agents against indirect prompt injection via out-of-band policies.
Study of performance gaps in deep learning fault diagnosis techniques across seen vs. unseen programs.
RAG system addressing temporal validity problem where stale facts cause AI agents to serve outdated information.
Formalizes scientific discovery as meta-optimization where LLMs simultaneously modify evaluation criteria and optimize within expanded theory spaces.
Presents MIRROR, a memory-guided MCTS framework for systematic red-teaming of multimodal agentic RAG systems across multiple attack surfaces.
Studies memory depth and parametric consolidation in long-running language agents, introducing loop-drift protocol to test durable behavior adaptation.
Examines automated jailbreak selection using bandit algorithms for non-expert malicious actors to craft effective LLM attacks.
Describes RolloutPipe, a system for overlapping pipelined rollout and training in disaggregated RL architectures for LLM post-training with verifiable rewards.
Proposes semantic early-stopping for multi-agent LLM loops using embedding similarity to halt when output meaning stops improving, reducing token waste.
Introduces parametric open-source games, a continuous model where players choose parameters converted to actions, with equilibrium existence results.
Proposes DMuon, distributed training method for matrix-orthogonalization optimizers reducing communication overhead compared to element-wise optimization.
Describes prizewinning bimanual garment folding solution combining vision-language-action policy with reinforcement learning loop for robotic manipulation.
Uses sparse autoencoders to inspect LLM internal states for forecasting tasks, identifying time-specific knowledge versus generalizable patterns.
Introduces Hierarchical Muon (HiMuon), tiled Newton-Schulz optimization for efficient dense neural network training with reduced computational overhead.
Proposes CARVE, memory-aware recurrent architecture with content-aware gating for efficient chunk-parallel linear attention in sequence models.
Introduces Ribbon, scalable approximation to Dirichlet-reweighted bootstrap for efficient uncertainty quantification in high-dimensional models.
Analyzes fundamental ceiling on multi-model LLM systems (routing, voting, mixture-of-agents), showing accuracy limited by co-failure rate across 67 frontier models.
Proposes methods for implementing generative models on analog hardware with physics-determined dynamics for low-power computation.
Introduces planning experience exploration for GUI agents using small open-source MLLMs, improving task planning and cross-website generalization.
Investigates alignment between sequence probability and correctness in LLMs, quantifying when higher likelihood corresponds to correct outputs.
Theoretical analysis of frequency principle phenomenon showing DNNs learn target functions from low to high frequencies during training.
Proposes kernel distance method for ranking generative models in distributed settings based on output fidelity and diversity.
Studies relationship between over-parameterization in neural networks and adversarial robustness, analyzing vulnerability to adversarial examples.
Byzantine-robust aggregation algorithms for secure decentralized federated learning without central servers.
Chisme: gossip learning framework addressing heterogeneity in resource-constrained edge devices for privacy-preserving distributed learning.
Training-free hallucination mitigation in vision-language models via inter-layer consistency aggregation during decoding.
DMSC: dynamic multi-scale coordination framework addressing static decomposition and inflexible fusion in time series forecasting.
Hybrid neural architecture for instance-aware algorithm selection on maximum clique problems, combining ML and neural networks.
Two-component framework for tabular data generation in low-data regimes combining GANs and fine-tuned LLMs.
Reinforcement learning framework for penetration testing under partial observability, addressing belief state aggregation challenges.
Rotary position encodings applied to graph-structured data using graph Laplacian spectrum for improved attention mechanisms.
Eyes-on-Me: scalable poisoning attack on RAG systems using reusable attention attractors to manipulate retrieval and generation.
GRFs++: refined graph random features with walk-stitching for efficient kernel computations on graph-structured data.
Reinforcement fine-tuning of flow-matching Vision-Language-Action models through online interaction, improving performance beyond supervised data.
Theoretical bounds on private and robust alignment of language models under privacy constraints and adversarial corruption.
Investigation of spurious rewards paradox in RLVR for LLMs: models bypass reasoning when trained with incorrect rewards, identified via perplexity divergence.
Dual-Prototype Disentanglement framework for context-aware time series forecasting by dynamically separating temporal patterns.
DASH optimizer: faster Shampoo implementation via batched block preconditioning and efficient inverse-root solvers for second-order optimization.
Linear RNNs trained on code for state-tracking tasks, bridging sequence-to-sequence learning with next-token prediction in language models.
SEMixer: MLP-Mixer architecture with random attention for multiscale time series forecasting, addressing redundancy and noise in temporal data alignment.
Temporal Predictive Coding improved for learning long-range dependencies in recurrent systems on neuromorphic hardware through better credit assignment mechanisms.
BrepCoder is a multimodal LLM for CAD tasks using B-rep format instead of point clouds/images, enabling unified multi-task reasoning without task-specific modifications.
Training-free protein sequence generation using stochastic attention on small sequence alignments.