Zhonghao Luo

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

🚀 Excited to share our latest work on advancing Retrieval-Augmented Generation (RAG) systems! Our paper --- Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding --- will be presented at the KDD 2025(W) on Inference Optimization for Generative AI. 📄 Paper: https://lnkd.in/dErdTDf7 What's the Challenge? Traditional RAG systems struggle with: Static retrieval strategies that don't adapt to context Inefficient memory usage in long-context applications Suboptimal balance between retrieval fidelity and response quality Computational bottlenecks during inference Our Solution: PORAG + ATLAS Framework We introduce two complementary techniques: PORAG (Policy-Optimized RAG): Extends Group Relative Policy Optimization (GRPO) to RAG settings Uses dual reward heads for retrieval fidelity AND response quality Employs QLoRA for parameter-efficient fine-tuning Prevents catastrophic forgetting while optimizing retrieval utilization ATLAS (Adaptive Token-Layer Attention Scoring): Multi-Layer Attention Gradient (MLAG) analysis detects information gaps Layerwise Representation Pooling (LRP) constructs targeted queries Dynamic retrieval scaling based on computational load Only retrieves when truly necessary CRITIC for Memory Optimization Our Cache Reduction via Importance-based Token Inclusion Criteria (CRITIC) technique: Reduces memory usage significantly Maintains performance with minimal quality trade-offs Enables longer context processing Test-Time Scaling Integration We demonstrate how inference techniques like Self-Consistency and Monte Carlo Tree Search further enhance performance on complex reasoning tasks. Looking forward to presenting at KDD 2025! Thanks to my amazing teammates Akash Das, SHIVAM GUPTA and Venkataramana Runkana for their incredible contributions! #KDD2025 #RAG #LLM #MachineLearning #AI #RetrievalAugmentedGeneration #DeepLearning #NLP #InferenceOptimization

35

Most existing #lean4 datasets contain only correct proofs. Models learn error correction with RL, that's expensive. With UW Math AI lab we release a dataset of 260k erroneous Lean proofs with - compiler feedback - reasoning trace - corrected proof Improvements in Error Correction: - Goedel 8B: 2x - Kimina 8B: 3x Paper: https://lnkd.in/gaybt4bd

117

2 Comments

Excited to share our new paper published in Human Factors: “Inferring Hidden Attentional States in Driving: A Bayesian Approach to Modeling Distraction and Secondary Task Engagement.” This work was led by Lekhapriya Dheeraj Kashyap as part of her PhD research, and I’m grateful to have collaborated on it. Many real-world systems face the same challenge: the most important states are hidden. In driving, we can observe signals such as speed, eye movements, or pupil dilation — but not the driver’s true attentional state. In this work, we develop a Bayesian decision framework based on a Partially Observable Semi-Markov Decision Process (POSMDP) to infer latent attentional states and model how drivers allocate attention between competing tasks. The model detects distraction earlier than common heuristic rules and reveals substantial heterogeneity in attention strategies. Beyond driving safety, problems like this appear in many AI systems where human states are latent and decisions unfold sequentially. Great collaboration with Lekhapriya Kashyap, Yanling Chang, Maryam Zahabi, and Alfredo Garcia. 📄 Paper: https://lnkd.in/g23ubzht

9

👉 My Key Takeaways from Chip Huyen's Recent Interview on Lenny's Podcast Chip Huyen is the author of the widely recognized "AI Engineering: Building Applications with Foundation Models". Link to the podcast: https://lnkd.in/dxZ-tFWX 💡 Importance of post-training.  Pre-training gives you raw capabilities (next token prediction on massive data), but post-training is what makes the model actually usable. SFT on high-quality examples + RLHF. Fine-tuning should be your last resort, not first. Most problems can be solved with better prompts, better data, or RAG. 💡 Evals.  You can't improve what you can't measure. Need multiple types: unit tests (does this specific prompt work?), integration tests (does the whole pipeline work?), regression tests (did we break something?), and user feedback loops. The hardest part isn't writing evals; it's maintaining them as your product evolves. 💡AI products.  Reliability and UX matter more than models. Most AI product failures aren't about bad models: they're about reliability (API limits, latency spikes, poor monitoring) and UX (users don't understand how to use it, doesn't fit workflow). Building reliable platforms and talking to users constantly beats chasing SOTA models. Most insights come from watching users, not from benchmarks. 💡How to improve AI-powered apps.  What people think improves apps: staying current on AI news, chasing newest agentic framework, obsessing over vector database choice, constantly evaluating model benchmarks, fine-tuning models. What actually improves apps: talking to users, building reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Better prompt engineering beats switching models 90% of the time. A well-crafted system prompt, clear instructions, good examples (few-shot), and proper output formatting can transform a mediocre experience into a great one. 💡 Advice for builders.  Start with user problem, not with cool AI technique. Use the simplest solution that works (often that's a good prompt, not a fine-tuned model). Build evals early. Focus on end-to-end experience. Don't fine-tune unless you've exhausted everything else. Don't treat AI as deterministic (it's not, you need to handle variability). Don't ignore data quality (garbage in, garbage out).

1

1 Comment

Our latest paper has been accepted at AAAI 2026. We introduce Structural Alignment, a reinforcement learning approach that aligns large language models with human-like discourse structures, enabling more coherent and well-organized long-form text generation. Our paper, “Structural Alignment: Aligning Large Language Models with Structural Information,” is available here: https://lnkd.in/gx3ZRTZu

49

ByteDance Seed and Tsinghua AIR just dropped a paper called CUDA Agent that I think deserves serious attention from anyone working on LLM post-training or AI for systems programming. The headline result: an RL-trained model that beats Claude Opus 4.5 and Gemini 3 Pro by ~40% on the hardest CUDA kernel generation benchmarks. But the methodology is where it gets interesting. Three things stand out: 1. Agentic RL, not fixed loops Previous approaches (Kevin, CUDA-L1, ConCuR) all used hand-designed multi-turn cycles: generate code, compile, get feedback, regenerate. The loop structure was static. CUDA Agent gives the model a full shell environment and lets it autonomously decide when to compile, when to profile, when to refactor, and when to stop. Up to 200 interaction turns with 128K context. The model learns the debugging strategy itself, not just the code. 2. Training stability through staged warmup They found that naive agentic RL collapsed after just 17 training steps. The root cause: CUDA code represents less than 0.01% of pretraining data, creating a severe distribution mismatch that destabilizes importance sampling ratios. Their fix is a three-stage pipeline: single-turn RL warmup, then rejection fine-tuning on agent trajectories, then full agentic RL with PPO. This mirrors the classic "SFT before RL" pattern from RLHF, but applied at the agent level. Elegant and practical. 3. Discrete reward normalization Instead of using raw speedup (noisy, biased toward easy kernels), they use a simple 4-level reward: -1 (incorrect), 1 (correct but slow), 2 (faster than eager), 3 (faster than torch.compile). Combined with serious anti-hacking measures (locked profiling scripts, no fallback to torch.nn.functional, randomized inputs). The broader signal here is clear: when you upgrade coding tasks from fixed pipelines to autonomous agents and combine that with large-scale RL, models don't just learn to write code. They learn development strategies. The distribution mismatch problem they solved is fundamentally the same OOD challenge that anyone doing RLHF at scale has encountered, just manifested in a new domain. The limitation worth noting: all 6K training problems are operator-level fusions, not end-to-end kernels like FlashAttention. There's still a meaningful gap between benchmark performance and real-world kernel engineering. Paper: arxiv.org/abs/2602.24286

22

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments

don't miss out this unique opportunity to join forces on benchmarking time series foundation models on different downstream validation tasks: classification, prediction, anomaly detection. A community effort led by Thomas Moreau from the #MIND team (Centre Inria de Saclay, Inria & CEA Joliot). Still a pending question: Do these model perform well for online real-time forecasting such as in BCI applications, like our play field in BrainSync. Alexandre Gramfort Pedro L. C. Rodrigues Théo Gnassounou Mathieu Dagréou Lucas Struber Jean Faber Joseph Paillard Merlin Dumeur Mathurin Massias

26

GEPA over RL. "GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning" link: https://lnkd.in/gJfNJQZc "At the heart of this paper is a singular, transformative claim that challenges the RL paradigm for agent optimization. "The optimal set of instructions governing a compound Al system is discovered more efficiently through an evolutionary process of reflective, language-based self-correction than through numerical policy optimization guided by scalar rewards" #ai

4

We at Haladir (YC W26) released a report on RLFR: Reinforcement Learning from Formally-Defined Rewards for Code Generation (https://lnkd.in/guCH7w7C). The core idea is simple: we present a simple offline and single-turn RL framework that uses formal verification as the primary reward signal for program synthesis in LLMs rather than unit test-based rewards (RLVR), showing improvements over a swath of different coding benchmarks. We fine-tune both Qwen2.5-Coder-7B-Instruct and Qwen3-8B using RLFR and Dafny, a specification language that utilizes Boogie (IVL) and Z3 (SMT solver) to check verification conditions. Despite solely training on Dafny, we show improvements on Python-native coding benchmarks, suggesting generalization past the formal verification domain.

90

8 Comments