Romal Thoppilan

History is repeating itself. 10 years ago the recsys industry was migrating from linear model recsys to sparse neural network recsys. Back in 2016, many people in the field were skeptical that deep learning would deliver positive ROI for recsys, the fear being rooted in the perceived higher training and inference cost. I see people repeat the same fear, uncertainty, and doubt about this year's migration from sparse neural network recsys to LLM-native recsys. Those who do not lean into the LLM-recsys migration this year will fall behind.

78

1 Comment

🧠💬 Memory in LLMs A practical guide showing how to implement conversational memory in LLMs using LangGraph, demonstrated through a therapy chatbot. Features code examples for basic retention, trimming, and summarization approaches. Learn to build memory-aware apps 👉 https://lnkd.in/gybcrV5v

967

21 Comments

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Out team just posted our latest paper “Evaluating the Promise & Pitfalls of LLMs in Hiring Decisions” on arXiv! We found some exciting results: • Benchmarked leading LLMs (GPT-4o, o3, Claude, Gemini, Llama, DeepSeek) against Eightfold’s “Match Score” model on real-world data. • Evaluated both performance (ROC AUC, PR AUC, F1) and fairness (impact-ratio across gender, race, intersectional groups). • Eightfold’s Match Score beat the best LLM on accuracy (ROC AUC 0.85 vs 0.77) and fairness (min race Impact Ratio 0.957 vs 0.809). • Off-the-shelf LLMs still propagate measurable demographic bias without safeguards. • The trade-off between accuracy and fairness is a false dichotomy: carefully engineered, domain-tuned models like Eightfold’s can achieve both accuracy of hiring and fairness of outcomes. https://lnkd.in/guQ2TAYp #machinelearning #ai #eightfold #arxiv #datascience #bias #fairness #ml #data #genai #llms

38

2 Comments

A new RL algorithm that fixes a hidden flaw in PPO The authors propose CE-GPPO (“Coordinating Entropy via Gradient-Preserving Policy Optimization”), a variant of PPO that restores gradient contributions from clipped actions in the policy update. They argue that traditional clipping discards useful gradient signals from low-probability tokens, which play an important role in controlling the agent’s entropy during training. By bounding those gradients in a controlled way, CE-GPPO maintains exploration–exploitation balance more stably than prior methods. They provide a theoretical analysis showing that CE-GPPO mitigates entropy instability, and empirically test it on mathematical reasoning benchmarks. Their results indicate consistent improvement over strong baselines across different model scales, demonstrating that preserving clipped gradients can lead to better performance in reinforcement learning for reasoning tasks. https://lnkd.in/gYZsJ-8f

1

$1B+ bet on the next frontier of AI: Physical & Spatial Intelligence Today, Yann LeCun’s new startup, AMI - Advanced Machine Intelligence, announced a massive $1.03 billion funding round to challenge the dominance of Large Language Models (LLMs). Current AI is "trapped" in a digital box of text. To reach human-level autonomy, AI needs to understand the physical world. It needs to know that if you push an object, it falls; it needs to navigate a room without a map; it needs to plan complex tasks in real-time. From Digital to Physical: AI is moving into robotics, smart glasses (like Ray-Ban Meta), and industrial automation. From Prediction to Planning: Moving away from probabilistic guessing toward goal-oriented reasoning. The "Ami" (Friend) Approach: Building AI that is controllable, safe, and grounded in reality. The next frontier isn't just "generative"—it’s spatial and physical. We are moving from AI that talks to us, to AI that works alongside us in the physical world. Congrats to the AMI team on this milestone. The era of "World Models" is here. https://lnkd.in/gE99YfGJ #AI #Robotics #MachineLearning #YannLeCun #AMI #Innovation #SpatialIntelligence #TechNews

2

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments