Lu Jiang

Anyscale has contributed Ray to the PyTorch Foundation, joining PyTorch, vLLM, and DeepSpeed under open governance to advance scalable, distributed AI infrastructure. Janakiram MSV (Forbes) explains how this governance model reduces fragmentation and supports transparent collaboration across the AI ecosystem in this article: 🔗 https://lnkd.in/dwy4HpTC #PyTorchFoundation #Ray #OpenSourceAI #AIInfrastructure #DistributedAI

137

1 Comment

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Physical Intelligence turns terabytes (TBs) of robotics sensor and video data into high-value training datasets. Before Ray on Anyscale, scaling multimodal data pipelines across diverse hardware slowed progress and demanded deep infrastructure expertise, pulling developers away from differentiated research work. With Anyscale, Physical Intelligence enables more than a dozen of its developers to run experiments concurrently and deploy workloads seamlessly across a multi-cloud environment, accelerating iteration and keeping teams focused on their mission to build advanced AI for robotics. Read the full story: https://lnkd.in/g29uj4vi

29

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments

🚀 Smarter, Faster RAG: A Policy-Optimized, Memory-Efficient Framework for Test-Time Reasoning 📄 New arXiv print 🔗 Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding https://lnkd.in/dErdTDf7 🔍 The Problem Retrieval-Augmented Generation (RAG) boosts LLMs with external knowledge but suffers from: ❌ Static retrieval schedules ❌ Distractor-induced hallucinations ❌ Linear memory growth in KV caching ❌ Inefficient test-time compute without reward alignment. 🧠 Our Framework: PORAG + ATLAS + CRITIC We propose a three-component architecture that dynamically optimizes retrieval, rewards factual and high-quality outputs, and reduces compute/memory costs — all without altering the pretrained base model at inference. 1️⃣ PORAG: Policy-Optimized Retrieval-Augmented Generation • Built on Group Relative Policy Optimization (GRPO) • Fine-tunes with QLoRA adapters, guided by two specialized reward heads: – Retrieval Fidelity (factual correctness) – Response Quality (fluency, coherence, utility) • Enables single-shot decoding with KL-regularized policy updates for stability 2️⃣ ATLAS: Adaptive Token-Layer Attention Scoring • Detects when to retrieve via Multi-Layer Attention Gradient (MLAG) • Builds what to retrieve via Layerwise Representation Pooling (LRP) • Leverages hybrid attention-embedding relevance, with entropy-aware scaling to manage retrieval load 3️⃣ CRITIC: Cache Reduction via Importance-based Token Inclusion Criteria • Selectively retains high-impact tokens in KV cache • Hybrid importance signal: attention strength, entropy spread, gradient sensitivity • Reduces memory and latency for long-context generation, with adaptive compression ⚙️ Exhaustive Test-Time Inference Scaling — No Fine-Tuning Required Our framework integrates advanced inference-time scaling techniques that allow LLMs to trade off compute, latency, and answer quality — dynamically and intelligently. These include: ✔️ Entropy-Guided Decoding — invoke deeper reasoning under uncertainty ✔️ Monte Carlo Tree Search (MCTS) — strategic decoding via multi-branch simulation ✔️ Mixture of Agents — task-routed decoding using a pool of specialized policies ✔️ Lookahead Decoding — speculative multi-token generation and verification These methods are plugged into our test-time control module, which uses token-level entropy, compute budget, and reward models to dynamically select the optimal strategy for each input. 🧠 Conclusion Our framework enables scalable, retrieval-aware LLM inference by combining reward-optimized generation, introspective retrieval, and memory-efficient decoding. It is: ✔️ Transformer-agnostic ✔️ Plug-and-play for SLMs ✔️ Ready for enterprise-scale, compute-sensitive deployments 📌 Full paper on arXiv: https://lnkd.in/dErdTDf7 🤝 With Akash Das, SHIVAM GUPTA, and Venkataramana Runkana #LLMInference #TestTimeScaling

17

Kyunghyun Cho shares his expertise on neural machine translation and deep learning. His contributions to sequence modeling have fundamentally shaped modern NLP approaches. https://lnkd.in/gBvWqsTq #NeurIPS2025 #NeurIPSanDiego

116

AMD’s Instinct and ROCm results show the scalability and efficiency AI customers are asking for, and Computacenter is ready to help translate this into real world deployments.

2

Longfeng Wu is an applied scientist intern at Amazon in Sunnyvale, CA, where she has been focusing on developing large-scale generative recommendation models based on semantic identifiers (IDs) aimed at fundamentally transforming how recommendations are generated and delivered. A Ph.D. student in computer science, Longfeng is advised by Dawei Zhou at the Sanghani Center for Artificial Intelligence and Data Analytics.

60

3 Comments

Compressing LLMs but worried about accuracy? 🎯 EoRA uses eigenspace low-rank approximation to compensate for errors—no retraining needed. A promising direction for scalable, task-adaptive LLMs. New paper from #NVIDIAResearch.

1