Kumar Chellapilla

New work on Rethinking Thinking Tokens: LLMs as Improvement Operators: https://lnkd.in/errhNuuz Reasoning training encourages LLMs to produce long chains of thought (CoT), improving accuracy via self-checking but increasing context length, compute cost, and latency. This work studies whether frontier models can achieve better trade-offs, higher accuracy with lower cost. The paper develops a simple yet effective Parallel-Distill-Refine (PDR) procedure: Generate diverse drafts in parallel, Distill them into a compact textual workspace, and Refine conditioned on this workspace. This decouples context length from total token count, allowing control over compute via parallelism. PDR yields higher accuracy than long CoT at lower latency. Training an 8B model with RL to align with PDR further shifts the Pareto frontier. On math benchmarks, PDR achieves +11% (AIME 2024) and +9% (AIME 2025) over single-pass baselines. With Lovish Madaan, Aniket Didolkar, Suchin Gururangan, John Quan, Ruan Silva, Manzil Zaheer, Sanjeev Arora, and Anirudh Goyal.

78

1 Comment

Hello SmartCAE friends, In our last webinar, we addressed the role of AI and Digital Twins in the supply chain. https://lnkd.in/grYbn-YR We are often asked why we don’t use AI at the moment and instead focus strongly on Digital Twins. We always address the topic of hallucinations, which can be a major issue, and the fact that AI typically provides results based on probabilities. Our Digital Twins are based on first-principles physics, and thermodynamics is a very well-known field that is described by physical laws we use every day we drive cars and fly planes. AI is a great tool to support ideas in fields where you are an expert, but if you can use a first-principles, science-based model, it is much easier to reproduce results and to prove assumptions. Today, I watched a great podcast by Alex Kantrowitz with one of the gurus of AI, Yann LeCun, former Meta AI Chief and winner of the 2018 Turing Award, where he explained what a realistic picture of AI really looks like beyond all the hype we currently see. Very worth watching. Highly recommended. https://lnkd.in/gd-pUGUv John Thompson, Yann LeCun, Alex Kantrowitz #DigitalColdChain

15

5 Comments

This is likely one of the most comprehensive analytical models (so far) for estimating throughput capacity in human-robot collaborative order pick systems. Together with Mahdi Ghorashi Khalilabadi and René De Koster, we obtain an exact expression for throughput for varying human-robot matching policies and order pick configurations. Extensive validation work! This can form the foundation for evaluating more sophisticated matching policies. Transportation Science #human #robot #collaboration #logistics #orderpick #informs #ecommerce #supplychains https://lnkd.in/gcj9d884

208

11 Comments

Mercor, a startup connecting AI labs with domain experts for model training, has raised $350 million at a $10 billion valuation, led by Felicis Ventures V LP with participation from Benchmark , General Catalyst, and Robinhood Ventures. Initially an AI hiring platform, Mercor pivoted to supplying specialized professionals (e.g., scientists, doctors, lawyers) who train foundational AI models, earning the company hourly finder’s fees. It has since expanded into reinforcement learning infrastructure and aims to build a broader AI powered recruiting marketplace. Mercor’s growth accelerated after OpenAI and Google DeepMind reportedly cut ties with Scale AI, following Meta’s $14 billion investment in that company. The firm claims it’s on track to reach $500M ARR faster than Anysphere (Cursor). Currently, Mercor: Pays $1.5M daily to over 30,000 experts earning an average of $85/hour. Plans to focus on: Expanding its expert network Enhancing matching systems Automating internal processes Mercor’s mission is to help AI systems capture economic subtleties, like judgment, intent, and trade offs, that current models often miss. Mercor’s rise signals a powerful new layer in the AI value chain, one that bridges human expertise with machine learning at scale. As foundation models race to master nuanced reasoning and decision making, Mercor’s network of vetted domain experts provides the critical “human feedback infrastructure” these models need to move beyond surface level intelligence. This positions Mercor not just as a staffing platform, but as a strategic enabler of AI evolution, transforming expert insight into structured, monetizable training data. With $500M ARR in sight and deep adoption by top AI labs, Mercor is emerging as a potential “human capital backbone” for the next generation of foundation models, where judgment, ethics, and domain precision become as valuable as compute. #aiinvestment #aiforall #ai

17

In my last blog, I covered Distributed Inference Serving for LLMs. This time, I’ve written about training, breaking down DDP vs FSDP and what to use when for large-scale deep learning. Link: https://lnkd.in/gcgtSZEk #AI #DeepLearning #DistributedTraining #DDP #FSDP #PyTorch #MLTraining #Scalability #MLOps #GPUComputing #AIInfrastructure

10

1 Comment

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58

Soumith Chintala, 𝗰𝗿𝗲𝗮𝘁𝗼𝗿 of PyTorch and one of the most respected builders in 𝗺𝗼𝗱𝗲𝗿𝗻 𝗔𝗜, has joined Mira Murati ’s fast-rising startup, Thinking Machines Lab. After 11 years at Meta (Facebook), where PyTorch grew from a 𝗻𝗶𝗰𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝘁𝗼 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝗴𝗹𝗼𝗯𝗮𝗹 𝗔𝗜 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻, Chintala’s move signals a new wave in human–𝗔𝗜 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻. Thinking Machines Lab isn’t just gathering top talent from Meta, OpenAI, Anthropic, and leading universities—they’re offering 𝗴𝗮𝗺𝗲-𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘀𝗮𝗹𝗮𝗿𝗶𝗲𝘀 (up to $500K), 𝗮 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 $𝟮 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝗲𝗲𝗱 𝗿𝗼𝘂𝗻𝗱, and are aiming for a $50 billion market valuation. Their new product, Tinker, is already in use at Princeton, Stanford, and by major enterprises. Meta, meanwhile, is regrouping under the 𝗻𝗲𝘄 𝗦𝘂𝗽𝗲𝗿𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗟𝗮𝗯𝘀 division led by Alexandr Wang, with other AI leaders reportedly on the move. Chintala’s reasoning: time to pursue “𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙨𝙢𝙖𝙡𝙡, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙣𝙚𝙬, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙪𝙣𝙘𝙤𝙢𝙛𝙤𝙧𝙩𝙖𝙗𝙡𝙚.” His legacy—the PyTorch tool now thriving in classrooms from 𝗠𝗜𝗧 𝘁𝗼 𝗿𝘂𝗿𝗮𝗹 𝗜𝗻𝗱𝗶𝗮—𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲. #AI #TechNews #PyTorch #Meta #ThinkingMachines #Startup #AIJobs #TalentWar #Innovation #BigFunding

8