Chandra Khatri

Soumith Chintala, 𝗰𝗿𝗲𝗮𝘁𝗼𝗿 of PyTorch and one of the most respected builders in 𝗺𝗼𝗱𝗲𝗿𝗻 𝗔𝗜, has joined Mira Murati ’s fast-rising startup, Thinking Machines Lab. After 11 years at Meta (Facebook), where PyTorch grew from a 𝗻𝗶𝗰𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝘁𝗼 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝗴𝗹𝗼𝗯𝗮𝗹 𝗔𝗜 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻, Chintala’s move signals a new wave in human–𝗔𝗜 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻. Thinking Machines Lab isn’t just gathering top talent from Meta, OpenAI, Anthropic, and leading universities—they’re offering 𝗴𝗮𝗺𝗲-𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘀𝗮𝗹𝗮𝗿𝗶𝗲𝘀 (up to $500K), 𝗮 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 $𝟮 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝗲𝗲𝗱 𝗿𝗼𝘂𝗻𝗱, and are aiming for a $50 billion market valuation. Their new product, Tinker, is already in use at Princeton, Stanford, and by major enterprises. Meta, meanwhile, is regrouping under the 𝗻𝗲𝘄 𝗦𝘂𝗽𝗲𝗿𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗟𝗮𝗯𝘀 division led by Alexandr Wang, with other AI leaders reportedly on the move. Chintala’s reasoning: time to pursue “𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙨𝙢𝙖𝙡𝙡, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙣𝙚𝙬, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙪𝙣𝙘𝙤𝙢𝙛𝙤𝙧𝙩𝙖𝙗𝙡𝙚.” His legacy—the PyTorch tool now thriving in classrooms from 𝗠𝗜𝗧 𝘁𝗼 𝗿𝘂𝗿𝗮𝗹 𝗜𝗻𝗱𝗶𝗮—𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲. #AI #TechNews #PyTorch #Meta #ThinkingMachines #Startup #AIJobs #TalentWar #Innovation #BigFunding

8

'BharatGen' - Indian Institute of Technology, Bombay's AI venture. BharatGen's ambition is not just to build LLMs, but to build ones that sound and think like India. It's strength said Prof. Ganesh Ramakrishnan, lies in training systems on home-grown datasets and Indian languages. Rishi Bal Milind Atrey Shireesh Kedare Shashi Shekhar Vempati Abhay Karandikar Abhishek Singh

782

17 Comments

🧾 Manifold Constrained Hyper Connections by DeepSeek Thanks to Uday Kamath, Ph.D. - who has a fantastic write-up on this. ❓What is it? A new technique that enables training higher quality LLMs with minimal increases in overhead. Modern Transformers have a highway “fast lane” to remember the initial “query”. To date, this has been limited to a small number of lanes. Previously, Deep Seek had shown that increasing the number of lanes improves quality…but that came with a lot of risk. Large-scale trainings could explode out of control, at immense capital cost. By constraining the manifolds, they shift the efficient frontier and increase reward with minimum increase in risk. A big breakthrough. 📈 Why it matters to business • Foundation models can have sizable jumps in quality with minimal costs changes. • The largest models can now be more ambitious without an increased fear of failed trainings. 🛠️ The technology details • Residual connection lanes are widened to incorporate more features. The new residuals have learned pre and post transformations. • In order to achieve stability, they incorporated the 1960s Sinkhorn-Knopp algorithm, which transforms the residual connection into a doubly stochastic matrix. • This markedly increased stability of training, measured by the Amax Gain Magnitude. • The team also devised some interesting innovations in memory management and gradient checkpointing in order to achieve these results. Uday’s Writeup https://lnkd.in/eSD8i9wP Paper https://lnkd.in/eWAgV_Sa

13

I’m glad to share our recent work with Prof. Manish Shrivastava published in #PACLIC39 - “Non-Idiomatic Conventionalised Expressions: A New Pain in the Neck?” where we explore a type of expression that sits in an overlooked space between fully compositional phrases and idioms. Expressions like “as far as I know” or “for the most part” seem straightforward, yet their meaning and usage can’t be fully predicted from the individual words. These are what we call Non-Idiomatic Conventionalised Expressions (NICEs)—phrases that are mostly transparent but still behave in fixed, conventional ways. In this work, we • automatically identify candidate NICEs from a large corpus, • filter and validate them through substitution-based tests, and • show that current NLP systems—especially machine translation—often mishandle them, leading to incomplete or awkward interpretations. Our results suggest that these expressions deserve more focused attention, both linguistically and in downstream NLP applications. #nlp #compositionality #machinetranslation

48

The recommendations of the Why LLMs Hallucinate reminded me of Prof SAK’s course on Formal Methods of Program Verification when I was an undergrad at IIT Delhi. The content of the course was great fun to extremely mathematical people but the grading was hell. His surprise quizzes awarded +1 for the correct answer and -2 for the wrong answer. The people in the class mostly had top 150 ranks in the JEE, and very high GPAs. Most had spent their last few years obtaining extremely certain answers with perfect reasoning. These were not people you would accuse of confident guessing - the JEE used to be dominated long form symbolic and numerical problems. The highest score in one of the quizzes in this course was +2 out of 20. I write “+” because the average score was -2, minus two. Some students missed the quiz. They had the above average score of zero… Thirty years later, this course still features in our conversations. Excessive penalties on confident guessing make for great stories but rewarding abstention with above average scores is not a good idea. I suspect we will end up hobbling models into analysis paralysis.

64

4 Comments

𝐎𝐧𝐞-𝐢𝐧-𝐚-𝐦𝐢𝐥𝐥𝐢𝐨𝐧 𝐟𝐚𝐢𝐥𝐮𝐫𝐞𝐬 𝐜𝐚𝐧 𝐨𝐜𝐜𝐮𝐫 𝐞𝐯𝐞𝐫𝐲 𝐡𝐨𝐮𝐫 𝐢𝐧 𝐥𝐚𝐫𝐠𝐞-𝐬𝐜𝐚𝐥𝐞 𝐝𝐢𝐬𝐭𝐫𝐢𝐛𝐮𝐭𝐞𝐝 𝐬𝐲𝐬𝐭𝐞𝐦𝐬 At scale, no edge case is too obscure. For 𝐃𝐞𝐞𝐩 𝐄𝐧𝐠𝐢𝐧𝐞𝐞𝐫𝐢𝐧𝐠 #𝟓, I spoke with Dhirendra Sinha (Google) and Tejas Chopra (Netflix), engineering leaders and authors of 𝑆𝑦𝑠𝑡𝑒𝑚 𝐷𝑒𝑠𝑖𝑔𝑛 𝐺𝑢𝑖𝑑𝑒 𝑓𝑜𝑟 𝑆𝑜𝑓𝑡𝑤𝑎𝑟𝑒 𝑃𝑟𝑜𝑓𝑒𝑠𝑠𝑖𝑜𝑛𝑎𝑙𝑠 (Packt, August 2024), about designing for failure, AI Ops, and what system design interviews actually measure. We cover: ✅ How Big Tech designs for failure, not perfection ✅ Why AI makes fundamentals more important, not less ✅ What successful senior engineers do differently in system design interviews ✅ When infrastructure-as-code (IaC) becomes a resilience strategy, not just automation We have also included a full excerpt of Chapter 2 from Sinha and Chopra's book covering a practical breakdown of distributed system attributes, theorems, and trade-offs. 🛠️𝐓𝐨𝐨𝐥 𝐨𝐟 𝐭𝐡𝐞 𝐖𝐞𝐞𝐤: Diagrams to define cloud architecture visually, using Python. If you're building distributed systems or preparing for a system design interview, this issue may be worth your time. 🔗𝐑𝐞𝐚𝐝: https://lnkd.in/gs-AztfG #SystemDesign #SoftwareArchitecture #AIOps #EngineeringLeadership #TechInterviews #DeepEngineering

28

I spent the last three years inside the startup ecosystem — working closely on research, development, and manufacturing execution. And I saw the same pattern repeatedly. Startups build strong products. Solid engineering. Clear vision. But manufacturing breaks momentum. 1. Multiple vendors. 2. Multiple timelines. 3. Different quality standards. 4. No unified control. 5. You place an order. Vendor confirms. Then delays start. Sometimes they disappear midway. Sometimes quality slips. Sometimes revisions are ignored. Teams end up firefighting instead of building. MSMEs face it even harder. *Low volumes. *MOQ barriers. *Limited negotiation power. *No traceability. *Fragmented execution. Manufacturing becomes reactive instead of structured That’s why we built Unbound 3D Industries — a unified industrial execution platform. Services we offer *CNC. *Additive manufacturing. *Sheet metal. *Composite fabrication. *Prototype to batch production and much more All under one structured system. One accountability layer. But we don’t just manufacture. We audit. We standardize processes. We improve manufacturing efficiency. We implement traceability. We define acceptance criteria. We make manufacturing more effective and efficient — not just available. Through www.hanzof.com, teams can combine multiple manufacturing services within a single project without MOQ barriers — under one roof. At Unbound3D, one operating principle drives everything: " If we commit, we deliver " If you’re building a product company or running an MSME and have faced manufacturing challenges, let’s connect and discuss Cibhi Adhinath Padmanaban|Mahima Swetha|Vijay Raj J #Unbound3D #Hanzof #ManufacturingExecution #WeOwnTheProcess #HardwareStartup #ProductDevelopment #MSMEIndia #IndianManufacturing #DeepTech #CNC #AdditiveManufacturing #IndustrialEngineering #SupplyChain #QualityControl #MakeInIndia #StartupEcosystem #BuildInPublic #ManufacturingIndia

33

10 Comments

Introducing Agentic Culture — an open-source experiment in collective memory and in-context cultural accumulation for multi-agent systems. A hot topic from the recent Andrej karpathy // dwarkesh podcast is that LLMs don't yet have culture. >>> So we built the scaffolding for them to develop one. Agno Agents can now tap into a shared cultural memory — knowledge that persists beyond individual sessions, users or memories. Culture is collective, evolving context that shapes agent behavior over time. Now they can create, read, explore and learn from their culture. You can use the `CultureManager` class to create and manage `CulturalKnowledge` objects. These are stored in your database of choice (SQLite for now) and automatically retrieved as in-context cultural grounding for your agents. To give your agents access to shared culture, just set `add_culture_to_context=True`. That's it. Agents now learn from collective experience. Explore the code and README: https://lnkd.in/e7E3AcJX Agno is open source — come build with us: https://agno.link/gh

98

4 Comments