Weston W.

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

𝗜 𝘄𝗮𝗹𝗸𝗲𝗱 𝗶𝗻𝘁𝗼 𝘁𝗵𝗲 𝗠𝗶𝗰𝗿𝗼𝘀𝗼𝗳𝘁 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗜 𝘄𝗮𝘀 𝗿𝗲𝗮𝗱𝘆. 𝗜 𝘄𝗮𝗹𝗸𝗲𝗱 𝗼𝘂𝘁 𝗿𝗲𝗮𝗹𝗶𝘇𝗶𝗻𝗴 𝗽𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻 𝗶𝘀 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝘀𝗼𝗹𝘃𝗶𝗻𝗴 𝗟𝗲𝗲𝘁𝗖𝗼𝗱𝗲. It’s about clarity. Structure. And knowing what your work really means. I interviewed for SDE1 at Microsoft, Bellevue. Six rounds. Mix of coding, system thinking, and conversations about real-world work. Sharing my experience here because a lot of engineers prepare hard but sometimes prepare for the wrong things. 𝗪𝗵𝗮𝘁 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗵𝗲𝗹𝗽𝗲𝗱 𝗺𝗲: 🔹 𝗞𝗻𝗼𝘄 𝘆𝗼𝘂𝗿 𝗼𝘄𝗻 𝗽𝗿𝗼𝗷𝗲𝗰𝘁𝘀 𝗶𝗻𝘀𝗶𝗱𝗲 𝗼𝘂𝘁 They don’t just ask what you built, they ask why you designed it that way. In my case, discussing a recommendation system project turned into a deep conversation on optimization and PyTorch. That discussion mattered more than writing syntax on a whiteboard. 🔹 𝗖𝗹𝗮𝗿𝗶𝘁𝘆 𝗯𝗲𝗮𝘁𝘀 𝘀𝗽𝗲𝗲𝗱 One question was a classic matrix rotation. Pretty standard. The follow-up changed the shape from square to rectangle. There wasn’t one perfect solution. But talking clearly through edge cases impressed them more than solving fast. 🔹 𝗦𝘆𝘀𝘁𝗲𝗺 𝘁𝗵𝗶𝗻𝗸𝗶𝗻𝗴 𝗶𝘀 𝘂𝗻𝗱𝗲𝗿𝗿𝗮𝘁𝗲𝗱 They asked how I would detect deadlocks across threads. The trick wasn’t writing fancy code. It was mapping the situation to a cycle detection problem in graphs. If you can convert real problems into known concepts, you are already ahead. 🔹 𝗕𝗲𝗵𝗮𝘃𝗶𝗼𝗿𝗮𝗹 𝗿𝗼𝘂𝗻𝗱𝘀 𝗺𝗮𝘁𝘁𝗲𝗿 𝗺𝗼𝗿𝗲 𝘁𝗵𝗮𝗻 𝗽𝗲𝗼𝗽𝗹𝗲 𝘁𝗵𝗶𝗻𝗸 A lot of us ignore this. Microsoft doesn’t. They care about whether you can work in a team, handle disagreements, or explain decisions calmly. Practicing how to talk about your past experiences helps more than cramming 30 more DP problems. 🔹 𝗘𝗱𝗴𝗲 𝗰𝗮𝘀𝗲𝘀 𝗮𝗿𝗲 𝘆𝗼𝘂𝗿 𝗿𝗲𝗮𝗹 𝗶𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄𝗲𝗿𝘀 Questions that look simple usually hide tricky edges. The smallest rectangle covering multiple windows. Even odd sorting with internal descending order. These questions are not about syntax. They test how you think through unclear constraints. 𝗕𝗶𝗴𝗴𝗲𝘀𝘁 𝘁𝗮𝗸𝗲𝗮𝘄𝗮𝘆: Good engineering is communication plus logic. If you can explain your decisions, not just code them, you stand out. If you’re preparing right now, focus on explaining things out loud. Even to yourself. Pretend the interviewer is sitting right there. That practice changes how you think. Happy to chat or share more details if anyone wants to prep together or walk through mock interviews. Thanks for reading and wishing strength to everyone preparing right now.

46

15 Comments

In this post, we explore reinforcement fine-tuning (RFT) for Amazon Nova models, which can be a powerful customization technique that learns through evaluation rather than imitation. We'll cover how RFT works, when to use it versus supervised...

1

Run Claude Code Locally on Apple Silicon Using LM Studio and LiteLLM (Zero Cost) In this blog, Manjunath Janardhan walks us through how to run Claude Code locally on Apple Silicon using LM Studio + LiteLLM + Qwen3-Coder-30B — with zero API cost and no cloud dependency. The setup uses: - LM Studio for local inference - Qwen3-Coder-30B as the coding model - LiteLLM as a bridge between Claude Code and a local OpenAI-compatible endpoint This is a practical way to get the Claude Code experience fully offline on macOS, especially for M-series Macs where MLX models can make a real difference. Worth reading if you want to run agentic coding tools on your own machine without relying on the cloud. Read it from the comments!

58

3 Comments

🤔 Curious how Pinterest optimized analytics at scale? Watch this talk - 🔗 https://lnkd.in/gt6baevJ with Pinterest Senior Staff Engineer and Presto TSC member Zhenxiao Luo as he walks through how StarRocks became a cornerstone of Pinterest’s analytics stack. Get an in-depth look at how the team approached customer-facing analytics, overcame limitations with legacy tools like Apache Druid and HBase, and optimized their Partner Insights platform—cutting P90 latency by 50% while using just 32% of the previous instances. #DataAnalytics #DataEngineering  #DataLakeAnalytics #DataLake #DataLakeHouse

5

#ByteDance has released #BAGEL, an open-source #multimodal foundation model designed for unified understanding and generation across text, image, and video. Unlike models that treat these modalities in isolation, BAGEL handles them within a shared transformer architecture, allowing for precise visual edits, photorealistic scene generation, and long-context reasoning across sequences. One of BAGEL’s core strengths is its ability to maintain #artistic and structural intent during image manipulation. Rather than treating edits as surface-level changes, it interprets context, such as #lighting, #composition, and narrative logic, to produce outcomes that feel deliberate and visually coherent. This supports workflows where #creative control and continuity matter, including those involving multi-step transformations or variations within the same conceptual frame. Beyond still imagery, BAGEL was trained on interleaved data that includes video and web-based visual instruction, giving it the ability to understand and predict #temporaldynamics. It handles complex #visuallogic across sequences, including future frame prediction and spatial transformations, which are essential for building #interactive, time-sensitive visual systems. This capability extends to what is often referred to as “#worldmodeling.” BAGEL can generate environments and objects that follow internal consistency across time, space, and user interaction, a requirement for real-time, immersive experiences. The model doesn’t just render appealing visuals; it understands how they evolve and relate within a given system, enabling environments that behave as well as they look. Evaluation results show that BAGEL performs well across a range of benchmarks, from classical image editing to tasks involving compositional reasoning and grounded knowledge. It achieves this through a bottleneck-free design that *avoids separating understanding and generation, allowing both to inform each other within the same context.* BAGEL is open-source and publicly available, offering researchers, developers, and production teams the ability to fine-tune or extend it within their own systems. BAGEL and the Future of Visual Reasoning in Creative Workflows (ScalaBle PerceptuAl Generative ModEL, as world foundational model) #Research Paper: https://lnkd.in/dWqVHBti Website: https://bagel-ai.org/

1