Sean Xiaoyuan Cui

What I'm Reading This Week 📚 ⚡ Diffusion Coming to LLMs https://lnkd.in/gMrhRsvj Lots of announcements at Google IO this week. One that caught my eye is Gemini's new Diffusion model—applying image diffusion algorithms to text generation for faster speeds. This could be a game changer for latency-sensitive applications like IDE auto-completion and real-time code editing. 🔄 Real-Time Collaboration Architecture https://lnkd.in/ghwrBAxT Matthew Weidner delivers a brilliant deep dive into synchronizing state between central servers and multiple clients. If you've ever wondered how Google Docs manages concurrent edits without chaos, this is your answer. His exploration of operational transformation and server reconciliation patterns gives me new appreciation for the complexity behind seamless collaboration. 📈 Economic Reality Check https://lnkd.in/gfTf2TG9 Thomas Klitgaard from the NY Fed provides a down-to-earth explanation of why the US consistently runs trade deficits. While the current administration's trade war rhetoric continues, one thing is clear: rebalancing trade won't happen without painful adjustments for American consumers. The macroeconomic forces at play are more complex than political soundbites suggest. 🎯 Algebraic Effects Explained https://lnkd.in/g8zeFzWX A fascinating introduction to algebraic effects—essentially resumable exceptions that enable elegant control flow. While this cutting-edge programming language feature might be hard to wrap your head around, the Ante team does an excellent job explaining when and why you'd need such abstractions. 🤖 Google's Coding Agent https://jules.google/ Google launches Jules, their take on cloud-hosted agentic coding loops. Between AI-powered IDEs, local agents like Claude Code, and these cloud solutions, we're clearly still figuring out the right form factor for agentic programming. The overloaded "Codex" branding from OpenAI isn't helping clarity either, but the underlying trend toward autonomous code generation is undeniable. What's your take on the current state of AI coding assistants? Are you finding them more helpful or distracting in your development workflow? #TechArchitecture #AICodeGeneration #DistributedSystems #EconomicPolicy #FutureOfProgramming #AgenticDevelopment #DeveloperTools

15

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Google co-founder Sergey Brin joined the AI Stage at Google I/O where he shared his thoughts on our path to AGI - Artificial General Intelligence. “We fully intend that Gemini will be the very first AGI.” But he says he didn’t return to Google because of the race. He came back to be part of the most consequential scientific moment of our time. “Anybody who's a computer scientist should not be retired.” Exciting times ahead! #GoogleIO

576

13 Comments

Search relevance is the backbone of any RAG system. If the retrieval is bad, the answers are bad — no matter how good the LLM is. We just published a deep dive on how we approach this at Dropbox Dash, and I think the core idea is worth sharing because it applies well beyond our product. The challenge: Dash searches across millions (sometimes billions) of enterprise documents to find the handful that matter for a given query. A relevance model ranks those results, and that model needs training data — lots of it. Human labeling is the gold standard, but it's expensive, slow, and can't touch sensitive customer data. Pure LLM labeling is cheap and fast, but uncalibrated LLMs make systematic mistakes. Our approach: use humans to teach the LLM, then use the LLM to scale. A small set of human-labeled examples calibrates the LLM's relevance judgments. Once the LLM meets quality thresholds, it generates hundreds of thousands of labels that train the production ranking model. The result is a 100x amplification of human labeling effort. A few things that made this work in practice: We focused evaluation on the hardest cases — documents users clicked that the LLM rated low, and documents users skipped that the LLM rated high. That's where the learning happens. We gave the LLM tools to research context before judging. Enterprise search is full of internal acronyms and jargon (at Dropbox, "diet sprite" is a performance management tool, not a soft drink). Without context, even strong models get these wrong consistently. We used DSPy (Community) to programmatically optimize prompts against human judgments, which turned prompt tuning from an art into something measurable and repeatable. The pattern — small human reference set, calibrated LLM evaluation, scaled labeling, continuous monitoring — is generalizable to any domain where you need reliable judgments at scale. Full our post on the Dropbox tech blog: https://lnkd.in/gB_5TECm

69

We are reimagining PyTorch on TPU (https://lnkd.in/gCrM-TZD). This approach still calls XLA but in a profoundly different way, enabling a solid eager mode. In the limit, this new approach will enable a true JIT approach with recompilation and re-optimization, online feedback-directed optimizations, avoidance of OOMs, support for HW fungibility, hyper-optimizations of kernels, and more - all with close to zero code changes!

341

2 Comments

I rebuilt Cloud Foundry. Tiny. On Kubernetes. With AI. Years ago at Pivotal, I worked alongside the team that built Cloud Foundry — the platform that gave the world cf push. It was powerful, but it was also heavy: BOSH, Diego, 20+ VMs just to get started. The world has moved on. Kubernetes won the infrastructure layer. AI is reshaping how we build software. But that original developer experience — "here's my source code, run it on the cloud for me, I don't care how" — that idea never got old. So I built MicroFoundry. It's a single Go binary that gives you mf push on any Kubernetes cluster. Same workflow developers loved in Cloud Foundry, but backed by Kubernetes, Prometheus, Loki, and Grafana Beyla. No BOSH. No Diego. No fleet of VMs. What it does: mf push from source — just like the old days 56 backing services across local K8s, AWS, GCP, and Azure Built-in admin dashboard, observability, and 5-tier RBAC with Keycloak OIDC federation to all major cloud providers — no static credentials MCP Server so AI tools like Claude and Cursor can deploy and manage apps directly. What makes it different is how it was built. The entire project was developed through a structured Human-AI collaborative workflow using Claude Code with a 7-agent review process. I designed the architecture and made the decisions. Claude Code wrote the implementation, reviewed it through multiple specialized agents, and iterated. 93 commits later, it works. This isn't about nostalgia for Cloud Foundry. It's about recognizing that the best developer experience from the previous era deserves to exist in this one — lighter, smarter, and AI-native from day one. Open source, MIT licensed. Check it out: https://lnkd.in/gpBEksQ7 #CloudNative #Kubernetes #CloudFoundry #AI #ClaudeCode #DeveloperExperience #OpenSource #PlatformEngineering

45

9 Comments

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments

Google Research Director Denny Zhou, who founded the LLM Reasoning Team at DeepMind, recently gave a great talk at Stanford’s CS25 class: Large Language Model Reasoning. This was one of the most intuitive talks about reasoning model training and application. He outlined new directions for training models to handle questions where answers aren’t easily verifiable. My team is currently working on an enterprise LLM project focused on using reasoning to extract relevant context and generate responses for unverifiable cases. It’s a tough but promising area, and it’s an interesting time to be building. [Link to Denny Zhou's talk: https://lnkd.in/gdJ6Mzi8]

1,026

8 Comments

Claude just released a memory import tool: https://lnkd.in/eM8c7Gf5 You paste a prompt into ChatGPT (or whatever you're using), it spits out your context — preferences, projects, how you work — and you drop it into Claude's settings. That's it. Claude picks it up from there. The switching cost has always been the real barrier. Not the AI itself, just the fact that you've spent months building context somewhere else. This is Anthropic admitting that out loud and actually doing something about it. Quiet release. Worth knowing about.

2