Bozhao(Joshua) Li

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Building an Internal Component Library at Nalong Tech At Nalong Tech, I led the design and development of an internally developed component library, aimed at addressing real engineering challenges such as duplicated development effort, inconsistent UI behavior, and rising maintenance costs across multiple projects. Rather than building components in isolation, the library was shaped by real business needs and production use cases: Business-driven component abstraction: Designed and implemented ~10 core reusable components (e.g. Dynamic Form, Business Form, Common Modal, Lazy Load, Waterfall Flow), covering common backend systems and mobile scenarios Performance-oriented design: Optimized long-list rendering for 10k+ items using IntersectionObserver and react-window, improving rendering performance by ~40% across both PC and H5 Reusable interaction primitives: Built shared Hooks such as useMessage and useConfirm using React Portal, enabling consistent and reusable modal and notification behaviors Maintainability through documentation: Established a Storybook documentation system with visual examples and clear APIs, significantly lowering onboarding and adoption costs Quality and stability: Set up a Jest + React Testing Library testing framework, achieving ~83% test coverage to ensure reliability across multiple projects The component library became a foundational layer reused across projects, reducing redundant development and improving overall delivery efficiency and consistency. For me, the true value of a component library lies not in the number of components, but in how well it supports long-term engineering scalability.

2

Thanks to everyone who attended our lab, “Learning to Steer Large Language Models”, at AAAI in Singapore 🇸🇬 We provided participants with a comprehensive overview of our recently open-sourced toolkits: AI Steerability 360 (https://lnkd.in/ekuBbufr) and In-Context Explainability 360 (https://lnkd.in/gmnFhsJG). We dug deep into how to steer models to be better instruction followers via a variety of steering methods, used novel explainability methods to see if steering actually worked (for the right reasons), and demonstrated how to find the breaking point of the model where steering too strongly starts to become counterproductive (degrading not only response quality but also instruction following ability, the exact thing we were steering for). We have much more planned along the lines of improving the (modular) controllability of generative models and providing tools that both guide better steering and enable richer interpretability of their behavior. Thanks for the great turnout and engagement. We’re looking forward to next time! Association for the Advancement of Artificial Intelligence (AAAI) IBM IBM Research #AAAI #IBM

79

RAG is becoming one of the most critical techniques for improving LLM accuracy — not just a research concept, but a practical system design pattern. It's exciting to see Googlers sharing real-world implementation experience and helping more developers build reliable AI

3

👉 My Key Takeaways from Chip Huyen's Recent Interview on Lenny's Podcast Chip Huyen is the author of the widely recognized "AI Engineering: Building Applications with Foundation Models". Link to the podcast: https://lnkd.in/dxZ-tFWX 💡 Importance of post-training.  Pre-training gives you raw capabilities (next token prediction on massive data), but post-training is what makes the model actually usable. SFT on high-quality examples + RLHF. Fine-tuning should be your last resort, not first. Most problems can be solved with better prompts, better data, or RAG. 💡 Evals.  You can't improve what you can't measure. Need multiple types: unit tests (does this specific prompt work?), integration tests (does the whole pipeline work?), regression tests (did we break something?), and user feedback loops. The hardest part isn't writing evals; it's maintaining them as your product evolves. 💡AI products.  Reliability and UX matter more than models. Most AI product failures aren't about bad models: they're about reliability (API limits, latency spikes, poor monitoring) and UX (users don't understand how to use it, doesn't fit workflow). Building reliable platforms and talking to users constantly beats chasing SOTA models. Most insights come from watching users, not from benchmarks. 💡How to improve AI-powered apps.  What people think improves apps: staying current on AI news, chasing newest agentic framework, obsessing over vector database choice, constantly evaluating model benchmarks, fine-tuning models. What actually improves apps: talking to users, building reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Better prompt engineering beats switching models 90% of the time. A well-crafted system prompt, clear instructions, good examples (few-shot), and proper output formatting can transform a mediocre experience into a great one. 💡 Advice for builders.  Start with user problem, not with cool AI technique. Use the simplest solution that works (often that's a good prompt, not a fine-tuned model). Build evals early. Focus on end-to-end experience. Don't fine-tune unless you've exhausted everything else. Don't treat AI as deterministic (it's not, you need to handle variability). Don't ignore data quality (garbage in, garbage out).

1

1 Comment

If you're a Senior Engineer looking to move up, the next role will likely be a Staff Engineer. And in this guide, Shruti shares tips from her own experience of getting promoted to Staff Engineer at PayPal and Slack. She talks about what Staff Engineers do (and how it's different from Seniors), why you might not be getting promoted, and how to take that next step. https://lnkd.in/g58dnFEG

103

3 Comments