Jin Gu

🏗️🚢🚀Transformation of the Week: Hou C. - from full-stack chat builder, to advanced multi-agent and RAG AI engineer for UI component generation. Meet Hou Chia, a veteran engineer who brought 10+ years of production discipline to his AI Engineer Bootcamp journey. 📚 𝐒𝐭𝐚𝐫𝐭: 𝐀 𝐬𝐨𝐥𝐢𝐝 𝐟𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧  Hou's journey began by building the full-stack blueprint: a fast, deployed chat application complete with a Python API and a Next.js frontend, demonstrating mastery of the core LLM application stack. 🎯 𝐇𝐢𝐬 𝐥𝐞𝐯𝐞𝐥 𝐮𝐩: 𝐂𝐨𝐦𝐩𝐨𝐧𝐞𝐧𝐭 𝐅𝐨𝐫𝐠𝐞  For his certification challenge, Hou pivoted from general chat to a complex problem: automating the creation of production-ready UI components from abstract design input. The key technical innovations included: - 𝐌𝐮𝐥𝐭𝐢-𝐀𝐠𝐞𝐧𝐭 𝐏𝐢𝐩𝐞𝐥𝐢𝐧𝐞: An orchestrator delegating component requirements (props, states) to specialized sub-agents. - 𝐀𝐝𝐯𝐚𝐧𝐜𝐞𝐝 𝐑𝐀𝐆: A Hybrid Search system combining BM25 and Semantic Retrieval to ground code generation in design patterns. - 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧-𝐚𝐬-𝐚-𝐒𝐞𝐫𝐯𝐢𝐜𝐞: Automated validation of generated component quality against accessibility and coding standards. This moved his work from a 𝐬𝐢𝐧𝐠𝐥𝐞-𝐜𝐚𝐥𝐥 𝐩𝐚𝐭𝐭𝐞𝐫𝐧 to a sophisticated, 𝐦𝐮𝐥𝐭𝐢-𝐬𝐭𝐞𝐩, 𝐝𝐞𝐜𝐨𝐦𝐩𝐨𝐬𝐚𝐛𝐥𝐞 𝐰𝐨𝐫𝐤𝐟𝐥𝐨𝐰. 🚀Through the AI Engineer Challenge and AI Engineer Bootcamp, Hou received the structure, mentorship, and hands-on projects to turn his ideas into sophisticated LLM applications with industrial-grade patterns. 𝐑𝐒𝐕𝐏 𝐭𝐨 𝐰𝐚𝐭𝐜𝐡 𝐡𝐢𝐬 𝐝𝐞𝐦𝐨 𝐚𝐧𝐝 𝐦𝐨𝐫𝐞 https://lnkd.in/gjszCU9Z #AIEngineer #AIMakerspace #ComponentForge #MultiAgent #AdvancedRAG #LLMEvaluation #AIinProduction #FullStackAI #ComponentGeneration #AIBootcamp

7

1 Comment

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

⚡️ Frontier open models run faster on #NVIDIA. New benchmarks show the NVIDIA #Blackwell platform maximizes #inference ROI by optimizing user interactivity and throughput per GPU. ⚙️ OpenAI gpt-oss-120b reaches over 60,000 tokens per second (TPS) per #GPU. ⚙️ AI at Meta Llama-3.3-70b tops 15,000 TPS per GPU. ⚙️ #DeepSeek #AI DeepSeek-R1 achieves over 10,000 TPS per GPU. Our continuous software improvements and extreme hardware-software co-design enable real-world AI inference to balance cost, efficiency, throughput, and responsiveness. 📈 Explore the interactive performance charts: https://bit.ly/3LbWgeU #GPT #DeepLearning #Education #Research

1

🎬 How Crescendo AI Simplifies Enterprise AI with Google Agentspace? At Google Cloud Summit Taipei, Crescendo Lab officially announced that Crescendo AI now uses Google #Agentspace as its core engine. This upgrade brings a powerful boost for businesses ready to activate AI in their operations. With Crescendo AI, companies can instantly transform scattered data into proactive insights across marketing, sales, and customer service. 🔹Builds an enterprise-grade knowledge base by connecting BigQuery and Vertex AI 🔹Eliminates data silos and accelerates data-driven decision-making 🔹Provides real-time, intelligent suggestions to enhance MAAC and CAAC performance 🔹Requires no additional infrastructure, making AI activation fast and effortless ▶️ Watch the highlights: https://lnkd.in/gTx5yFcR #CrescendoAI #GoogleAgentspace #AIFirst #BigQuery #VertexAI #EnterpriseAI

16

Researchers from National Tsing Hua University (NTHU) in Taiwan have developed the smallest quantum computer in the world, using just a single photon to perform complex calculations. Led by Professor Chih-Sung Chuu, the team created Taiwan’s first optical quantum computer. Unlike traditional quantum systems that need bulky, freezing-cold setups, this device runs at room temperature and is small enough to fit on a desk. What makes it special is its ability to store 32 pieces of information in one photon—comparable to turning a 1-seater bike into a 32-seater. This helps solve a big challenge in quantum tech: the difficulty of generating and syncing multiple photons. This breakthrough opens the door to smaller, more powerful quantum machines with uses in AI, cybersecurity, and medicine. It also puts Taiwan on the global map in the race for quantum innovation. #QuantumComputing #TaiwanInnovation #PhotonTech #OpticalQuantumComputer #NextGenAI

13

Introducing the M5 The M5 chip brings significant advancements in performance and efficiency across various applications. 1. Massive GPU / AI performance boost The M5 features a next-generation 10-core GPU architecture, with each core equipped with an integrated Neural Accelerator. This results in over 4× the peak GPU compute performance for AI tasks compared to the M4. 2. Stronger CPU and enhanced Neural Engine With up to a 10-core CPU (6 efficiency cores + 4 performance cores), the M5 offers approximately 15% faster multithreaded performance over the M4. Additionally, the Neural Engine has been upgraded to 16 cores, enhancing performance for on-device AI and intelligent features. 3. Higher memory bandwidth / unified memory improvements The M5 provides around 153 GB/s of unified memory bandwidth, which is approximately 30% more than the M4. This unified memory architecture allows all components (CPU, GPU, Neural Engine) to access a larger memory pool, enabling larger AI models to run entirely on-device. 4. Graphics & rendering enhancements The improvements in the GPU also boost graphics performance, including ray tracing capabilities. Apple claims up to a ~45% uplift compared to the M4 in ray-traced workloads, enhancing visual fidelity, rendering speed, and smoothness in demanding 3D and graphics applications. 5. Deployment in new hardware + energy efficiency / environmental goals The M5 will power new devices, including the 14-inch MacBook Pro, iPad Pro, and upgraded Vision Pro, all of which are available for pre-order.

7

🤖 Deploy OpenAI gpt-oss models on serverless NVIDIA GPUs using NVIDIA NIM microservices. Microsoft and NVIDIA share how to scale open models and unlock new agentic AI workflows through: ✔️ Spinning up gpt-oss for reasoning and tool-calling ✔️ Deploying NVIDIA NIM microservices ✔️ Delivering enterprise AI on Microsoft Azure Container Apps 🎥 Watch the developer session on-demand.

5

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

👉 My Key Takeaways from Chip Huyen's Recent Interview on Lenny's Podcast Chip Huyen is the author of the widely recognized "AI Engineering: Building Applications with Foundation Models". Link to the podcast: https://lnkd.in/dxZ-tFWX 💡 Importance of post-training.  Pre-training gives you raw capabilities (next token prediction on massive data), but post-training is what makes the model actually usable. SFT on high-quality examples + RLHF. Fine-tuning should be your last resort, not first. Most problems can be solved with better prompts, better data, or RAG. 💡 Evals.  You can't improve what you can't measure. Need multiple types: unit tests (does this specific prompt work?), integration tests (does the whole pipeline work?), regression tests (did we break something?), and user feedback loops. The hardest part isn't writing evals; it's maintaining them as your product evolves. 💡AI products.  Reliability and UX matter more than models. Most AI product failures aren't about bad models: they're about reliability (API limits, latency spikes, poor monitoring) and UX (users don't understand how to use it, doesn't fit workflow). Building reliable platforms and talking to users constantly beats chasing SOTA models. Most insights come from watching users, not from benchmarks. 💡How to improve AI-powered apps.  What people think improves apps: staying current on AI news, chasing newest agentic framework, obsessing over vector database choice, constantly evaluating model benchmarks, fine-tuning models. What actually improves apps: talking to users, building reliable platforms, preparing better data, optimizing end-to-end workflows, writing better prompts. Better prompt engineering beats switching models 90% of the time. A well-crafted system prompt, clear instructions, good examples (few-shot), and proper output formatting can transform a mediocre experience into a great one. 💡 Advice for builders.  Start with user problem, not with cool AI technique. Use the simplest solution that works (often that's a good prompt, not a fine-tuned model). Build evals early. Focus on end-to-end experience. Don't fine-tune unless you've exhausted everything else. Don't treat AI as deterministic (it's not, you need to handle variability). Don't ignore data quality (garbage in, garbage out).

1

1 Comment