Linpeng Lyu

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

An agent has no consistent definition. You can think of an agent as an autonomous or semi-autonomous system that can take actions on behalf of the user in a given environment, state, and make decisions or take actions that can accomplish certain tasks along a given time-frame, either by calling tools, dynamically selecting the next action to take or use a deterministic "workflow" (systems where LLMs and tools are orchestrated through predefined code paths). Using an Agent framework that gives you the ability to built systems in both, a reliable and a dynamic way can accelerate your agent development journey using agent abstractions. This means building Agentic systems that can reliably call tools, store memory (episodic/semantic/procedural), have comprehensive logging and observability, human in the loop workflows and the ability to build various multi-agent patterns flexibly based on your use case. A successful Agentic system in production is usually a combination of both, dynamic and predictable/reliable multi-agent systems. Strands Agents SDK gives you exactly these capabilities by treating each “agent” as a combination of a foundation model plus a suite of tools. You define a prompt and register your tools (decorated functions) in code, then Strands handles reasoning→planning→tool-execution cycles, local testing, and cloud deployment (ECS, Fargate, Lambda, EC2), along with support for all other agent abstractions provided above. Excited to share a hedge-fund analyst multi agent system: This uses the newest Anthropic's Claude 4 Sonnet/Opus that powers the Lead Analyst Agent, routing incoming queries to specialized sub-agents for fundamental, technical, and market analyses. Each specialist is wrapped as a callable tool (using the “agents-as-tools” multi-agent pattern), so the orchestrator never has to implement domain logic itself and can handoff the task to an agent as a tool. For sensitive operations (insider lookups), we utilize a HITL approval step that halts execution until a human grants consent. We also use meta-tooling that enables the Lead Analyst to generate, load, and invoke new custom tools at runtime—whether it’s a portfolio beta calculator or a pricer—without redeployment. Strands also embeds observability (Langfuse) and OpenTelemetry tracing so you can trace reasoning events, tool invocations, errors, and end-to-end workflows in real time. View more information on the code implementation here: https://lnkd.in/gJmwVyGi Code implementation: https://lnkd.in/gzTtJvJq Thanks to 🏄‍♂️ Cagatay Cali for being a reviewer/collaborator on this! Feel free to try it out and reach out with any questions/ideas. #aws #agenticAI #strands #agents #generativeAI

60

6 Comments

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

🚀 New Update: Dynamic Image Transformation for Amazon CloudFront We’re excited to roll out the latest version of Dynamic Image Transformation for Amazon CloudFront, making it easier than ever for teams to deliver fast, optimized, device-aware images through Amazon CloudFront's global CDN. This release introduces: ✨ Automatic image optimizations, tailoring image format, compression, and size to each user's device ✨ Support for delivering large, high-resolution images without compromising performance ✨ Support for external image origins, including popular CMS platforms ✨ An intuitive configuration experience to get up and running faster If you’re at AWS re:Invent this week, stop by the Edge & Networking area on the Expo floor to see a live demo and walkthrough of DIT. You can also explore other new CloudFront and Edge capabilities launching this week! https://lnkd.in/guc3pqhm

18

Introducing the M5 The M5 chip brings significant advancements in performance and efficiency across various applications. 1. Massive GPU / AI performance boost The M5 features a next-generation 10-core GPU architecture, with each core equipped with an integrated Neural Accelerator. This results in over 4× the peak GPU compute performance for AI tasks compared to the M4. 2. Stronger CPU and enhanced Neural Engine With up to a 10-core CPU (6 efficiency cores + 4 performance cores), the M5 offers approximately 15% faster multithreaded performance over the M4. Additionally, the Neural Engine has been upgraded to 16 cores, enhancing performance for on-device AI and intelligent features. 3. Higher memory bandwidth / unified memory improvements The M5 provides around 153 GB/s of unified memory bandwidth, which is approximately 30% more than the M4. This unified memory architecture allows all components (CPU, GPU, Neural Engine) to access a larger memory pool, enabling larger AI models to run entirely on-device. 4. Graphics & rendering enhancements The improvements in the GPU also boost graphics performance, including ray tracing capabilities. Apple claims up to a ~45% uplift compared to the M4 in ray-traced workloads, enhancing visual fidelity, rendering speed, and smoothness in demanding 3D and graphics applications. 5. Deployment in new hardware + energy efficiency / environmental goals The M5 will power new devices, including the 14-inch MacBook Pro, iPad Pro, and upgraded Vision Pro, all of which are available for pre-order.

7

Closing the week by celebrating another great customer story — this time from Kakao Corp In this latest blog, Kakao shares how they overcame infrastructure scaling bottlenecks by moving to JAX + TPUs, enabling faster training of their Kanana model family. Their goal was to build more efficient, adaptable AI systems at production scale. With the right hardware + software combination — JAX for flexibility and TPUs for performance — customers can unlock breakthrough speed, efficiency, and scale in their AI workloads. Excited to help bring these innovations to organizations worldwide. https://lnkd.in/gu8UAk57 #AIleadership #Jax #TPUs #ScalableAI

10

this is super important because TPUs have definitely advantages for cost and performance over other accelators but were much more difficult to use. no longer the case. "vLLM TPU is now powered by tpu-inference, an expressive and powerful new hardware plugin unifying JAX and PyTorch under a single lowering path. It is not only faster than the previous generation of vLLM TPU, but also offers broader model coverage and feature support." https://lnkd.in/diDKzYbS

35

1 Comment

RAG is becoming one of the most critical techniques for improving LLM accuracy — not just a research concept, but a practical system design pattern. It's exciting to see Googlers sharing real-world implementation experience and helping more developers build reliable AI

3