Xi Lin

🌟 XGBoost 3.0 now has new capabilities for GPUs: ✅ Train up to 1 TB of data on a single Grace Hopper GPU: Up to 8x faster than CPU with External-Memory Quantile DMatrix. ✅ Faster and lighter: GPU hist/approx methods an additional ~2x speed up and reduced memory use. ✅ Full feature support: External memory now supports categorical features, all objectives, and SHAP. ✅ Distributed mode: Experimental support for out-of-core training across a cluster. Technical deep dive ➡️ https://bit.ly/3UWdmz2

1

📝 Learn how to use #GenAI-Perf to benchmark the Meta Llama 3 model when deployed with NVIDIA NIM, including popular metrics and parameters. Technical deep dive with a step-by-step guide ➡️ https://bit.ly/42Sm3zt This is part 2 in our #LLM benchmarking series.

2

📣 Blackwell sets a new inference speed world record — A single NVIDIA DGX B200 server with eight #NVIDIABlackwell GPUs can generate over 1,000 tokens per second (TPS) per user on the Llama 4 Maverick model, the largest and most powerful model in the AI at Meta Llama 4 family. ⚡⏱️ Additionally, a system with eight Blackwell GPUs can also deliver up to 72,000 tokens/second in a maximum throughput scenario. 🏆 Blackwell is the first platform to achieve this model performance, demonstrating how it delivers the best combination of throughput, output speed and accuracy for LLM token generation. 🏎️🏁 See our tech blog for details.

3

🧠 Agent Memory that Scales: Short-Term vs Long-Term (Semantic / Episodic / Procedural) • Reality check: LLMs are stateless. Any “memory” is an engineering layer you add. • Memory model: • 🗣️ Short-term → session context, tool outputs, scratchpad. • 🧾 Long-term → • Semantic: facts/preferences (user profile, org policies). • Episodic: past tasks/interactions with outcomes. • Procedural: how the agent executes routines (playbooks/skills). • Why it matters: Improves personalization, continuity, and continual learning without retraining. • Implementation sketch (e.g., Zep): • Graph store for entities/events; embed + link conversations, docs, tool runs. • Memory policies: TTL, PII redaction, summarization at thresholds (token/age). • Quality gates: Retrieval eval (hit rate, precision@k), hallucination checks vs stored facts. • Pro tips: Separate read/write paths, version memories, and surface citations in responses to build user trust. #dailydoseofds #datascience #machinelearning #MLOps #AIagents #Memory #KnowledgeGraphs #ContinualLearning

3

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

Introducing the M5 The M5 chip brings significant advancements in performance and efficiency across various applications. 1. Massive GPU / AI performance boost The M5 features a next-generation 10-core GPU architecture, with each core equipped with an integrated Neural Accelerator. This results in over 4× the peak GPU compute performance for AI tasks compared to the M4. 2. Stronger CPU and enhanced Neural Engine With up to a 10-core CPU (6 efficiency cores + 4 performance cores), the M5 offers approximately 15% faster multithreaded performance over the M4. Additionally, the Neural Engine has been upgraded to 16 cores, enhancing performance for on-device AI and intelligent features. 3. Higher memory bandwidth / unified memory improvements The M5 provides around 153 GB/s of unified memory bandwidth, which is approximately 30% more than the M4. This unified memory architecture allows all components (CPU, GPU, Neural Engine) to access a larger memory pool, enabling larger AI models to run entirely on-device. 4. Graphics & rendering enhancements The improvements in the GPU also boost graphics performance, including ray tracing capabilities. Apple claims up to a ~45% uplift compared to the M4 in ray-traced workloads, enhancing visual fidelity, rendering speed, and smoothness in demanding 3D and graphics applications. 5. Deployment in new hardware + energy efficiency / environmental goals The M5 will power new devices, including the 14-inch MacBook Pro, iPad Pro, and upgraded Vision Pro, all of which are available for pre-order.

7

Here comes the Voice Agents with ExecuTorch! Our latest PyTorch blog on building voice agents with ExecuTorch is out. As open-source voice models proliferate (ASR, diarization, translation etc), deploying them natively across devices has been fragmented. ExecuTorch aims to close that gap by letting developers export directly from PyTorch and run efficiently across CPU/GPU/NPU on Linux, macOS, Windows, Android, and iOS. In the post, we share reference implementations and apps for models including Voxtral Realtime, Parakeet TDT, Sortformer, Whisper, and Silero VAD, plus an adoption story from LM Studio shipping transcription powered by ExecuTorch. If you’re building voice features that need low-latency, offline, edge-native inference, let us know what you’re working on and don't hesitate to reach out. More details here: https://lnkd.in/gNEVuxJy

117

From Kernel to User Mode: The Ring 3 🦀Lets Run GTA 5 in JC-OS 😎 #RustLang #OsDev #SystemsProgramming #X86_64 #JCOS #TheRustEagle

2

📝 Learn how to use #GenAI-Perf to benchmark the Meta Llama 3 model when deployed with NVIDIA NIM, including popular metrics and parameters. Technical deep dive with a step-by-step guide ➡️ https://bit.ly/3ShbUGr This is part 2 in our #LLM benchmarking series.

2