Lu Wang

** Big update on the roadmap for Meta's MTIA roadmap (our Meta-designed AI chips) ** We just shared today a big update on the MTIA (Meta Training & Inference Accelerator) roadmap. The main takeaway is we're accelerating the whole set of programs to get rapid iterations on chips (given how fast the model needs are evolving). 👉 Overall blog is here: https://lnkd.in/gaHuxUfg For the networking aficionados, you can see a whole lot of networking that we've been working on inside MTIA - we're excited to finally be able talk about it! 👉 The scale-up and scale-out networks we've already talked about. It's sort of buried in there, but we are already running distributed training clusters with MTIA 300 for core ranking & recommendation (even as we focus future generations on Inference). 👉 The communication library (HCCL) that is purpose-built to take advantage of networking-specific capabilities we've built into the chips to offload the main compute. 👉 The in-built support for fused compute/communication kernels to minimize latency (key for both training and inference workloads). Super-proud of the whole MTIA team!

143

4 Comments

Inference at scale is evolving. The future is composable systems where offline batch, online batch (near-realtime), and realtime traffic can all be served from a single, unified inference endpoint for most use-cases. This simplifies your architecture, reduces operational overhead, and increases throughput. Our new tutorial series will show you how to build it step by sep. In the first installment, Erik Saarenvirta walks you through creating a scalable image classification system on GKE that can be adapted to a variety of use-cases. Leave us feedback in comments or ask questions we are happy to answer. Kent Hua Ishmeet Mehta Erik Saarenvirta https://lnkd.in/dVNiUhwE

16

What are the most underrated Databricks capabilities? Watch my conversation with Shiv Trisal (Global Industrials GTM @ Databricks) on the topic below. For the full, practical, no-fluff discussion on real-world data + AI: https://lnkd.in/gMPsFHbY

89

AstroBee generates ontologies...but what is an ontology, and if they're important, why don't you already have one? I've written a short piece on Kimball star schemas, ontologies, and how each relate to Bill Inmon's original approach to data warehousing in the early 90s. Dig in to find an overview of a fascinating technological debate and the re-emergence of an approach to data modeling that's been sleeping for 20 years.

18

3 Comments

🧠 While LLMs continue to grab the spotlight, Small Language Models (SLMs) are quietly reshaping how we think about efficiency in AI. I recently explored how SLMs can run entirely on CPUs, unlocking new possibilities for low-cost, privacy-friendly, and scalable inference, ideal for teams experimenting with AI or deploying it at the edge. In my latest piece for Towards AI publication, I cover: 💡 Why CPUs are back in the AI conversation: smarter models, quantization, and optimized runtimes like llama.cpp ⚙️ SLMs vs LLMs: how a hybrid strategy can balance capability and cost 🧩 CPU Inference Tech Stack: GGUF, Ollama, and other tools making CPU inference practical 🧠 Hands-on section: building a translation app using llama.cpp, Streamlit, and AWS EC2, fully CPU-based The best part? You can try it locally - no GPU required. It’s been exciting to see the response so far, the article has already crossed 200+ likes on Medium! 📖 Read my full blog post here: 👉 https://lnkd.in/gby-EBmJ #SLM #llamacpp #cpu #BeginnerFriendly #AWS

7

The goal of Machine Learning University (MLU) is to "enable Amazonians to identify ML and generative AI problems, have productive conversations about ML and generative AI projects, and get ML and generative AI models into production faster." I highly recommend the ML for Leaders Generative AI course to my fellow Amazonians.

50

5 Comments