Kari Ann Briski

The most impactful and important SDK release Trainiun Neuron team has worked on since inception: Native PyTorch support for Trainium, open source kernel (NKI) compiler and pre-optimized NKI kernel library, Neuron Explorer (IMO, the fastest and most intuitive ML profiler in industry), optimized inference implementation for major MoE and video models (GPT-OSS, Qwen, Pixtral….), complete vLLM v1 support !! All with full support for Trainium3. Using Trainium, whether for research, training , or large scale inference just became more intuitive and simple. https://lnkd.in/gjfhjv4j

626

10 Comments

On-device AI becomes powerful only when it works reliably on real hardware. In this Melange Walkthrough, Yeonseok Kim and Sun-Q Kim break down how the full on-device pipeline works, from model selection and quantization to real-device benchmarking and SDK deployment. Select. Benchmark. Deploy. This is how production-ready on-device AI should operate. Watch the full walkthrough below. #OnDeviceAI #EdgeAI #MobileAI #AIInfrastructure #ZETIC #Melange

26

Excited about the recent StorageGRID 12.0 release! The update brings a host of new features tailored for Object Storage in AI/ML applications. Dive into my latest blog to explore how StorageGRID 12.0 streamlines the management of extensive AI content repositories. #StorageGRID #AI #ML #ObjectStorage https://lnkd.in/esnbpfMH

60

Are mixture of expert (MOE) models the future for professionals using local LLMs? Buying enough VRAM for 30-100B MOEs on a Strix Halo mini PC is quite affordable, and the inference speed for the 30B MOE is just awesome. Filmed on Ryzen AI MAX 395+ with 128GB RAM, Vulkan backend, using Lemonade: https://lnkd.in/eDZW7ZZn Great work by Daniel Holanda Noronha sprucing up the interface! Ramakrishnan Sivakumar Victoria Godsoe Tomasz Iniewicz Kalin Ovtcharov

58

2 Comments

Incredibly proud of our team. Today we introduced Maia 200, our next-generation AI accelerator now powering production workloads in Microsoft’s U.S. datacenter regions delivering faster, more efficient AI inference at scale. Learn more in our blog: aka.ms/Maia200blog.

20

After 9 years at NVIDIA, James Wang left and joined Cerebras. In this Big Chip Club episode, James Wang exposes the early days of NVIDIA, working with Jensen himself, and taking a bet on the next generation of compute.

158

7 Comments

Live from AI Infra Summit 2025 — LightningAI is powering the future of GenAI inference. This morning, Pliops unveiled breakthrough demos showing how our GenAI-native memory stack delivers real-time performance across NVIDIA and AMD GPUs. From LLMs to RAG and vLLM, LightningAI is enabling hyperscalers and enterprises to scale with confidence. 🚀 What’s live at booth #726: • The Viking Enterprise Solutions LightningAI server — purpose-built for multi-GPU inference • Collaboration with Tensormesh to accelerate vLLM deployment (the team behind LMCache Lab) • Unified compute + memory for turnkey GenAI infrastructure Come see how LightningAI is transforming GenAI workloads—fast, efficient, and ready for scale.

4

New jobs. New business models. New markets. Our CEO, Jensen Huang, joined the All-In Podcast to lay out where AI is taking us and what comes next. Read the Forbes breakdown: https://nvda.ws/3UBgUXl

3

1 Comment

Kicked off AMD DevDay 2025 with an inspiring & insightful fireside chat Ion Stoica, UC Berkeley Professor, Director of the Sky Computing Lab, co-creator of Spark, Ray, and vLLM… cofounder of LMArena, Anyscale, Databricks, Conviva… Fellow at both IEEE & ACM! Key takeaways: • LLMs are not so quadratic anymore. With all the tricks, they effectively scale quasi-linearly • did we run out of data? Synthetic data & CoT still boost intelligence (LLMs reached IMO gold level) • LLMs can generate optimized GPU kernels guided by simulated and realtime execution feedback #AMDDevDay #AI #AMD

160

1 Comment