Kirit Thadaka

At the upcoming GTC, you can learn how system-level innovations are accelerating LLM inference. A good example is this new AWS blog on P-EAGLE, introducing GPU parallel speculative decoding in vLLM to generate and verify multiple tokens in parallel. Why this matters? Inference performance is increasingly a systems problem. Techniques like this improve tokens/sec per GPU, helping reduce latency and cost for production GenAI workloads. Good read for anyone working on LLM serving and inference optimization. Huang Xin Florian Saupe Jaime Campos Salas Benjamin Chislett Zhenghang (Max) Xu Faradawn Yang Omri Almog Jiahong Liu AWS blog: https://lnkd.in/g-GjMgvX⁠�

43

5 Comments

Is the computational ceiling about to break? The "SAT Problem" is the invisible bottleneck of our era. From EDA chip design at Nvidia and Intel to the formal verification of AWS cloud security and the scaling of DeepMind’s next-gen AI—we are all running into the same wall: Exponential Complexity. Current solvers (CDCL/Local Search) are masterpieces of engineering, but they still "stall" when faced with high-treewidth, structured monoliths. At Digital Dynamics AI Inc., we have developed a new framework: tEPIC (Topological Exact Partitioning & Integrated Certification). We have moved past heuristic-based guessing. Our solver, SOSAT, utilizes two novel, peer-reviewed-grade theorems to navigate the logic landscape deterministically: 1️⃣ The Charging Inequality (GO-360-CH): A combinatorial guard that forces any SAT instance to reveal its internal "triage" state. Every sub-component is mathematically boxed into one of three exits: a global fracture (Separator), a governing bottleneck (Core), or a high-density collision (Cycle). 2️⃣ Semantic Splice-or-Pivot (GO-361): A proof-complexity compiler that serializes local Unit Propagation traces into globally verifiable DRAT/RUP certificates. This allows us to "splice" overlapping micro-proofs into a unified solution path in strictly polynomial time. The result? A structural engine that doesn't just "try" to solve a problem—it exhausts the instance's topology until a refutation or a witness is found. No metaphors. No "magic." Just discrete graph accounting. I am now looking to present the SOSAT/tEPIC architecture to the Automated Reasoning and AI Safety teams at Microsoft Research, Google DeepMind, and Amazon Web Services. We are ready to demonstrate performance on industrial-scale "unsolvable" monoliths via a secure, black-box audit. If you are a Principal Engineer or Researcher in Formal Methods or AI Scaling, let’s talk about moving the world past the exponential wall. #SAT #DeepTech #ComputerScience #AI #FormalMethods #DigitalDynamicsAI #PvsNP #Innovation

2

30 Comments

The most impactful and important SDK release Trainiun Neuron team has worked on since inception: Native PyTorch support for Trainium, open source kernel (NKI) compiler and pre-optimized NKI kernel library, Neuron Explorer (IMO, the fastest and most intuitive ML profiler in industry), optimized inference implementation for major MoE and video models (GPT-OSS, Qwen, Pixtral….), complete vLLM v1 support !! All with full support for Trainium3. Using Trainium, whether for research, training , or large scale inference just became more intuitive and simple. https://lnkd.in/gjfhjv4j

626

10 Comments

Compressing LLMs but worried about accuracy? 🎯 EoRA uses eigenspace low-rank approximation to compensate for errors—no retraining needed. A promising direction for scalable, task-adaptive LLMs. New paper from #NVIDIAResearch.

1

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58