Kazi Zaman

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58

On-device AI becomes powerful only when it works reliably on real hardware. In this Melange Walkthrough, Yeonseok Kim and Sun-Q Kim break down how the full on-device pipeline works, from model selection and quantization to real-device benchmarking and SDK deployment. Select. Benchmark. Deploy. This is how production-ready on-device AI should operate. Watch the full walkthrough below. #OnDeviceAI #EdgeAI #MobileAI #AIInfrastructure #ZETIC #Melange

26

Andrew Ng reported from Davos, that AI deployment for transformative impact will require workflow redesign. It is "theory of constraints" all over again (remember Eli Goldratt's "The Goal"?), we have to understand that real business benefits come only from working across silos, from analyzing flow bottlenecks, from understanding that optimizing non-bottlenecks creates waste. As Ng noted, letting a thousand (AI) flowers bloom has failed to lead to significant payoffs. Obviously! We in the silicon industry are no strangers to this. Workflows have been stagnant for decades. Technologies for high level specification, high level synthesis, and formal verification are not yet integrated into the full silicon development flow. They are often done piece-meal, and enthusiasts' pilots are dropped at the hint of a failure or delay. People say they want formal verification in their flow, but want it yesterday and do not have the time to invest in training engineers for its rigorous usage. Ashish Darbari, responding to a question on "if formal verification is so powerful then how come it is not widely used", noted that the problem is in the name - the actual task to be done is formal analysis while the bug detection is only a corollary, but the industry puts the cart before the horse. Based on my experience with industrial practice of high level specifications and formal verification, I tend to agree with his succinct comment. Jan Rabaey, in a DAC keynote speech, said that someday he expects to see a comic book series entitled "adventures in formal verification". AI deployment faces the same problem, if not actively worked on by business leaders. To be fair, deployments of all disruptive technologies face this mental barrier, we are so used to thinking in terms of the old flow (using the old technology) that thinking of a new flow literally means us having to begin again from first principles - few leaders have the courage and the time to do that. Goldratt mentioned that he himself had faced the same mindset problem when deploying the early ERP solutions and was surprised why he wasn't getting the expected large benefits, and he said it took him many years before he could identify the core problem and define solutions for it, which he named (rather academically) as "theory of constraints". AI leaders would do well to rework the conventional workflows in a holistic manner in order to get the real benefits from this new technology. As Henry Ford had said, "If you always do what you've always done, you'll always get what you've always got".

21

The real engineering challenge isn’t squeezing more capability out of LLMs. It’s figuring out how to actually run them safely. When an agent can write and execute code, you’re not dealing with AI-powered software anymore. You’re dealing with autonomous software running inside your infrastructure. Most stacks were not built for that. https://lnkd.in/gEScgXdR

7

Engineering teams are turning AI into real results, not just experiments. Last month at the ELC Annual in SF, Redis Principal Field Engineer Josh R. joined Vinay Perneti from Augment Code, Anand K Sankaran from Workday, and Forrest Thomas from Garner Health for a fireside chat on where AI is truly adding value. Hear how they drove adoption across their teams, improved workflows, and made AI deliver real results: https://lnkd.in/gDAjZsDF

23

Great article by Ashpreet B. on adaptive AI agents and continuous learning AI. Highly recommended reading for agentic engineering. https://lnkd.in/gZQknWrn

16

MDPI, Eldar Sultanow, Dr. Mark Tehrani, Siddhant Dutta and Muhammad Shahbaz Khan, “Defining Quantum Agents: Formal Foundations, Architectures, and NISQ-Era Prototypes” This work introduces a formal definition of quantum agents and outlines architectures that integrate quantum computing with agent-based systems. As concrete proof-of-concept implementations, the authors develop and evaluate three quantum agent prototypes: (i) a Grover-based decision agent for quantum search-driven action selection, (ii) a variational quantum reinforcement learning agent for adaptive policy learning in a multi-armed bandit setting, and (iii) an adaptive quantum image encryption agent that autonomously selects encryption strategies based on entropy-driven feedback. These prototypes demonstrate practical realizations of quantum agency in decision-making, learning, and security contexts under NISQ-era constraints. CONCURRENCE  

23

2 Comments