Gang Hua

Barbarians at the Gate (https://lnkd.in/gkEBX79s) is a very interesting new paper from some great folks at Berkeley, with some exciting results showing the potential for AI in systems research. But I think the authors aren't quite asking the hardest problem about where this takes systems as a field. I wrote a new blog post about it: https://lnkd.in/gxS3s4v4 The core point of the paper is that AI can be effective at automatically driving the 'number go up' part of systems research. Given an evaluator, AI can automate progress on implementation. I believe this is 100% correct. The paper also talks about how more focus will have to go into developing evaluators. Again, I 100% agree, but think they're understating their case. Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade. Which leaves researchers with the hard problem of choosing which problems to go after. That's never been easy in academic systems, and is only going to become more of a challenge. But the best systems research is driven by choosing problems well! This vision for AI could lead to lots more of the least interesting systems research. The "number go up" kind. Which could make it harder than ever to identify the truly innovative and visionary work. We're already seeing that elsewhere in science. So, mostly, I think the paper is right, but not yet asking the biggest question: what should systems research be?

276

12 Comments

A 12-month professional, voluntary collaboration has finally borne fruit. This morning, I was honored to see the University of Washington News (https://lnkd.in/gRRpDqjA) and the UW Milgard School of Business (https://lnkd.in/gKJ_pwbm) highlight our Responsible GenAI Framework. Why do we care about responsible generative AI? Because GenAI systems are no longer experimental curiosities. They shape decisions, influence behavior, generate content at scale, and increasingly affect trust, fairness, privacy, and accountability in organizations and society. Without intentional design and governance, these systems can amplify bias, create opaque risks, and erode confidence rather than enable innovation. This framework is about making GenAI useful, ethical, and sustainable at the same time. It emphasizes responsibility not as a constraint, but as an enabler of long-term value, credibility, and impact. I am grateful to the collaborators, reviewers, and institutions who supported this work, and excited to see responsible GenAI move from aspiration to practice. To learn more: - Read the RGAF Framework (PDF): RGAF Framework https://lnkd.in/gTaxnndV #GenAI #ResponsibleGenAI International Society of Service Innovation Professionals (ISSIP) Milgard CBA

23

1 Comment

I'm excited to share three research papers from our talented team at Together AI that are advancing the frontiers of efficient and effective AI systems: ## 🚀 Accelerating Large Model Inference Our researchers have developed "Ladder-residual," a novel parallelism-aware architecture that significantly accelerates large model inference through communication overlapping. This innovation addresses one of the key bottlenecks in deploying large language models at scale, making AI systems faster and more cost-effective. 📄 Read more: [Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping](https://lnkd.in/g825uRRz) ## 🤝 Enhancing Model Alignment Through Collective Intelligence By leveraging the collective intelligence of open-source models, we've developed new methods to improve AI alignment without the computational expense of traditional alignment techniques. This research demonstrates how cooperation between diverse models can lead to superior performance and better alignment with human values. 📄 Read more: [Improving Model Alignment Through Collective Intelligence of Open-Source Models](https://lnkd.in/g9FZ83P8) ## 📱 Optimizing On-device and Cloud Collaboration Our collaboration with researchers from Stanford introduces cost-efficient methods for on-device and cloud language model cooperation. This work enables more powerful AI capabilities on edge devices while intelligently leveraging cloud resources, resulting in better user experiences and reduced costs. 📄 Read more: [Cost-efficient Collaboration between On-device and Cloud Language Models](https://lnkd.in/ggM6v3R3) These papers represent our commitment to developing AI that is more efficient, aligned, and accessible. I'd love to hear your thoughts on these research directions or discuss potential collaborations! #TogetherAI #MachineLearning #AI #ResearchAndDevelopment #LLM #EdgeComputing #AIAlignment

50

Agentic AI breaks static governance This new survey paper on Agentic Reasoning for Large Language Models brings together researchers from UIUC, Meta Facebook, Amazon, Google DeepMind, UC San Diego, and Yale. The core shift is simple and consequential: We are moving from models that reason to systems that plan, act, adapt, remember, and collaborate over time. The paper frames agentic AI across three layers: • Foundational agents that plan, search, and use tools • Self-evolving agents that adapt via feedback and memory • Multi-agent systems that coordinate roles and objectives Why this matters for AI governance: Once reasoning is coupled with action and memory, static Responsible AI controls stop working. Pre-deployment reviews, model cards, and one-off risk assessments cannot govern systems that change behaviour at inference time. The authors explicitly flag governance of agentic reasoning as an open problem. That is the real takeaway. If governance does not operate at deployment, inference, and autonomy thresholds, it is not governing the system in motion. I recommend reading the paper in full. It is one of the most comprehensive mappings of agentic systems to date. Paper: https://lnkd.in/gtZsfQYB Question How are you thinking about control and accountability once reasoning, memory, and action are coupled in live agentic systems? #AIGovernance #ResponsibleAI #AgenticAI #AISafety #EnterpriseAI #AIControl

38

13 Comments

In NVIDIA’s Q2 earnings this week, Jensen Huang highlighted physical AI—alongside reasoning AI—as a key growth driver. Analysts echoed this, noting breakthroughs in robotics and autonomous systems as the next stage of AI adoption. It’s a welcome acknowledgment that AI’s impact won’t be confined to digital spaces. The real opportunity lies in the physical world—transportation, logistics, energy, construction—where safety, efficiency, and sustainability gains can be profound. That’s the frontier many of us are working on: bringing AI to the systems and workers that keep the real economy moving.

72

Excited to share that we are organizing a workshop at Amazon Research Day 2026 in Palo Alto on April 30 on next generation code development with collaborative AI agents, with Shweta Garg Jun (Luke) Huan Narayanan Sadagopan. We will discuss agent architectures, human AI interaction patterns that preserve agency, and how to ensure trust and reliability at scale, with speakers including Graham Neubig from Carnegie Mellon University, Eddie Aftandilian from GitHub Next, and Varun Kumar from Kiro Amazon Web Services (AWS). If you are attending Amazon Research Day and care about the future of AI for code, I would love to connect.

51

3 Comments

Since the Dijkstra theoretical improvement is going viral again, I want to point out how irrelevant the improvement is for PRACTICAL applications. Imagine your program uses Dijkstra to find shortest paths for graphs with 1M nodes. Say your application grows and now the graphs are twice as big: 2M nodes (something that doesn't happen too often for *practical* applications). All that means for Dijkstra with a plain min-heap is that the heap will have one extra level (~21 levels instead of ~20). Thus, heap operations will have to do one more iteration: about ~10 LoC of basic arithmetic and array reads/writes. That's how insignificant the impact of the O(log n) factor is to the scaling of Dijkstra. Now, the new result doesn't eliminate a log factor. It elimintes the CUBIC ROOT of a log factor. Totally insignificant. In exchange, it brings in a set of non-trivial steps and techniques, a lot of additional bookkeeping, recursive partitioning, etc.

5

1 Comment

New work on Rethinking Thinking Tokens: LLMs as Improvement Operators: https://lnkd.in/errhNuuz Reasoning training encourages LLMs to produce long chains of thought (CoT), improving accuracy via self-checking but increasing context length, compute cost, and latency. This work studies whether frontier models can achieve better trade-offs, higher accuracy with lower cost. The paper develops a simple yet effective Parallel-Distill-Refine (PDR) procedure: Generate diverse drafts in parallel, Distill them into a compact textual workspace, and Refine conditioned on this workspace. This decouples context length from total token count, allowing control over compute via parallelism. PDR yields higher accuracy than long CoT at lower latency. Training an 8B model with RL to align with PDR further shifts the Pareto frontier. On math benchmarks, PDR achieves +11% (AIME 2024) and +9% (AIME 2025) over single-pass baselines. With Lovish Madaan, Aniket Didolkar, Suchin Gururangan, John Quan, Ruan Silva, Manzil Zaheer, Sanjeev Arora, and Anirudh Goyal.

78

1 Comment

I came across the Agentic Context Engineering paper by Stanford, SambaNova and UC Berkeley earlier this week and it reframes how we think about adaptation in language models. Instead of fine-tuning model weights, ACE evolves the context itself - transforming prompts and agent memories into living playbooks that self-improve over time. ACE introduces a modular generation, reflection and curation loop that enables LLMs to: · Prevent context collapse and overcome brevity bias · Achieve scalable self-improvement through grow-and-refine cycles · Learn effectively from natural execution feedback, without labeled supervision It's exciting to learn how ACE expands the agentic design space beyond autonomous reasoning, positioning context as a first-class learning substrate. As learning shifts from model-centric to context-centric paradigms, ACE paves the way for self-improving agentic architectures. Paper: https://lnkd.in/g4QXJcAb

21

3 Comments

This section of Jensen Huang's blog post today on NVIDIA clarified for me the push in AI investment we're seeing right now. "Around the world, we are seeing chip factories, computer assembly plants, and AI factories being constructed at unprecedented scale. This is becoming the largest infrastructure buildout in human history. "The labor required to support this buildout is enormous. AI factories need electricians, plumbers, pipefitters, steelworkers, network technicians, installers, and operators. "These are skilled, well-paid jobs, and they are in short supply. You do not need a PhD in computer science to participate in this transformation. "At the same time, AI is driving productivity across the knowledge economy. Consider radiology. AI now assists with reading scans, but demand for radiologists continues to grow. That is not a paradox. "A radiologist’s purpose is to care for patients. Reading scans is one task along the way. When AI takes on more of the routine work, radiologists can focus on judgment, communication, and care. Hospitals become more productive. They serve more patients. They hire more people. "Productivity creates capacity. Capacity creates growth." Take five minutes and read the entire post. https://lnkd.in/g5-765Tb

4