Boyu Zhang

After the release of iOS 26, I feel confident that AI is not replacing senior engineers any time soon. The progress of LLMs has been remarkable. Scaling compute and data has unlocked surprising capability. There are many well understood tasks that models can now handle, and that will continue to reshape the jobs market. But senior engineers spend most of their time on work that is not well understood. They operate in ambiguity. They make sense of new platforms, new APIs, new constraints. They combine broad experience with intuition to navigate the unknown. Most importantly, their context window grows over years when focused. Over the past few months I used ChatGPT 5 and Claude Sonnet 4.5 inside Xcode while updating Land of Livia to iOS 26. Both often sent me down dead ends with the confidence of a sociopath. Edge cases, platform inconsistencies, silent failures. The models could not reason about a new system based on limited training data. It’s not a matter of scaling data centers. That kind of reasoning requires approaches beyond LLMs. Until then, the frontier still belongs to humans.

42

7 Comments

Super exciting to see reports this morning that Waymo is in advanced discussions to raise $10–15B at a valuation near or above $100B, with Alphabet expected to anchor the round. That would be a major step up from Waymo’s prior ~$45B valuation in 2024 and a clear step change in investor confidence. Notably, Waymo has not raised this year, despite the broader AI funding frenzy. The size and structure of the round also make this feel like a step toward an eventual spinout or IPO. This matters because capital supercharges growth. Waymo is past the “does this work?” phase and firmly in “how fast can we scale?” mode. The company has already logged ~14M rides this year and expects to reach ~1M rides per week by late 2026 as it expands into markets like Miami, Dallas, and Philadelphia, including expanding onto freeways. At the same time, competition and consumer expectations are accelerating. In markets where Waymo operates, fully driverless rides are becoming normal rather than novel. Once people can reliably hail a car with no one in the driver’s seat, tolerance for “pilot programs” and distant timelines drops quickly. The race is no longer about whether robotaxis work. It is about who can scale them first, safely, and everywhere — a win for consumers who get cheaper, safer, and more reliable rides.

25

Waymo collected ~20M miles over 15 years before launching L4 in SF. That’s years of data collection, billions in capital, and a scale most companies simply can’t afford. And in many industries — especially defense — you don’t even have the luxury of commercial-scale deployment in the first place. The reality? Autonomy doesn’t fail because of models. It fails because of missing edge cases, weak coverage, and slow data iteration cycles. That’s exactly the problem we’re solving at DiffuseDrive. In this Forbes piece, Sabbir Rangwala breaks this topic down. Synthetic data isn’t about replacing reality. It’s about completing it. https://lnkd.in/dqNSA4t7 Roland PinterJordan KretchmerOutlander VCPresto Tech Horizonse2vc #PhysicalAI #Autonomy #DefenseTech #SyntheticData #Robotics #GenerativeAI

91

2 Comments

Most people evaluate LLMs by just benchmarks. But in production, the real question is- how well do they perform? When you’re running inference at scale, these are the 3 performance metrics that matter most: 1️⃣ Latency How fast does the model respond after receiving a prompt? There are two kinds to care about: → First-token latency: Time to start generating a response → End-to-end latency: Time to generate the full response Latency directly impacts UX for chat, speed for agentic workflows, and runtime cost for batch jobs. Even small delays add up fast at scale. 2️⃣ Context Window How much information can the model remember- both from the prompt and prior turns? This affects long-form summarization, RAG, and agent memory. Models range from: → GPT-3.5 / LLaMA 2: 4k–8k tokens → GPT-4 / Claude 2: 32k–200k tokens → GPT-OSS-120B: 131k tokens Larger context enables richer workflows but comes with tradeoffs: slower inference and higher compute cost. Use compression techniques like attention sink or sliding windows to get more out of your context window. 3️⃣ Throughput How many tokens or requests can the model handle per second? This is key when you’re serving thousands of requests or processing large document batches. Higher throughput = faster completion and lower cost. How to optimize based on your use case: → Real-time chat or tool use → prioritize low latency → Long documents or RAG → prioritize large context window → Agentic workflows → find a balance between latency and context → Async or high-volume processing → prioritize high throughput My 2 cents 🤌 → Choose in-region, lightweight models for lower latency → Use 32k+ context models only when necessary → Mix long-context models with fast first-token latency for agents → Optimize batch size and decoding strategy to maximize throughput Don’t just pick a model based on benchmarks. Pick the right tradeoffs for your workload. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

357

61 Comments

Building trust in LLMs starts with rigorous evaluation, not intuition. Great breakdown on why structured, evidence-based evals are becoming essential for production-grade models.

14

3 Comments

Waymo’s recent robotaxi incidents surface one of the most important and least discussed questions in AI product design: What should a system do when it doesn’t know what to do? Waymo’s answer is refreshingly honest, i.e. stop. Its vehicles are explicitly trained to halt when uncertainty rises (power outages, broken traffic lights, unexpected human behavior). This isn’t a model failure. It’s a product decision. When the safest action is inaction, the system chooses restraint over guesswork. At the level of one car, this is rational. At the level of ten cars, still sensible. At the level of hundreds, it becomes a system-wide failure. Waymo optimizes for local correctness, i.e. when unsure, do no harm. But without orchestration above the vehicle, local safety becomes global brittleness. Correct behavior in isolation can cascade into city-scale paralysis. The real lesson: building AI products isn’t about maximizing intelligence. It’s about designing systems that behave predictably when intelligence runs out and understanding how those behaviors compound at scale. Contrast this with China’s Apollo Go (Baidu) and Tesla and you will see three different strategies with different trade-offs. Baidu reduces ambiguity upstream with an “infrastructure-first” strategy. They’re not just building autonomous cars, they’re building V2X cities where roads, signals, and networks talk to vehicles. When things break, the city steps in. Fleets are centrally directed into predefined “safe harbor” states. The result is system-level resilience albeit at the cost of heavy coordination, regulation, and long-term capital alignment. Tesla takes the opposite bet. It assumes ambiguity will reach the vehicle and designs for it. No smart roads. Vision-only autonomy with end-to-end neural networks learning from millions of edge cases. Tesla isn’t optimizing for certainty in the moment, but for learning at scale over time. The product takeaway is clear: robust AI systems can’t rely on a single philosophy. The most resilient products layer their defenses: + Local safety behaviors that fail predictably + System-level coordination that prevents cascading failure + Continuous learning loops that improve with scale In AI, uncertainty isn’t the edge case, it’s the design constraint.

15

3 Comments

Despite the hype and anecdotes circling around, many companies now fall into one of two groups: 1. Engineers want to use LLMs, but can’t. Budgets. Security reviews. Procurement. IT restrictions. 2. Engineers can use LLMs, but don’t. Bad early experiences. Skepticism. No time to experiment.

2

4 Comments

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9