Bo Zhao

OSS Bench for Agentic Coding.. in steps Cline-Bench: "A task that frontier LLMs struggle to complete are ideal candidates for cline-bench. If you find yourself having to intervene, or write code manually because cline was not able to complete the job - your task captures the failure boundary of state-of-the-art models and is precisely what cline-bench seeks to formalize." https://lnkd.in/gHcGYxiZ

6

Among other things, my team has been focusing on AI research for agents. We just blogged about a recent project: the Test-Time Diffusion Deep Researcher (TTD-DR). This framework uses a Deep Research agent to draft and revise its own drafts using high-quality retrieved information. This approach achieves new state-of-the-art results in writing long-form research reports and completing complex reasoning tasks. The work achieves better results than any other deep research agent we compared to. It also shows that self-evaluation and self-improvement are must-have components of a successful agent. https://lnkd.in/gBZpVMQy

297

8 Comments

🧠 While LLMs continue to grab the spotlight, Small Language Models (SLMs) are quietly reshaping how we think about efficiency in AI. I recently explored how SLMs can run entirely on CPUs, unlocking new possibilities for low-cost, privacy-friendly, and scalable inference, ideal for teams experimenting with AI or deploying it at the edge. In my latest piece for Towards AI publication, I cover: 💡 Why CPUs are back in the AI conversation: smarter models, quantization, and optimized runtimes like llama.cpp ⚙️ SLMs vs LLMs: how a hybrid strategy can balance capability and cost 🧩 CPU Inference Tech Stack: GGUF, Ollama, and other tools making CPU inference practical 🧠 Hands-on section: building a translation app using llama.cpp, Streamlit, and AWS EC2, fully CPU-based The best part? You can try it locally - no GPU required. It’s been exciting to see the response so far, the article has already crossed 200+ likes on Medium! 📖 Read my full blog post here: 👉 https://lnkd.in/gby-EBmJ #SLM #llamacpp #cpu #BeginnerFriendly #AWS

7

In the era of LLMs, delivering reliable, robust, and trustworthy conversational systems means more than picking a model — it means embedding rigorous evaluation throughout the entire development lifecycle. 🔍 What we cover in the blueprint: - From dataset curation (public & internal data) to synthetic query generation - To designing metrics and rubrics that go beyond BLEU / ROUGE - To building an evaluation platform to manage versioning, experiments, and traceability - To layered gates and automated checks at pull-request, staging, and production levels - To continuous improvement loops, mining real-world failures to guide the next iteration 💡 Key insight: Evaluation is not a “post-hoc step” — it must be treated with the same rigor as production code. Every prompt tweak, retriever update, or model swap must pass through reproducible, automated checks. Tiny changes can cascade, and without guardrails, regressions bleed into production. If you’re building conversational AI — whether retrieval-augmented generation systems, QA assistants, or multimodal agents — we hope this blueprint helps you construct a robust evaluation-first pipeline that scales. Check out the full write-up here: https://lnkd.in/eAkpm2yg We’d love to hear your thoughts or experiences — what evaluation practices have worked (or failed) for you? #AI #MachineLearning #ConversationalAI #LLMs #Evaluation #MLOps

51

3 Comments

Comparing models.. Finding that a small 1B #llm hallucinates less than a 4B for semantic to structural matching. Small llms for targeted tasks for the win. The new Meta llama3.2 1B is better than expected and cheap on token cost.

3

Barbarians at the Gate (https://lnkd.in/gkEBX79s) is a very interesting new paper from some great folks at Berkeley, with some exciting results showing the potential for AI in systems research. But I think the authors aren't quite asking the hardest problem about where this takes systems as a field. I wrote a new blog post about it: https://lnkd.in/gxS3s4v4 The core point of the paper is that AI can be effective at automatically driving the 'number go up' part of systems research. Given an evaluator, AI can automate progress on implementation. I believe this is 100% correct. The paper also talks about how more focus will have to go into developing evaluators. Again, I 100% agree, but think they're understating their case. Building evaluators (basically tests) is, I believe, going to become the focus of systems and software over the next decade. Which leaves researchers with the hard problem of choosing which problems to go after. That's never been easy in academic systems, and is only going to become more of a challenge. But the best systems research is driven by choosing problems well! This vision for AI could lead to lots more of the least interesting systems research. The "number go up" kind. Which could make it harder than ever to identify the truly innovative and visionary work. We're already seeing that elsewhere in science. So, mostly, I think the paper is right, but not yet asking the biggest question: what should systems research be?

276

12 Comments

Document-centric tasks sit at the core of many enterprise, business, and government workflows. Search is important, but the real challenge is going beyond retrieval, enabling systems to reason across documents, verify facts, and handle multi-step information tasks. Great to see a new model from Databricks moving in this direction. I expect we’ll see many more models, including open-weight ones, designed specifically for document-driven workflows, an area with huge potential across enterprise and government use cases.

23

1 Comment

We developed an open-source AI tool to design concrete mixes that are stronger, more sustainable, and ready to build with faster—speeding up construction while reducing environmental impact. https://lnkd.in/gPCk8tCM But the impact of this AI tool is not just hypothetical! Amrize used Meta’s AI-based technologies to design a new low-carbon mix, and successfully deployed it in an at-scale slab-on-grade application at Meta's new data center in Rosemont, MN. Compared to the legacy mix, this new AI-designed mix is: 🦁 Stronger ⏱ Faster 🍃 Lower carbon ⏱ ️The ideal set time All this was achieved without needing any new materials, nor special equipment. Best of all, the AI is open-sourced. https://lnkd.in/g2KA7KZW This work was featured in a Meta engineering blog article published today! https://lnkd.in/gBU9HY8H

98

9 Comments