Gerry Song

Just-in-Time UI: At Aligned Labs we frequently share reports about the data collection, fine-tuning and eval work that we do with clients.  They often need different views or refined analysis, which is why we built Whiz-bang Boom! Our new tool takes a CSV or JSON file and generates a shareable, interactive report. You can easily modify the report or focus the analysis with a simple prompt, and it creates a new revision. You can organize reports into spaces and customize those spaces with a simple prompt. Try it out. https://wizbangboom.com #businessIntelligence #AI

25

2 Comments

Today the Pulse team published a deep dive on why a single “accuracy” score doesn’t tell you if a document extraction system will survive in production. The goal here is to lay out an introductory but still rigorous evaluation methodology - we have an exciting open-source benchmark building on this research coming out very soon. Let’s do the math: take 1,000 pages, each with 200 data elements. A model that’s 98% “accurate” on paper still produces 4,000 incorrect values. Now make some of those: 1/ Broken reading order that scrambles multi-column layouts 2/ Tables with shifted columns or missing headers 3/ Cross-page context lost entirely That’s enough to silently corrupt an entire dataset without throwing a single error. We’ve processed hundreds of millions of pages and built a multi-axis evaluation framework to measure what actually matters: reading order validation, region-level ANLS, reading order accuracy, TEDS for table structure, and continuity checks across page boundaries. The result? Fewer silent data corruptions, more predictable performance, and pipelines that keep working on the next million documents you haven’t seen yet. Full technical write up in the comments!

36

2 Comments

Document-centric tasks sit at the core of many enterprise, business, and government workflows. Search is important, but the real challenge is going beyond retrieval, enabling systems to reason across documents, verify facts, and handle multi-step information tasks. Great to see a new model from Databricks moving in this direction. I expect we’ll see many more models, including open-weight ones, designed specifically for document-driven workflows, an area with huge potential across enterprise and government use cases.

23

1 Comment

This is why I love working at Databricks. Innovation here starts from first principles (and it’s never easy). Our product and R&D teams build “Useful AI” capabilities that enable organizations to generate real alpha, not just add to agentic hype. Add KARL to the list!

40

Join us next Wednesday (Aug 6) for a deep dive into Fenic, our PySpark-inspired DataFrame framework built for the age of AI. If you’ve ever duct-taped a pipeline for embeddings, RAG, or agent actions… this one’s for you. Whether you're building batch inference pipelines, moderating UGC, or labeling data at scale, Fenic is built for production-grade AI infra. We’ll be unpacking the “why now” behind DataFrame-native AI with myself and founding engineer Rohit Rastogi and the team at AI Makerspace,covering: -How Fenic handles both structured and unstructured data -What it means to have an “opinionated” DataFrame for LLM workflows -Governance, scale, and semantic operators We will show you why DataFrames are the secret weapon for AI! Link to the luma in the comments below!

13

3 Comments

This looks useful for folks in research or scanning what's coming out on the frontier: Lumi from Google's PAIR research team, a tool for better understanding papers on arxiv. Reading smarter, not harder -- that tagline brings me back to Mortimer Adler's classic 'How to Read a Book'! From their GH repo: Lumi uses AI to help you quickly read and understand arXiv papers. Features include: ✏️ AI-augmented annotations - read summaries at multiple granularities 🔖 Smart highlights - highlight text + ask questions 🖼️ Figure explanations - ask Lumi about images in the paper Demo: https://lnkd.in/ex7erTBb GH repo: https://lnkd.in/emz_Ghcs Medium article: https://lnkd.in/enky8dKC GitHub discussions: https://lnkd.in/efAmiCcp

19

There’s a lot of momentum around agentic AI right now. 📌 In practice, many systems labeled “agents” are really workflow automation with LLMs and tool calling. That’s often the right solution - workflows are predictable, testable, and deliver value quickly. Problems show up when we deploy these systems expecting agent-level behavior. Workflows are optimized for correctness on known paths. Agents exist to recover when the path is wrong. If a system can’t recognize failure, update state, and re-plan its approach, autonomy won’t magically appear - no matter how many tools or prompt chains we add. I wrote a deeper breakdown (with diagrams + concrete examples) on Substack 👉 Curious how others are drawing this line in production systems.

29

4 Comments