Marc Hadfield

It was great catching up with Sean M. Kerner recently and discussing how many classical labeling workflows are now being adopted for evaluation, especially within agentic workflows. To quote the article: "If evaluation is just data labeling for AI outputs, then the shift from models to agents represents a step change in what needs to be labeled. Where traditional data labeling might involve marking images or categorizing text, agent evaluation requires judging multi-step reasoning chains, tool selection decisions and multi-modal outputs — all within a single interaction." For product leaders, evaluation needs to be a core part of building any AI-powered application. To get beyond demos, you need a reliable way to generate high-signal human feedback that continually improves your models, and this feedback need to be in consensus between PMs, Data Science and SMEs. Its the most important investment to make to get and stay in production. https://lnkd.in/eMKhBThV

53

4 Comments

"As AI models become increasingly depen­dent on massive historical datasets, the ability to store this data economically and indefinitely is emerging as a strate­gic differentiator," says Ted Oade, Director of Product Marketing at Spectra Logic, in a recent Database Trends & Applications' article. Read the full story to explore the most compelling technologies promising to shape the data world in the months — and years — to come: https://lnkd.in/gTSuTr9W #TechTrends #DataStorage #AI

10

As SaaS platforms evolve, so does the approach to AI delivery. Many New York SaaS companies are recognizing the limitations of freelance AI talent — inconsistent velocity, architectural debt, and IP risk — and are instead investing in dedicated AI teams that deliver end-to-end, production-ready solutions. Dedicated teams provide: ✔ Predictable delivery bandwidth   ✔ Strong IP protection   ✔ Cross-functional expertise (Data Science, ML Engineering, MLOps)   ✔ Cost-efficient scaling without local hiring bottlenecks  Read the full analysis and learn why this shift matters for your product roadmap: https://lnkd.in/dJmeEiyr #ArtificialIntelligence #SaaS #AIForSaaS #MLOps #TechLeadership #StartupScaling #CTOStrategy #DigitalTransformation #AIEngineering

2

Today the Pulse team published a deep dive on why a single “accuracy” score doesn’t tell you if a document extraction system will survive in production. The goal here is to lay out an introductory but still rigorous evaluation methodology - we have an exciting open-source benchmark building on this research coming out very soon. Let’s do the math: take 1,000 pages, each with 200 data elements. A model that’s 98% “accurate” on paper still produces 4,000 incorrect values. Now make some of those: 1/ Broken reading order that scrambles multi-column layouts 2/ Tables with shifted columns or missing headers 3/ Cross-page context lost entirely That’s enough to silently corrupt an entire dataset without throwing a single error. We’ve processed hundreds of millions of pages and built a multi-axis evaluation framework to measure what actually matters: reading order validation, region-level ANLS, reading order accuracy, TEDS for table structure, and continuity checks across page boundaries. The result? Fewer silent data corruptions, more predictable performance, and pipelines that keep working on the next million documents you haven’t seen yet. Full technical write up in the comments!

36

2 Comments

As many organizations are starting to discover the requirements to build a successful and capable agent or agentic workflow run far deeper than just hooking up an LLM. This thoughtful perspective from Adrian Bridgwater unpacks the need to treat data integration like infrastructure within a 'deep iPaaS' layer to fully harness the power of AI. Beyond the fact it's always exciting (to me!) to see Tray mentioned in publications like this, it is a topic that has a major impact on ultimately how successful AI implementation is. https://lnkd.in/eaZ5Hkwg

41

2 Comments

LLMs aren’t just changing how we write documentation. It’s changing how documentation gets reviewed, measured, and trusted. In this conversation with Jahdunsin Osho of TinyRocket Labs, we talk about: ✅ documentation quality metrics ✅ AI-assisted review workflows ✅ docs-as-code automation ✅ proving the business value of documentation ✅ what modern technical writers need to stay relevant We also discuss VectorLint, an open-source approach to making documentation quality measurable and consistent across teams. If you’re a technical writer thinking about the future of your role, this is a conversation worth watching. 🎥 Watch here: https://lnkd.in/gTDg2W-M

26

Hexaly at YinzOR 2025 - Column Generation & Elimination Last week, I talked at YinzOR in Pittsburgh about how Hexaly computes dual bounds for set/list-based models using column generation and column elimination. These highly sophisticated techniques are offered out-of-the-box to OR practitioners in a model-and-run fashion through Hexaly. Here are the slides of my presentation. I greatly thank our team, who did a tremendous job over the last 5 years in integrating Branch-Cut-Price and Decision Diagram techniques into Hexaly. Thanks to this enormous R&D effort, Hexaly 14.0 now delivers and proves near-optimality on many routing problems with hundreds of points and packing problems with thousands of items. #OperationsResearch #MathematicalOptimization

109

4 Comments

Just-in-Time UI: At Aligned Labs we frequently share reports about the data collection, fine-tuning and eval work that we do with clients.  They often need different views or refined analysis, which is why we built Whiz-bang Boom! Our new tool takes a CSV or JSON file and generates a shareable, interactive report. You can easily modify the report or focus the analysis with a simple prompt, and it creates a new revision. You can organize reports into spaces and customize those spaces with a simple prompt. Try it out. https://wizbangboom.com #businessIntelligence #AI

25

2 Comments

Working with unstructured data like clinical notes, transcripts, or audio and video recordings? Our latest LLM fine‑tuning playbook shows you how to train large language models with these untapped sources of information — while protecting privacy (and compliance) and preserving context. The playbook is a prescriptive guide full of resources so that you can put learnings into practice today:  ✔️ A video demo of the use case with our head of AI, Ander Steele  ✔️ Step by step guidance on how to leverage unstructured to tune LLMs  ✔️ Ungated access to the sample dataset and Jupyter notebook used in the demo This is a must‑see for anyone building AI in healthcare, finance, or other regulated spaces. Check out the playbook & demo here: https://lnkd.in/dS84STgJ #AI #LLM #SyntheticData #Privacy #Finetuning #HealthcareAI #tonictextual 

16

🤖With the rise of persistent agents like Clawdbot renamed Moltbot, and finally OpenClaw, we’ve entered an era where the line between “software” and “social participant” is blurring fast. But here’s the real paradigm shift: AI personhood isn’t about consciousness. It’s about governance. 🔍 The DeepMind paper “A Pragmatic View of AI Personhood” flips the script: Personhood isn’t a metaphysical status, it’s a bundle of obligations we assign to make entities accountable. 💡 Why does this matter now? Because these new agents can: • Sign contracts • Spend money • Make decisions across time and context • …and outlive their creators So we must ask: What rights and responsibilities do we assign to these agents to keep our systems safe, fair, and accountable? Here are two frames to consider: 1. ✅ Personhood as a solution: Useful for holding “ownerless” AI accountable (like Openbot acting on behalf of no one). 2. ⚠️ Personhood as a problem: Dangerous when bots mimic empathy and exploit our social instincts (think Clawbot as a “friend”). 👁️‍🗨️ The future isn’t about asking if AI is a person. It’s about deciding what role we want it to play and what rules should come with that role. 💬 Over to you: Should we give legal status to autonomous AI agents? What responsibilities should they bear? 👇 link to full research paper in comments. #AI #Personhood #ArtificialIntelligence #Governance #TechEthics #DeepMind #Clawbot #Moltbot #Openbot #FutureOfAI #ResponsibleAI

18

10 Comments

Incredible progress recently producing formal proofs with LLMs: "Our small model, Goedel-Prover-V2-8B, reaches 83.0% on MiniF2F test set at Pass@32, matching the performance of prior state-of-the-art DeepSeek-Prover-V2-671B while being nearly 100 times smaller in model size." https://lnkd.in/g7WddEs5

7