Selman Yilmaz

New research from Databricks. It's about training enterprise search agents via RL. KARL introduces a multi-task RL approach where agents are trained across heterogeneous search behaviors, constraint-driven entity search, cross-document synthesis, and tabular reasoning. It generalizes substantially better than those optimized for any single benchmark. KARL is Pareto-optimal on both cost-quality and latency-quality trade-offs compared to Claude 4.6 and GPT 5.2. With sufficient test-time compute, it surpasses the strongest closed models while being more cost efficient.

396

8 Comments

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

I believe that out of the big 3 data platforms, Databricks generally beats others in performance and costs. Still, my biggest pain point with Databricks remains the very abstract nature of DBUs. Depending on the product family, that 1 DBU can be different $, with each of the products themselves having different DBU consumption rates. Two - three levels of abstraction on pricing just doesn’t make sense to me. *Note: if competing vendors use this to trash on Databricks to position their product, comments might be deleted.

51

34 Comments

🚀 I’ve just published a new article on the Snowflake Engineering blog. By combining R, Snowflake, ADBC, and Apache Arrow, we achieved data processing at the 100,000-rows-per-second level—and I walk through exactly how we got there. When R handles massive tables, the usual bottlenecks are row⇄column conversions, memory pressure, and data transfer overhead. Leveraging Apache Arrow’s columnar format and the ADBC driver lets us slash those costs and keep R’s vectorized processing intact, dramatically shortening the cycle from data retrieval to analysis and visualization. 🧠⚡️ In the post, I cover everything from installing and using ADBC in R to code snippets, Snowflake-side optimizations, and how Arrow actually delivers speed under the hood. It’s not just “hey, it got faster”—I dive into why it’s faster, how broadly the approach can be applied, and how to maintain reproducibility and portability across workflows. 📈🧰 If you want to run production-grade data processing in R, adopt Arrow/ADBC in your stack, or push the limits of analytical workloads on Snowflake, I hope you’ll give it a read. 🔗👇 https://lnkd.in/gm_TbkDE #Snowflake #ApacheArrow #R #Rstudio #Posit #ADBC #DataEngineering #SnowflakeSquad

17

2 Comments

Palantir AIP demonstrates what it truly means to operationalize AI at enterprise scale starting with trusted data, structured through Foundry Ontology, powered by AIP Core, and executed through intelligent agents with human oversight. This end-to-end flow Data → Ontology → AIP Core → Agents → Actions → Feedback enables secure, auditable, and continuously improving AI that moves beyond POCs into real business impact. Palantir AIP: https://lnkd.in/edcuDSeG

21

Inference scaling is quietly becoming one of the most important ideas in modern LLMs. It lets you trade extra compute at inference time for major gains in accuracy, all without touching model weights.

Today we launched Databricks’ document parsing system. This is the first project I worked on after joining, and in 9 months we delivered SOTA quality at 3–5× lower cost, outperforming frontier proprietary LLMs and long-standing service providers across multiple document-understanding benchmarks. Excited for what’s coming next. Blog: https://lnkd.in/gshsVJpA

132

3 Comments

A team led by The Data Science Institute at Columbia University's postdoctoral fellow Eric Feltham set out to understand how people perceive the social ties in their communities—and how those perceptions differ from reality. Feltham's study, coauthored by Laura Forastiere and Nicholas Christakis, was published in Nature Portfolio Human Behaviour and is based on in-person interviews with more than 10,000 adults across 82 rural Honduran villages — one of the largest field studies of social perception to date. Read more about it: https://lnkd.in/e7Ywy_rX

38