Ajinkya More

While benchmark performance often guides large language model selection, production environments reveal a different reality. Reinforcement Learning from Human Feedback (RLHF) and preference-optimization methods impose behavioral constraints that define the architectural success boundaries of multi-agent systems. Preference-optimization aligns generalist conversational models (Gemini, Claude) toward helpful, fluent responses, optimizing for rapid engagement and concise, confident answers. These models excel at intent extraction and iterative exploration. However, in multi-file codebases, they exhibit premature solution convergence, emitting code before fully modeling the repository architecture. This yields structurally fragile changes unless the system enforces a strict read–analyze–plan–edit loop. Conversely, specialized reasoning models (o1-style reasoning families, DeepSeek-R1) show an inverse profile. They allocate substantial test-time compute to internal chain-of-thought and context assimilation before generating final tokens, often producing hundreds to thousands of reasoning tokens on harder tasks. This yields deeper architectural awareness and stronger structural coherence; however, the added latency and verbosity break rapid-prototyping loops when deployed naively as drop-in chat replacements. Production routing patterns: effective architectures route tasks by cognitive load, not benchmarks: ↳ Conversational models handle exploratory analysis and requirements extraction to leverage low latency and high interactivity. ↳ Synthesis/reasoning models execute state transformations across 10k+ LOC and complex refactoring, where high context fidelity and structural coherence dominate. ↳ Sequential routing with explicit handoff contracts governs hybrid workflows, strictly defining which agent plans, edits, and verifies. Failure boundaries: orchestration overhead exceeds gains when tasks are below ~500 LOC or require fewer than three meaningful context switches. In these cases, single-model invocation with an extended context window typically outperforms multi-agent routing by bypassing coordination, retries, and glue logic. Observability requirements: without instrumentation for per-model token usage and context-window saturation, engineers debug RLHF mismatches by feel. Observability must come before routing decisions; teams cannot tune cognitive profiles they do not measure. Commercial narratives emphasize general-purpose intelligence. In practice, engineering teams deploy specialized inference profiles within deterministic topologies—performance scales when the system architecture accommodates RLHF constraints rather than fighting them. #SystemsArchitecture #SoftwareEngineering #MLOps #EnterpriseAI #AgenticAI #LLMEngineering

🚀 Just wrapped up a hands-on project applying Retrieval-Augmented Generation (RAG) to a real-world use case: interpreting LIC’s New Jeevan Shanti policy document using LLMs. The core idea? Combine vector search over regulatory PDFs with GPT-4 to generate grounded, faithful answers to complex policy questions. Key capabilities we built: ✅ LCEL-based RAG architecture using LangChain ✅ Contextual compression retriever for focused document lookup ✅ Modular LLM & embedding selection (OpenAI, HuggingFace, Mistral) ✅ Evaluation using LLM-powered metrics: faithfulness, relevance, correctness 📄 The document used is publicly available here: LIC New Jeevan Shanti Policy PDF 💻 Full code available on GitHub: https://lnkd.in/gS9mEzWA 💡 This architecture brings the best of both worlds — retrieval accuracy and generative flexibility — especially in regulated domains like insurance and financial services. 🔜 Next step: Incorporate Agents that can reason across multiple steps — not just answer a question, but decide which tools to use, when to search, how to validate, and when to stop. That’s where the future of enterprise GenAI lies. Would love to hear how others are pushing RAG, agents, and LLMs in production! #GenAI #LangChain #LLM #RAG #AIForOps #InsuranceTech #LLMEvaluation #LangChainAgents

61

1 Comment

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

In physical autonomy, evaluating a plan based on its semantic plausibility guarantees failure. The architectural requirement is proving that an action is safe under bounded uncertainty. Language models generate plans through token prediction, a probabilistic process optimized for semantic coherence rather than physical safety. When those plans activate motors and servos, failure modes shift from hallucinated text to physical collision, drift, and hardware damage. The mismatch is structural: token prediction cannot validate the real-time state space. Autonomous systems fail due to accumulated sensor error, dynamic overshoot, and latency-driven desynchronization. A model that proposes "move the gripper 15cm left" cannot verify whether that instruction violates joint limits, exceeds payload capacity, or intersects an obstacle that appeared moments earlier. Scaling model parameters improves pattern recognition, not constraint enforcement. Safe autonomous architecture pairs probabilistic cognition with deterministic execution controls. Four boundaries enforce that separation: ↳ State reconciliation gate: The control system continuously reconciles intended motion with measured reality using proprioception, vision, and Inertial Measurement Unit (IMU) telemetry at control-rate frequencies (10–1000 Hz+, depending on the embodiment). If the measured state diverges beyond tolerance, the system halts or replans. ↳ Deterministic feasibility gate: A validation engine gates every proposed motion through deterministic checks before execution—forward kinematics for reachability, inverse dynamics for torque limits, and collision checks (e.g., swept-volume constraints). If any constraint fails, the gate rejects the motion unconditionally. ↳ Independent safety interlocks: Firmware-level circuit breakers and emergency-stop logic override model outputs when sensors detect force spikes, proximity violations, or thermal limits. These mechanisms operate independently of the probabilistic stack; a crashed inference server cannot disable them. ↳ Platform decoupling: The autonomy stack must treat physical embodiment as a swappable dependency. Architectures that bind the reasoning layer directly to proprietary hardware interfaces create structural fragility. When vendors deprecate interfaces, tightly coupled stacks break. Hardware abstraction layers are load-bearing constraints, not optimization targets. This separation builds on established engineering foundations. Traditional motion planning and control already rely on deterministic methods such as RRT* and Model Predictive Control (MPC) to enforce feasibility. Generative models add valuable intent synthesis, inferring semantic goals from ambiguous instructions, but the final arbiter of safe execution must remain verifiable. Architects must treat the large language model as an untrusted planner, requiring deterministic evidence before committing any actuator. #SystemsArchitecture #EnterpriseAI #AIGovernance

5

2 Comments

Single-method retrieval often fails in #financial and #regulatory domains, exact-term search misses semantic relationships, while dense models overlook structured identifiers. A multi-path approach resolves this gap. The Multi-Path Retrieval (#MPR) pipeline implements four complementary retrieval strategies with a domain-specialized re-ranking mechanism to balance lexical precision and semantic understanding: #BM25 Sparse Retriever – captures exact matches for regulatory codes, ticker symbols, and key phrases. #Dense Semantic Retriever – identifies paraphrases and conceptually similar expressions through high-dimensional embeddings. #Metadata-Aware Retriever – filters at section-level granularity, narrowing retrieval to relevant document parts before detailed chunk search. #HyDE Retriever – generates a hypothetical answer, embeds it, and retrieves real text segments with similar structure and terminology, bridging the question–answer linguistic gap. Final results can be aggregated through weighted rank fusion (like semantic > BM25 > metadata > HyDE ), producing top-ranked chunks enriched through context bundling when adjacent similarities exceed 0.85. This ensures coherent, context-complete passages are provided to downstream models. Such architecture is especially effective in financial compliance and regulatory analysis, where both formal precision and contextual interpretation are essential. The hybrid fusion enhances #recall, #contextualaccuracy, and #interpretability in retrieval-augmented systems(#RAG). #GenAI #Banking #AIEngineer #LLM #Fraud

4

Great article from Kristin Milchanowski, Ph.D. showing how, at BMO, we go above algos and models: we examine how humans interact with them. If the experience is right, the value will follow!

19

2 Comments

A paper by Cornell University argues that SLMs are the future of agentic AI: https://lnkd.in/g688UxTJ Check out #Couchbase's Mohan Varthakavi's POV on when to use SLMs over LLMs when designing AI apps: https://lnkd.in/gnVP2n_5

13

Over the past few weekends (and a few late nights), I’ve been “back at school” with the Problem‑First AI course by Aishwarya Naresh Reganti and Kiriti Badam on Maven: https://lnkd.in/e8Pcqyvy Instead of just reading about agentic systems, I’ve been getting hands-on and actually building them. A few ideas from the final session really stuck with me: - Don’t just learn — meta‑learn: Not just collecting tools and links, but learning how to learn and what questions to ask. - You can’t catch up — stay curious: No one is a complete expert in a space moving this fast. Openness and “I don’t know (yet)” beat pretending. - Start with the simplest solution: KISS before you build agents. Ask if a simpler approach would solve the problem just as well. - Concepts first, tools second: Design thinking is expensive; implementation keeps getting cheaper. The real leverage is in how you frame and design the system. - Say “it depends” more often: Good AI answers depend on data, constraints, environment, and business context — so ask deeper questions first. - You belong here: AI is not just for people who love math, transformers papers, or coding. Curiosity and problem-solving matter just as much. This course has given me stronger fundamentals and more confidence in how to navigate a rapidly evolving AI landscape. And this is just the beginning — the journey continues. Huge thanks to Aishwarya Naresh Reganti and Kiriti Badam for building such a thoughtful, practitioner-focused program. #AI #AgenticAI #GenAI #LLMs #AIEngineering #ProblemFirstAI #SystemsThinking All views expressed here are my personal opinion.

25

3 Comments

We open-sourced our framework for validating #ExplainableAI methods! Samuel Sharpe, Sr. Lead Machine Learning Engineer, shares how his team developed guardrails for ablation studies on tabular data. Check out the framework and test your own XAI methods. https://bit.ly/4esirrV

43

2 Comments

If your datasets are poorly labeled, you’re inviting a host of issues like noise, bias, and inconsistencies that can seriously hurt your model's accuracy and reliability when it’s out in the wild. That's why data labeling has evolved from being just a preliminary step into a vital part of engineering and operations in the AI lifecycle.

7