Yandong Liu

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

270

11 Comments

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

156

7 Comments

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,671

190 Comments

You don't need a separate reference model for RL. We built an RL pipeline built on a multi-turn-focused modification of Self-Distillation Policy Optimization (SDPO) that uses the same model in two roles: student and teacher. The difference? Context window. → The student generates actions with no knowledge of the outcome. The teacher re-evaluates the same tokens with hindsight: the observation from each action, terminal test output, and a successful rollout from the same prompt. The per-token advantage is just the logit gap between these two views. → This gives you dense, per-token credit assignment instead of a single scalar reward for the whole trajectory. Think tokens get 0.25× weight (loosely guided, not tightly constrained). Action tokens get full gradient. System/user/tool result tokens get nothing. → The "teacher" is not a frozen checkpoint or a separate reward model. It's your current policy, conditioned on richer information. This is well-defined under MoE because same parameters with different routing is normal operating behavior. → We use a tiered reward system with hard gates only for truly unrecoverable failures: sandbox escapes, test corruption, harness crashes, timeouts, infinite loops. Everything else like wrong directory, failed commands, incorrect approaches are learnable signal, not wasted compute. The key insight: most of the information you need for credit assignment is already inside the model. You just need to ask it the right question with the right context. On LiveCodeBench v6 SDPO reached 48.8% vs GRPO's 41.2% beating Claude Sonnet 4 with an 8B model! That's bonkers.

5

We at Haladir (YC W26) released a report on RLFR: Reinforcement Learning from Formally-Defined Rewards for Code Generation (https://lnkd.in/guCH7w7C). The core idea is simple: we present a simple offline and single-turn RL framework that uses formal verification as the primary reward signal for program synthesis in LLMs rather than unit test-based rewards (RLVR), showing improvements over a swath of different coding benchmarks. We fine-tune both Qwen2.5-Coder-7B-Instruct and Qwen3-8B using RLFR and Dafny, a specification language that utilizes Boogie (IVL) and Z3 (SMT solver) to check verification conditions. Despite solely training on Dafny, we show improvements on Python-native coding benchmarks, suggesting generalization past the formal verification domain.

90

8 Comments

We created a ready-to-run dev-environment to run Unsloth AI with SGLang, so you can start running the examples under 3 minutes with multiple GPU options: 1. Login at velda.cloud, create a new instance, choose "unsloth" as the image, and connect. 2. Start SGLang on a GPU instance: vrun -P gpu-l4-1 -s svc python3 -m sglang.launch_server --model-path unsloth/Llama-3.2-1B-Instruct --host .... 3. After step 2 shows ready, in another terminal, use svc:30000 as the endpoint to run your model: python unsloth_sglang. py 4. You will see the output immediately. With Velda, your "dev machine" can scale with infinite GPUs with just one command, and only billed when GPU workload(training, inference) is actively running. Get started on velda.cloud for free today!

6

Trustworthy and accurate memory for AI Most LLMs plateau as conversations stretch across days and teams: context windows bloat, retrieval drifts, and costs creep. The fix is architectural — not another prompt or more data or another vector database. "Memory" for businesses has long meant databases and the software that makes them accessible. That hasn’t changed. AI is excellent with data in the context window — the trick is using the right storage and retrieval layers, just like a data warehouse. Also, trustworthiness matters: hallucinations erode user confidence fast. Projects need to treat LLMs as new hires - Highly skilled, but requires onboarding and domain semantics. - Needs colleagues to consult when uncertain about a problem or solution. - Needs access to databases and knowledge systems. - Needs tools to interpret data. ++ What reliable memory looks like Rather than hoarding tokens, this pattern combines five elements working together to provide a well-crafted context.  - Business Intelligence: create a taxonomy based on the semantics of the business; use an LLM‑supported pipeline to extract these and store as documents or rows.   Example — Taxonomy: Order → Return → Shipping Label; Payments → Chargeback → Evidence; surfaces become filters and facets at retrieval time.  - Hybrid retrieval: blend lexical analysis (NLP‑derived features such as tokens, n‑grams, phrases, and entities), approximate nearest neighbor (ANN) vectors, and structured predicates; each compensates for the others’ blind spots and provides trustworthy output.   Example — Query: "Can I return without the box?" Lexical analysis catches "return", vectors surface "refund"/"RMA", predicates filter to "country = UK" and "policy_version = current".  - Feedback loops: track accepted suggestions, reformulations, and task closure; feed those signals into re‑ranking and query generation.  - Strategic memory: store “what worked” patterns, not just facts; retrieve proven approaches for similar tasks.   Example — For "delayed shipment" complaints, the pattern "apologize + expedite + 10% coupon" repeatedly resolves cases → recall that playbook.  - Evidence and queries: bind every answer to auditable sources, and persist typed facts with provenance so you can inspect and join them with SQL.   Example — Answer cites Policy §3.2 and links the exact paragraph; facts saved as rows: (order_id, promised_date, delivered_date, source_doc_id). ++ Closing stance - Memory is a product choice, not a prompt trick. - Reuse what you already have: databases, knowledge systems, services, and governance — your data lives there. - Add semantics (taxonomies, entities, relationships) to sharpen retrieval, ranking, and decisions. - Focus on ingestion first not query optimization: Garbage in still means garbage out - There are no quick wins for trust. Treat memory like any core system: plan, architect, and implement deliberately. #AI #LLM #Retrieval #GenAI #RAG

9

1 Comment

👉️ https://lnkd.in/gAUt3gZB 🧠 Writing PyTorch extensions? Your memory allocator might be the hidden bottleneck. When you're building custom operations—or integrating specialized hardware—PyTorch's default memory management won't always cut it. This blog dives deep into how to write efficient, debuggable, and performance-tuned allocators for your extensions. Key insights include: 🚀 When (and why) to use custom allocators 🎯 Memory alignment, pooling, and fragmentation strategies 🔍 Debugging tools and sanity checks you wish you had sooner 🤝 How to coexist with PyTorch’s built-in allocator without reinventing the wheel 💬 Building custom extensions or facing memory headaches? Let’s trade strategies—what’s worked for you? #PyTorch #AIEngineering #DeepLearning #MemoryManagement #CUDA #PerformanceEngineering #PythonDev #MachineLearning

1