Mo Sarwat

Our latest PyTorch community blog from the SGLang Team covers how SGLang advances hybrid Mamba plus Attention models with improved memory efficiency, prefix caching, speculative decoding, and serving performance. 📖 Read the blog: https://lnkd.in/gfDZkZWF #PyTorch #AIInfrastructure #SGLang #LLM

110

1 Comment

🧠💬 Memory in LLMs A practical guide showing how to implement conversational memory in LLMs using LangGraph, demonstrated through a therapy chatbot. Features code examples for basic retention, trimming, and summarization approaches. Learn to build memory-aware apps 👉 https://lnkd.in/gybcrV5v

967

21 Comments

Fireworks AI is thrilled to announce our strategic partnership with SuperAnnotate — combining our blazing-fast, customizable model inference with their best-in-class data workflows to supercharge GenAI innovation. This collaboration empowers teams to go from high-quality annotated data to optimized model deployment in record time, accelerating iteration cycles, and unlocking real competitive advantage with open-source and custom LLMs. Whether you're building voice agents, code assistants, or multimodal enterprise apps — this partnership delivers the end-to-end infrastructure needed to bring your GenAI products to life. 🗽 Catch us at the AWS NY Summit next week and join us for Fine Wine & Fine-tuning – an exclusive event for GenAI leaders and practitioners.

47

5 Comments

Building a Production-Ready RAG System for Persian Language Processing I’m excited to share insights from developing an advanced Retrieval-Augmented Generation system specifically engineered for Persian/Farsi document processing, leveraging cutting-edge ML infrastructure and architectural patterns. 🔬 Technical Architecture Highlights: Advanced RAG Implementation: ∙ Self-Corrective Agentic RAG with adaptive query rewriting and iterative refinement ∙ Hybrid search architecture combining dense embeddings with SPLADE sparse representations ∙ Multi-agent orchestration enabling context-aware document interaction and general conversation modes Production Infrastructure: ∙ NVIDIA Triton Inference Server deployment on H100 GPUs with vLLM backend ∙ Milvus vector database for high-performance similarity search ∙ PostgreSQL for structured data with MinIO object storage for scalable file management ∙ Model ensemble: Qwen2.5-72B-Instruct-AWQ, DeepSeek-OCR, and sentence-transformers ML Pipeline Innovations: ∙ Automated synthetic QA pair generation using LangGraph workflows ∙ Cross-encoder reranking (ms-marco-MiniLM-L6-v2) for precision optimization ∙ Dynamic prompt management system with A/B testing capabilities and versioning ∙ Fine-tuning pipeline with quality-scored synthetic datasets Engineering Principles: ∙ Clean Architecture with dependency injection and separation of concerns ∙ Docker containerization for reproducible deployments ∙ Comprehensive observability with metrics tracking and performance monitoring This project demonstrates how to architect enterprise-grade RAG systems that combine theoretical ML advancements with practical production considerations, particularly for low-resource languages. Open to discussing technical implementation details, architectural decisions, or challenges in deploying LLMs at scale. #MLEngineering #RAG #NLP #LLMs #VectorSearch #ProductionML #AI #DeepLearning #Persian #TritonInference #CleanArchitecture

1

Query structured geological data using natural language with a large language model: MCP server for accessing geologic data with the Macrostrat API. Came across Github code repositories for setting up MCP servers for geological data using Claude desktop. Kudos to Blake Morgan, P.G. Sharing for my network. Link to the site in the comments. “This Model Context Protocol (MCP) server enables AI assistants and applications to: Query comprehensive geologic data and information provided by the Macrostrat API Access geologic units, columns, minerals, timescales, and more Get geologic map tiles that the LLM can interpret (questionable utility, but might be useful for some applications) Analyze geologic data to improve geological understanding Create graphics and perform data analysis from the data Users can run queries using natural language.” #geology #geoscience #earthscience #subsurface #largelanguagemodel #llm #artificialintelligence #ai #agents

86

5 Comments

Today marks another step toward making Reinforcement Learning (RL) faster, simpler, and scalable for builders harnessing the Essential Cloud for AI. Our support on the development of torchforge - a new PyTorch-native framework - is helping researchers take large-scale RL workloads from experimentation to production. In partnership with Meta and Stanford University, torchforge was validated on CoreWeave’s Cloud Platform proving stable, efficient, and highly performant – ready to power the next generation of RL workloads. torchforge is now supported on SUNK. More in our recent blog: https://hubs.la/Q03PLJC10

136

Soumith Chintala, 𝗰𝗿𝗲𝗮𝘁𝗼𝗿 of PyTorch and one of the most respected builders in 𝗺𝗼𝗱𝗲𝗿𝗻 𝗔𝗜, has joined Mira Murati ’s fast-rising startup, Thinking Machines Lab. After 11 years at Meta (Facebook), where PyTorch grew from a 𝗻𝗶𝗰𝗵𝗲 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝘁𝗼 𝗽𝗼𝘄𝗲𝗿𝗶𝗻𝗴 𝗴𝗹𝗼𝗯𝗮𝗹 𝗔𝗜 𝗶𝗻𝗻𝗼𝘃𝗮𝘁𝗶𝗼𝗻, Chintala’s move signals a new wave in human–𝗔𝗜 𝗰𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝗼𝗻. Thinking Machines Lab isn’t just gathering top talent from Meta, OpenAI, Anthropic, and leading universities—they’re offering 𝗴𝗮𝗺𝗲-𝗰𝗵𝗮𝗻𝗴𝗶𝗻𝗴 𝘀𝗮𝗹𝗮𝗿𝗶𝗲𝘀 (up to $500K), 𝗮 𝗺𝗮𝘀𝘀𝗶𝘃𝗲 $𝟮 𝗯𝗶𝗹𝗹𝗶𝗼𝗻 𝘀𝗲𝗲𝗱 𝗿𝗼𝘂𝗻𝗱, and are aiming for a $50 billion market valuation. Their new product, Tinker, is already in use at Princeton, Stanford, and by major enterprises. Meta, meanwhile, is regrouping under the 𝗻𝗲𝘄 𝗦𝘂𝗽𝗲𝗿𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 𝗟𝗮𝗯𝘀 division led by Alexandr Wang, with other AI leaders reportedly on the move. Chintala’s reasoning: time to pursue “𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙨𝙢𝙖𝙡𝙡, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙣𝙚𝙬, 𝙨𝙤𝙢𝙚𝙩𝙝𝙞𝙣𝙜 𝙪𝙣𝙘𝙤𝙢𝙛𝙤𝙧𝙩𝙖𝙗𝙡𝙚.” His legacy—the PyTorch tool now thriving in classrooms from 𝗠𝗜𝗧 𝘁𝗼 𝗿𝘂𝗿𝗮𝗹 𝗜𝗻𝗱𝗶𝗮—𝗶𝘀 𝘀𝗲𝗰𝘂𝗿𝗲. #AI #TechNews #PyTorch #Meta #ThinkingMachines #Startup #AIJobs #TalentWar #Innovation #BigFunding

8

By building a taxonomy for compound AI systems, researchers are showing that when agents are optimized using structured evaluation frameworks, downstream task accuracy improves. Better evaluation → better agents. Featuring Melissa Pan and Mert Cemri of University of California, Berkeley / CSGE Berkeley Interviewed by Jessica Dai and SAIL Media Thank you to ‪Lambda‬ for making these interviews possible! 👉 Subscribe to our calendar for priority invites to upcoming events: lu.ma/twill #AI #ArtificialIntelligence #MultiAgentSystems #LLM #LLMAgents #CompoundAI #MachineLearning #AIResearch #BerkeleyAI #CSGEBerkeley #TechEvents #StartupCommunity #SiliconValley #FutureOfAI #AI #ArtificialIntelligence #MultiAgentSystems #LLM #LLMAgents #CompoundAI #MachineLearning #AIResearch #BerkeleyAI #CSGEBerkeley #TechEvents #StartupCommunity

34

2 Comments

Rachit Jain's walkthrough shows how to profile kernels with PyTorch tools, import traces into NVIDIA Nsight or Intel Advisor, and read the slanted ceilings that separate compute-bound from memory-bound regions.

5