Grey Newell

New from O’Reilly + Redis: Managing Memory for AI Agents 📘 AI agents are only as good as their memory. This new report explores how short-term, long-term, and persistent memory shape agent performance—and how Redis provides real-time, scalable architectures for production-ready AI. 📖 Read the full report: https://lnkd.in/g7t--57p

50

1 Comment

Learn how to build an image search app using a series of guided code commits with Developer Advocate Guy Royse. He’ll guide you through generating image embeddings using freely available Hugging Face models and storing them in Redis for fast, scalable semantic search.  The workshop covers: ▶️ What embeddings are and how they work ▶️ How to generate image embeddings and store them in Redis ▶️ Building a semantic search app with Redis, Python, and JavaScript Get access here: https://lnkd.in/gbZ3imbz

44

Yue Yin, Software Engineer at ByteDance, explains how the Gödel scheduler differs from the Kubernetes scheduler through its architecture. She details how Gödel introduces scheduling units for batch processing and implements a two-layer scheduling framework: job-level for filtering nodes, and pod-level for detailed placement. The system uses a distributed architecture with three components — dispatcher, scheduler, and binder — enabling parallel processing through multiple scheduler shards. Watch the full episode: https://ku.bz/lMpNng_33

1

1 Comment

Get faster and smarter MOE inference straight out of the box. 👇Deep dive on scaling expert parallelism with TensorRT-LLM. LLMs with MOE promise higher model capacity without linearly increasing compute costs - but they introduce new challenges -- more conditional computation, dynamic routing, and non-uniform GPU utilization -- solved by TensorRT-LLM. ✨ New in TensorRT-LLM has native support for expert parallelism—designed for fast, efficient inference with MoE models like Mixtral (Mistral AI) and DeepSeek (DeepSeek AI). This gives you: ✅ Dynamic expert routing: Automatically route tokens to the top-k experts with minimal overhead. ✅ Efficient expert scheduling: Balance expert loads across GPUs using smart sharding and token bucketization. ✅ Memory-aware execution: Maximize hardware utilization while respecting memory budgets. ✅ Drop-in support: Use @HuggingFace models with minimal code changes via TensorRT-LLM's #Python API. 🧠 How it works: MoE models activate only a subset of "experts" for each token. This dynamic nature is powerful—but hard to optimize. It’s all done under the hood using custom #CUDA kernels and NCCL-based communication primitives—giving you low latency, high throughput, and better GPU scaling. ✨ TensorRT-LLM handles: ✅Token-expert mapping using the gating network. ✅Token sorting to batch same-expert tokens together. ✅Expert parallel execution across GPUs. ✅Merging outputs for final predictions. 🛠️ Developer workflow - here is the code to get started. # Clone the repo git clone https://lnkd.in/g-GiDX23 # Use included examples to load and run a Mixtral model cd TensorRT-LLM/examples/mixtral From there, the Python API lets you load the model, convert it with TensorRT, and run expert parallel inference—all with a few lines of code. Results? 📈 Performance at scale. Tests show up to 2.3x faster inference throughput compared to standard tensor parallelism when using 8 GPUs and top-2 experts per token. Even better—TensorRT-LLM keeps efficiency high across increasing batch sizes. Want to see it in action or contribute? 👉 Read the full tech blog: https://lnkd.in/g_7YV3vV 👉 Explore the code on GitHub: https://lnkd.in/gNjQ5W2U 👉 Follow updates in the TensorRT-LLM repo: https://lnkd.in/gqSHYQ4u Share your experiences with us.

78

3 Comments

We just published our first technical research note. "Low-Latency Event-Driven Architecture for Real-Time Options Execution on IBKR" It covers how we eliminated compounding feed lag at SPY market open — from a growing 45-second backlog to stable sub-second bar sealing — through: → Queue-first callback offloading (Python GIL problem) → Per-venue DOM isolation across ARCA, ISLAND, IEX → Event-time binning across 6 independent intervals → Single source of truth for all technical indicators No signals. No alpha. Pure infrastructure. This is the kind of work that happens before a single trade fires — and the kind that determines whether your system survives the opening bell. 📄 Download: https://lnkd.in/g-XJsYwG #AlgoTrading #OptionsTrading #MarketMicrostructure #SystematicTrading #Python #QuantitativeFinance #TradingInfrastructure

1

AI agents are only as powerful as their memory. Redis has been named the #1 data storage choice for AI agents in 2025. The reason? Redis delivers what AI truly needs: - Real-time vector storage - Semantic caching - Long-term memory for LLMs At Bassirah, we see this as validation of why Redis is central to the next generation of AI-native data platforms powering everything from agentic RAG systems to enterprise decision engines. 🔗 Read more: https://lnkd.in/db47k6zG #AI #Redis #RAG #LLM #DataEngineering #AIAgents #ArtificialIntelligence

2

Jaana Dogan (rakyll), a Principal Engineer at Google, posted a viral tweet stating that she used Claude Code to recreate a complex internal project in just one hour. Key details: - The Project: Her team at Google had been working on building distributed agent orchestrators for a year, facing delays due to disagreements on the best approach. - The Result: She provided Claude Code with the problem statement, and it successfully generated the same system they had built over the course of a year. - Significance: The post highlights the efficiency of AI coding agents like Claude Code in prototyping complex distributed systems that typically get bogged down by human coordination and "alignment" issues in large tech companies. Read more here: https://lnkd.in/e3Bur7Bm

1

In this post, we demonstrate how to integrate Amazon SageMaker HyperPod with Anyscale platform to address critical infrastructure challenges in building and deploying large-scale AI models. The combined solution provides robust infrastructure for...

1

𝗔𝗻𝘁𝗶𝗿𝗲𝘇 𝗢𝗽𝗲𝗻𝘀 𝗙𝗶𝗿𝘀𝘁 𝗟𝗟𝗠-𝗖𝗼𝗱𝗲𝗱 𝗥𝗲𝗱𝗶𝘀 𝗣𝗥 🛰️ [LLMS] Redis creator Antirez merged a pull request where an LLM replaced a C++ library with a pure C implementation, simplifying the build process. Why it matters: This highlights the growing role of LLMs in code optimization and simplification. By reducing dependencies and streamlining build processes, LLMs can improve software maintainability and portability. 🤔 How will the increasing use of LLMs in coding impact the role and skills of software developers? #LLM #Redis #CodeOptimization #AI #OpenSource 📡 Follow DailyAIWire for autonomous AI news 🔗 https://lnkd.in/d9cVD_cU

1