ngrok

Just shipped: bringing your own provider keys to AI Gateway is now private by default. Your app sends an AI Gateway key and we attach your OpenAI, Anthropic, or other provider keys as they pass through. Your apps get one authorization model instead of two. A few things that fall out of this: → Up to 15 keys per provider on a single gateway key, for failover and staged credential rotation without code changes. → AES-256 encryption, with plaintext held only in memory during upstream requests. → Standard OpenAI and Anthropic SDKs work unchanged. Point baseURL at the gateway, use your AI Gateway key, and the rest of your code is identical.  const client = new OpenAI({   baseURL: "https://lnkd.in/g4jic2KV",   apiKey: "ng-xxxxx-g1-xxxxx",  });

We gave ngrok domains a `resolves_to` property and, what do you know, made the *routing* part of data residency a one-click. Pin your traffic to specific PoPs so European data stays in Europe, then rearrange the whole thing tomorrow if you want. Set it up in the dashboard or API. Currently free on pay-as-you-you plans. https://lnkd.in/gFC6qq2h

We just published ngrok's first agent skill: expose-localhost. Tell your agent "expose my app on port 3000" or "share my local server with a coworker" and it sets up the ngrok tunnel. But it doesn't stop at a basic tunnel—it can layer on Google/GitHub/Microsoft OAuth, OWASP Core Rule Set protection, rate limiting, and email-based access restriction, all from the same conversation. The whole point of ngrok has always been collapsing infrastructure complexity into simple commands. Agent skills are a natural extension of that—now the command is plain English and the agent handles the rest. $ npx skills add ngrok/agent-skills https://lnkd.in/gPG3TNyx

Quantization can make an LLM 4x smaller and 2x faster, with barely any quality loss. But what *is* it? Sam Rose crafted a beautiful interactive essay explaining it from first principles, aimed at coders, not mathematicians. We hope it will cause you to, in the words of our CTO Peter S., "rethink everything you thought you knew about how LLMs fit in RAM." https://lnkd.in/gkzBNyPA

For those who don't follow Sam Rose (you really should). He does deep dives into technology in a way that makes it easily consumable and downright informative. You can check out all of his articles at https://lnkd.in/gvc_Whxd and he just dropped a new piece on quantization which you can find at https://lnkd.in/gYS7wRJt It caused me to rethink everything I thought I knew about how LLMs fit in RAM. You should definitely check it out.

You want to route requests across OpenAI and Anthropic with failover, load balancing, and observability. You don't want to manage two provider accounts to get there. ngrok's AI Gateway now has its own API keys and prepaid credits, which means you get one key for both providers. Point your SDK at your gateway, add a few dollars in credits, and start making requests. You can even set your model to "ngrok/auto" and the gateway picks the best provider for you. Bring-your-own-key still works the same, and you can now mix both modes—ngrok-managed keys for some providers, BYOK for others—in one AI gateway. https://lnkd.in/dTNmfaC4

Our latest Kubernetes Operator release adds support for many operator installs in one cluster. Use ngrok-operator to isolate ingress resources by team/customer/get-creative-with-it, or eke out performance by running separate operators in clusters that span multiple regions. Cool. https://lnkd.in/gus5WXjT

A few months ago, I got really curious about coding agents and wanted to know: What happens when you tell a coding agent to think about what it's done and do better next time? So, I built bmo, a coding agent designed to improve itself at runtime. It's got four self-improvement loops, shell access, and a hot-reloadable tool library. Over 100+ sessions, it built 11 new tools, 7 skills, and an elaborate system for procrastinating. What did bmo and I learn? Knowing isn't the same thing as doing. A detailed skill for capturing learning events at runtime got used twice in 60+ sessions, but a reflection template at session end worked every time. Clear triggers beat an LLM's ability for "vigilance." (I also made a big oopsie.) The instinct to defer is real. Despite "build tools IMMEDIATELY" in the system prompt, the moment I gave bmo a file to dump opportunities for future work, I accidentally made deferral the most natural next thing to do. Specific tools outperformed generic shell access. run_command was 84% successful and fraught with footguns. A bmo-built tool like search_code came with best practices, which created fewer decisions for the LLM and fewer failure modes for me. Telemetry is the only true "memory" that persists across sessions, and thus became the pivot point bmo acted on most. Without numbers, the very idea of self-improvement just drifts downstream in the overwhelming flow of context. My big lesson was to stop engineering the prompt to try and parallelize many tasks and start building a better harness for bmo to introspect on the data bmo already had. I'm still thinking there's a better way to handle all this context… sub-agents? Some way of dealing with context rot? I've laid out *all* my lessons, and some more introspection from bmo itself, on the ngrok blog 👇 https://lnkd.in/gzA8kGED

ngrok's AI gateway now supports the Anthropic SDK natively. Change your base_url, keep your prompt caching, extended thinking, all of it. The diff is one line & there's no step two. https://lnkd.in/gB8ciFxG

Ever wondered why some technical explanations just click? This week on Overcommitted, we're joined by Sam Rose from ngrok, a developer educator who creates stunning visual, interactive essays that break down computer science fundamentals. In this episode, Sam shares: - How he went from using a 500kb game framework (Pixi.js) to SVG-based web animations - Why topic selection is the most crucial element to educational content success - His process for creating interactive simulations that take 1-3 months to build - The secret behind making invisible systems (like branch prediction) fascinating - What he's learning about LLM benchmarking and how models actually get evaluated Sam's approach to education is refreshing: he picks topics that are relevant to your daily work but invisible—things your CPU does millions of times per second that you never think about. His articles on load balancing, memory allocation, and prompt caching have been widely shared for good reason. If you're interested in technical writing, developer education, or just want to understand the fundamentals running under the hood of your code, this episode is a must-listen. Plus: We play an LLM-generated Connections game based on Sam's prompt caching article. It's harder than it sounds! 🎧 Listen now on your favorite podcast platform https://lnkd.in/gVpgDTxx