Qiliang Zhu

I wrote an article on how we translate user-generated content at scale at Mercari. It summarizes two years of work with various models, LLMs but not only, and how cost evolved, reducing it 100x over the period. I also go over non-AI features and user experience. https://lnkd.in/gbpQAQmX

39

2 Comments

Over the past year, my team and I built Pinterest's Model Context Protocol (MCP) ecosystem from the ground up: a central registry, a growing fleet of domain-specific MCP servers, and production integrations across our IDEs, internal chat surfaces, and AI agents. MCP is an open standard that gives large language models a unified way to talk to tools and data sources. At Pinterest, we used it as the substrate for AI agents that can safely automate real engineering tasks, from querying Presto data on demand to diagnosing Spark job failures and surfacing institutional knowledge. A few things I'm particularly proud of: * We designed for security from day one: two-layer auth (end-user JWTs + mesh identities), business-group-based access gating, and mandatory human-in-the-loop for sensitive actions. * We made it easy for any team to ship a new MCP server by creating a unified deployment pipeline that handles all the infrastructure, so domain experts just write business logic. * The ecosystem has scaled to 66,000+ invocations per month across 844 monthly active users, saving an estimated 7,000 engineering hours per month. This was a cross-functional effort. Thank you to everyone on the Agent Foundations, Security Engineering, and Traffic Engineering teams who made this possible, and to our engineering sponsors for their support and guidance. Full write-up on the Pinterest Engineering Blog: https://lnkd.in/e-mH26Bw

146

7 Comments

Huge thanks to Zhe Jia, Shuai Wang, Aditi Bamba, and Martin Wang from DoorDash for publishing this incredibly practical breakdown of their RAG implementation for Dasher support. This post saved me countless hours of trial and error. The two-tier guardrail approach is what really clicked for me: * Run a cheap semantic similarity check first (sliding window comparison between response and KB articles) * Only if that fails, escalate to the full LLM evaluator Result: 90% fewer hallucinations, 99% fewer compliance issues, without killing your latency budget When you're actually shipping RAG to users, this kind of practical filtering makes all the difference. Everyone talks about prompt engineering, but nobody talks enough about how to catch bad outputs before users see them. Their broader setup—quality monitoring across 5 dimensions, regression test suites, the whole LLM Judge framework—gives you a real blueprint beyond the basic tutorials. You can't fix what you're not measuring. The article is https://lnkd.in/dbC9BG7a

2

Joakim Recht (Uber Distinguished Eng) on tech leads: "Software engineers need to write code. If you're not writing code, you're not a software engineer." Joakim Recht grew from a Senior Engineer to a Distinguished Engineer at Uber and I asked him what it took to get there. We covered: • The project that grew his career • What makes a great software engineer • Learnings from promo committees • When promos are unfair • Biggest technical mistake at Uber in hindsight • Advice for his younger self 𝗣𝗼𝗱𝗰𝗮𝘀𝘁 𝗹𝗶𝗻𝗸𝘀: • YouTube: https://lnkd.in/gTvsB3Ar • Transcript: https://lnkd.in/gG7bmeU2 • Spotify: https://lnkd.in/g4n-fUxe • Apple: https://lnkd.in/gm8r-2_R

219

10 Comments

"It seemed like a good idea at the time" For the Android Build systems, I had built most of it (not anymore thankfully other people have rebuilt most of my work). But this would lead to the interesting problem of that past bozo being yourself 8 years ago. I always try to make the best decisions possible with the information we have at the time that we make them. This prevents analysis paralysis where you fail to ever make a decision, but it also means that your decisions may not hold up for 10 years and that is OK. (Maybe an interesting later post, but this is very much a vector of tech debt. A decision that was great at the time it was made can be wrong when circumstances change later)

6

3 Comments

You are in a system design interview at Google for the L5 Senior Engineer role, and the interviewer leans in and asks: “Why does Spotify keep playing when I drive into a tunnel with no signal, but YouTube Music often stops or buffers? If you were designing a music streaming system, what different design choices would lead to these two behaviors?” Here is how you break it down. Btw, if you’re preparing for system design/coding interviews, check out our mock interview tool. You can use it for free here: https://lnkd.in/gpCn7t2T We’ve added new features as well:  -Company-specific interviews -In-built interview scheduler  -performance insights & trends — Both apps look like simple music players. Under the hood, they are optimized for very different priorities. [1] Spotify style – Cache first, stream second Idea: The client behaves like a smart offline player that happens to stream. The backend is built to support aggressive prefetching. What happens when you hit play - Client requests the track from a CDN or edge node. - Instead of tiny chunks, it downloads a big buffer ahead of the playhead. - In parallel, it starts pre downloading the next few tracks in the queue. - Data is written to an encrypted cache on disk, not just memory. - Playback reads from that local cache, not directly from the network socket. What this means for tunnels and bad networks - When the network drops, the player already has tens of seconds or entire tracks cached. - Because the next one or two songs are already downloaded, you can be offline for a while and never notice. - Cold start cost is a bit higher. First play might take slightly longer, but then everything feels smooth. - It burns more local storage and possibly more data, because not every prefetched song will be listened to fully. In a design answer, you can mention - Local disk cache with eviction policies (LRU per user, per device). - Background prefetch of N upcoming tracks based on queue. - Download manager that adapts how aggressively it prefetches based on network quality and user settings. - CDN tuned for larger object delivery and range requests. - Explicit offline mode that pins playlists into cache. [2] YouTube Music style – Stream first, cache is minimal Idea: Treat audio like video streams. Cost and bandwidth are optimized first. When you hit play - Player requests audio (and sometimes video) via HLS or DASH-style chunks. - Each chunk is only a few seconds long. - Client keeps a small rolling buffer in memory, not a large queue on disk. - Prefetch of future songs is limited, because video tracks are large and expensive to fetch speculatively. What this means for tunnels and bad networks - If your connection dies, the player only has a few seconds of buffered data. - As soon as those chunks are consumed, playback stalls. - Startup can feel snappy and data usage is controlled, especially for casual listeners. - Works very well on stable networks, feels fragile in spotty coverage.

1,219

38 Comments

𝗧𝗵𝗲 𝗯𝗲𝘀𝘁 𝗔𝗜 𝗰𝗼𝗱𝗶𝗻𝗴 𝗹𝗲𝘀𝘀𝗼𝗻 𝗰𝗼𝘀𝘁 𝗼𝘂𝗿 𝗲𝗻𝗴𝗶𝗻𝗲𝗲𝗿 $𝟲𝟬𝟬 𝗮𝗻𝗱 𝗮 𝗺𝗮𝗿𝗿𝗶𝗮𝗴𝗲 𝗮𝗿𝗴𝘂𝗺𝗲𝗻𝘁. Our VP of Engineering, Xiaofan(James) Luan, was supposed to buy his wife a Dior bag for their anniversary.  Instead, he bought three Claude Code subscriptions and spent the holiday trying to cross-compile 2 million lines of C++. Every fix on one platform broke two others.  $600 later, the only output was "git reset --hard" — and a very cold dinner table.😂 "Make it compile on Windows" is a trap. The real goal was "compile everywhere without hacks" — no AI is going to figure that out for you at 2 am. What worked: constraints before code, review tests not code, bottom-up, one layer at a time. Same task, two days. Then he ran six parallel Claude sessions across three machines with git worktree. The bottleneck stopped being intelligence and started being how fast one person can alt-tab. AI solves exactly the problem you give it. Engineering is in knowing which one to give. His wife is still waiting for that bag. Full story: https://lnkd.in/gtsW_Wvk ——— Follow Milvus, created by Zilliz, for everything related to unstructured data

9

Why does an E3 level SWE at Meta make only ~190k/year, while an E8 level engineer makes over ~$2M/year, even though both engineers are ICs and spend the same time at work? I have spent the last 5 years at Meta as an IC. I joined with a little over 15 years of experience, and I’ve worked with many solid engineers in this time. Here is how I think about that compensation jump. 1. Same hours, completely different “unit of work” An E3’s unit of work is usually a task or a ticket. An E8’s unit of work is a multi year problem for the company. E3: “Implement this service, fix this bug, write this feature.” E8: “How do we cut infra cost by 20 percent across this product” or “How do we make this platform safe to scale to 10x users.” One person is paid to execute. The other is paid to decide what is worth executing in the first place. 2. Radius of impact E3 usually impacts a file, a service, maybe a small team. E8 shapes whole orgs and product lines. If an E3 ships something great, the impact is great but local. If an E8 ships the right platform, hundreds of engineers become faster and the company saves or earns millions every year. Comp tracks the area of the circle you influence. 3. Risk and downside protection At junior levels, mistakes are usually contained and reversible. At senior staff levels, a bad call can burn tens of millions or damage the brand. E8s are paid for judgment under ambiguity. They decide which bets the company should not make, which migrations can wait, which “shiny idea” is going to kill reliability. You pay more to people whose good judgment protects you from very expensive failures.  4. They scale themselves This can happen in a few ways. 1. Delegation with ownership They define the shape of the problem, then hand large pieces to other senior and mid level engineers while keeping the bar and direction clear. 2. Knowledge that travels They write RFCs, public comments, FAQs, wikis, internal posts. One answer helps hundreds of people who will face the same issue next quarter. 3. Tools over heroics Instead of unblocking people manually all day, they build tools, libraries or guardrails so others can unblock themselves. One well designed tool can save thousands of engineering hours every year. This is what “scaling yourself” actually looks like. The company pays for that multiplier.  5. Ownership of the “uncomfortable problems” Junior engineers usually work inside a well defined box. E8s take ownership between the boxes. They pick up problems that:  Span many teams and no one really “owns”  Require aligning leaders who disagree  Have product, infra, legal and security angles at the same time Most people avoid those because they are messy, political and slow. Very senior ICs lean into them. That is where a lot of value sits. Continued ↓

201

15 Comments

Amazon scaled with small, focused teams. My former director Thomas Van de Velde is doing the same—with AI agents. Brilliant perspective on AI swarms, domain ownership, and system design!!! Worth a read 👇

3