Xi Jin

Comparing models.. Finding that a small 1B #llm hallucinates less than a 4B for semantic to structural matching. Small llms for targeted tasks for the win. The new Meta llama3.2 1B is better than expected and cheap on token cost.

3

We (w/ Shansan Gong, Ruixiang ZHANG, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong) released a family of 7B diffusion language models, DiffuCoder, that specializes on code generation, with a focus on understanding and improving masked diffusion models. A core analysis of DiffuCoder is the autoregressiveness (AR-ness) score, a novel metric that quantifies the causal patterns in decoding, revealing how diffusion models break from strict left-to-right generation for more flexible, non-linear code planning. Recent advances in autoregressive (AR) models dominate code generation, but diffusion-based LLMs (dLLMs) like DiffuCoder offer a promising alternative, especially for complex programming tasks. DiffuCoder explores how these models decode differently—showing less global AR-ness in code tasks compared to math—and how temperature affects both token selection and generation order, unlike traditional AR models. We also introduce coupled-GRPO, a post-training RL method with a coupled-sampling scheme, to reduce performance drops during accelerated decoding, boosting parallelism and efficiency. We use a self-improvement pipeline that leverages AR-ness analysis, coupled-GRPO optimization, and evaluation on benchmarks like AceCode-89k to refine decoding strategies. This approach enables DiffuCoder to navigate diverse code generation pathways and enhance performance with modest computational overhead. Looking ahead, we aim to further leverage Reinforcement Learning to steer code generation through these decoding patterns, with the discrete nature of AR-ness scores providing a foundation for search-based strategies—ideal for the sparse rewards of optimizing complex code structures. Check out our full paper and code for a deeper dive! Paper: https://lnkd.in/gVWU3BDJ Code: https://lnkd.in/gmXTZ_6n Models: https://lnkd.in/gTcKCDr9 #MachineLearning #AI #CodeGeneration #DiffusionModels #NLP

220

5 Comments

Catching this talk between Lloyd Blankfein and David Solomon. Regardless of the industry, there is a lot to learn from how these leaders define "partnership."

1

Google Antigravity exposes the critical flaw in autonomous AI agent architectures: trust based governance at scale. Antigravity represents a major shift in how developers work. Instead of writing code line by line, you delegate entire tasks to autonomous agents that can modify files, run tests, browse the web, and execute changes across your codebase in parallel. The problem? These agents operate on trust, not proof. One developer reported: On Day 3, an agent confidently refactored a utility function and silently deleted a critical edge case check. This is not a bug. This is the inevitable result of autonomous agents operating without cryptographic authority validation. When you have three agents working asynchronously across different files, two critical questions emerge: how do you enforce what each agent is authorized to do, and when multiple agents coordinate on shared resources, how do you maintain isolation between their operations? Policy based guardrails do not work at this scale. I have solved both problems through complementary cryptographic architectures. For individual agent authorization, my 13 layer cryptographic governance system validates AI agent authority mathematically before execution. For multi agent coordination, my YIN COLLAB architecture implements Agent Specific Compliance Tokens with per agent privacy isolation maintaining cryptographic boundaries preventing cross contamination. Every action carries immutable proof of authorization. Every decision boundary is pre validated cryptographically. 26 USPTO patents. 2,330 claims. Validated with 640x timing resistance and 500 plus concurrent agent support with sub 15ms latency. Mathematical proof, not policy promises. For developers using Antigravity, Cursor, or any autonomous agent platform: the question is whether you can prove mathematically that unauthorized operations are impossible and that multi agent coordination maintains isolation. Because as agent orchestration becomes the dominant development paradigm, the liability surface expands exponentially. One misconfigured agent with access to production systems is an organizational failure. One agent leaking sensitive data to another agent through shared context is an architectural vulnerability. Making it mathematically impossible for agents to violate boundaries is the only governance model that scales. Autonomous agents are the future of software development. Cryptographic governance is the only way to make that future safe. https://lnkd.in/ejPREk9D #GoogleAntigravity #AIGovernance #AutonomousAgents #Cybersecurity #AIAgents #DeveloperTools #CryptographicSecurity #ZeroTrust #AICompliance #SoftwareDevelopment #TechInnovation #AIEthics

2

Why "On-Device" is the most ethical AI strategy. We talk a lot about the size of models. Parameters. Trillions of tokens. The race for AGI. But in my academic life at EPFL and my work in Speech, I’ve always been obsessed with the opposite end of the spectrum: Accessibility. When we published our research in Nature Communications on AI for social projects, one conclusion was clear: If AI requires a massive data center and a high-speed fiber connection to run, it excludes half the world. Cloud models are undeniably impressive and they are pushing the boundaries of what is possible. But they shouldn't be the only way we deploy intelligence. I'm seeing a resurgence of interest in on-device solutions, not to replace the cloud, but to handle the sensitive, immediate, and personal tasks where the edge shines: - Privacy: Processing data right where it’s created. This is crucial for multimodal data (voice, video) that is highly identifiable and shouldn't leave the user's hand. - Latency: Real-time processing that feels like a conversation, not a transaction. - Energy Efficiency: Running a small, optimized model locally is often far more sustainable than round-tripping every query to a power-hungry data center. - Reliability: True accessibility means working offline. AI should be helpful even when the signal drops or the fiber hasn't been laid yet. Better locally-run alternatives make these powerful tools accessible to everyone, even for problems and data that are highly sensitive. I’m planning to dive deeper into this myself by experimenting with some of the latest open-weights models specifically for local deployment. If you’ve been testing small language models (SLMs) or edge-optimized architectures recently, I’d love to hear about your experience. What frameworks or models are you finding most promising for mobile/edge/small server hardware right now? #AIforGood #OnDeviceML #NLP #EthicalAI #TechForGood #EdgeAI #SustainableTech #OpenWeights

7

6 Comments

🚀 JSON is great — but not when you're prompting LLMs 🤖 . I just published a new article introducing TOON — a cleaner, safer, and more reliable alternative to JSON for structured AI outputs. Why TOON is worth knowing: ✅ Human-readable and indentation-based ✅ Less syntax-fragile (no commas, no quotes drama) ✅ Reduces hallucinated fields ✅ More consistent outputs in real prompts ✅ Easier to debug and scale If you're building with LLMs, agents, or structured outputs, this might save you hours of frustration 👇 🔗 https://lnkd.in/g9Tujy8P Would love your thoughts! 💬 #LLM #PromptEngineering #AI #Developers #JSON #TOON

19

2 Comments

Claude just released a memory import tool: https://lnkd.in/eM8c7Gf5 You paste a prompt into ChatGPT (or whatever you're using), it spits out your context — preferences, projects, how you work — and you drop it into Claude's settings. That's it. Claude picks it up from there. The switching cost has always been the real barrier. Not the AI itself, just the fact that you've spent months building context somewhere else. This is Anthropic admitting that out loud and actually doing something about it. Quiet release. Worth knowing about.

2

Want to train LLM to reason right on your mac? Check out my article on Reinforcement Fine-Tuning LLMs with GRPO on Apple silicon using MLX framework. I remembered vividly when DeepSeek-R1 burst onto the scene this year, it dominated discussions over Chinese reunion dinners, AI stocks crashed, and friends I knew (including me) were busy asking it to reason their Bazi. A key idea introduced along with Deepseek R1 was Group Relative Policy Optimisation (GRPO), which helps to incentivise reasoning capability in LLMs via reinforcement learning. Since then, there have been courses such as Predibase's course on implementing GRPO, and improvements to the algorithm such as Mistral AI's Magistral LLM which was trained on a more efficient GRPO algorithm. Following the course by Predibase hosted on DeepLearning.AI, I have written an article where I explain step by step each component of GRPO equation, with a worked example to illustrate the algorithm. If you have on hand an Apple silicon computer, I have also included a Jupyter notebook which implements GRPO using Apple's MLX framework. Credits to Awni Hannun for creating the base notebook for LoRA Fine-Tuning with MLX LM. Full disclosure that I created the article and the notebook with the assistance of Google Gemini. If you noticed any errors with either the article or the notebook, please let me know or file a pull request.

889

39 Comments

In my last blog, I covered Distributed Inference Serving for LLMs. This time, I’ve written about training, breaking down DDP vs FSDP and what to use when for large-scale deep learning. Link: https://lnkd.in/gcgtSZEk #AI #DeepLearning #DistributedTraining #DDP #FSDP #PyTorch #MLTraining #Scalability #MLOps #GPUComputing #AIInfrastructure

10

1 Comment