Vinay Rayini

"For these models to reach their peak value, you need to train them not just on publicly available data, but make privately owned data available for those models as well," - Ellison https://lnkd.in/gM6FskzJ

4

H1B Visa fees 100,000 Dollar. Time to awake indians. No other way then to get out of delusion of development. We should focus more on economy then cultural activities going on now a days. This visa row is indication of following 1. AI is really on the way to destruct and change IT sector forever. 2. Jobs placements will be lower then expected. 3. Even if this visa fees is reduced then also there will be an uncertainty on the AI changing the world order. 4. Narratives build that now india will be great again, is false i think. 5. Only way to avoid heavy losses is to truly invest in AI research.

18

Managing and scaling big-data infrastructure is never trivial—Pinterest just reminded us how complex this gets at scale. They recently revealed their internal orchestration framework, Hadoop Control Center (HCC), which automates the scaling and migration of Hadoop clusters running on AWS. I’ve seen how manually tweaking resource sizes, moving data across instances, and managing spot instance interruptions can be a nightmare. So hearing Pinterest built a system that pre-generates execution plans, handles AWS EMR quirks, and even manages cloud-to-cloud migrations—this is engineering maturity. Interesting bit: HCC is declarative, meaning engineers only specify what state they want their Hadoop cluster in, and HCC figures out how to get there. It’s like Kubernetes, but for Hadoop. Would love to see if other companies follow suit—or open source something similar. What’s your take on this kind of orchestration automation? #BigData #DevOps #CloudComputing #InfrastructureEngineering #Hadoop Read Article: https://lnkd.in/daMEGwmf

6

Microsoft Just Made 100B Parameter LLMs Run on Your Laptop. Who Governs Them? BitNet breaks the GPU barrier. 1.58 bit quantization. 100 billion parameters on a single CPU. No cloud. No API costs. Local inference at human reading speed. This is a genuine breakthrough for accessibility. But accessibility without governance is liability. Running locally solves the infrastructure problem. It does not solve the governance problem. A 100B parameter model on your laptop can still hallucinate with confidence. It can still produce biased outputs. It can still make decisions with no audit trail. It can still violate constitutional bounds you never defined. Privacy from cloud providers is not the same as privacy from the model itself. Local execution means your data stays on your device. It does not mean the model operates within governance constraints. The question is not where the model runs. The question is whether the model can prove it operated within bounds. YIN-GATEWAY solves this. YIN-GATEWAY is a cryptographic control plane that sits between any LLM and its users. Cloud model, local model, quantized model, full precision model. The deployment changes. The governance requirement does not. Layer 2 Admissibility Gates reject non-compliant queries before execution. The model never sees requests that violate policy. Layer 3 Constitutional Bounds verify every output before it reaches users. If the output would violate governance constraints, it does not proceed. Layer 14 SENTINEL Verification runs independent binary checkpoint validation. Pass or fail. No partial compliance. Layer 15 Human Override requires cryptographic signatures before high-risk outputs proceed. Not a checkbox. A mathematical proof that a human authorized this specific action. BitNet gives you a 100B parameter model on your CPU. YIN-GATEWAY gives you cryptographic proof that the model operated within constitutional bounds, regardless of where it runs. The hardware revolution is here. The governance layer was already built. https://lnkd.in/eb2EdEwR 32 USPTO applications. 3,057 claims. Any model. One governance layer. Enforced behavior. #BitNet #YINMazariArchitecture #YINGATEWAY #AIGovernance #CryptographicCompliance #Microsoft #LocalAI #LLM #PrivacyPreserving #ResponsibleAI #EdgeAI #CPUInference #OpenSource

4

1 Comment

𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗜 - 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝘀𝗼𝗹𝘂𝘁𝗶𝗼𝗻 𝗳𝗿𝗼𝗺 Snowflake Recently, I took a deep dive into Snowflake’s new product—𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗜. https://lnkd.in/g7HDWChB It’s extremely useful for building AI applications when your data is already stored within Snowflake. My first hands-on experience with Snowflake was about eight years ago, when we migrated a reporting system from Oracle to Snowflake. I found Snowflake particularly valuable because it was almost completely compatible with Oracle SQL dialect, making the migration process straightforward. Additionally, its columnar storage architecture and software-as-a-service model provided superior performance for OLAP (Online Analytical Processing) systems. Cost was also a significant advantage compared to Oracle. Since then, Snowflake has become the de facto standard for online reporting systems, with many companies adopting it for a wide range of projects. In one of my projects, I needed to run an 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗔𝗜 𝗮𝗽𝗽𝗹𝗶𝗰𝗮𝘁𝗶𝗼𝗻 on top of data stored in Snowflake to deliver actionable insights. After reviewing multiple approaches, I decided to use Snowflake’s 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗜 Agentic solution. Key Features of Cortex AI in Snowflake 1. 🧑‍💻 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗜𝗦𝗤𝗟 𝗙𝘂𝗻𝗰𝘁𝗶𝗼𝗻𝘀
Built-in generative AI functions (e.g., AI_COMPLETE, AI_CLASSIFY) for advanced tasks with text, images, and audio—directly in SQL, no external services or data movement needed. 2. 🤖 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗻𝗮𝗹𝘆𝘀𝘁
Converts natural language queries into accurate SQL using semantic models, executes them, and returns structured insights automatically. 3. 🔍 𝗖𝗼𝗿𝘁𝗲𝘅 𝗦𝗲𝗮𝗿𝗰𝗵
Advanced multimodal retrieval-augmented generation (RAG): embeds documents, performs semantic + keyword search, and delivers relevant results to LLMs. 4. 🕹️ 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗴𝗲𝗻𝘁𝘀
Orchestrates complex workflows by combining Analyst, Search, and AISQL functions. Enables multi-step analytics and chatbot flows across structured & unstructured data. Simple REST API integration with Streamlit, Slack, Microsoft Teams, and more. 5. 📊 𝗔𝗜 𝗢𝗯𝘀𝗲𝗿𝘃𝗮𝗯𝗶𝗹𝗶𝘁𝘆
Brings transparency and trust to LLM applications with integrated evaluation and tracing tools for monitoring, debugging, and performance optimization. 6. 🛡️ Governance
Ensures security and compliance with RBAC, audit logging, and model access controls—all managed within Snowflake’s secure perimeter. 7. 🤝 𝗔𝗰𝗰𝗲𝘀𝘀 𝘁𝗼 𝗟𝗲𝗮𝗱𝗶𝗻𝗴 𝗟𝗟𝗠𝘀
Supports top-tier large language models including Claude, DeepSeek, Llama, GPT-4, and Mistral. 8. 📈 𝗧𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗠𝗮𝗰𝗵𝗶𝗻𝗲 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴
Offers a strong suite of classic ML algorithms for anomaly detection, forecasting, classification, sentiment analysis, and more. 𝗠𝘆 𝗳𝗮𝘃𝗼𝗿𝗶𝘁𝗲 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 is the 🤖 𝗖𝗼𝗿𝘁𝗲𝘅 𝗔𝗻𝗮𝗹𝘆𝘀𝘁 tool, which lets you interact with structured data in your tables using natural language. #AI #AgenticAI #Snowflake #CortexAI

29

5 Comments

Ruokai Yin, our summer intern from Yale to Microsoft Azure, worked with Xuan Zuo, @Sattwik Deb Mishra, Hokchhay T. and Preyas Shah and me. We published a paper based on his project: https://lnkd.in/eHuG3BSt Summary of paper: Finding optimal sharding is hard especially for large LLMs. A large number of devices have to be divided into TP, PP, EP dimensions. For TP, we have to decide the tensor dimension e.g. head, hidden size to shard for every op. Additionally, TP can be one-dimensional/two-dimensional etc. This makes for a large sharding space that is hard to cover exhaustively. SoTA methods use heuristics to select sharding e.g. GSPMD, Sequence Parallelism, Helix etc. There are significant time gaps, months or years, between the discovery of each of these methods. We want to fast-forward the process of discovery by 1) using a formal representation of the sharding space, and, 2) using reinforcement learning to search the sharding space.    This paper demonstrates that our method is 3.5x better than random search, and on par with hand-tuned heuristics when looking only at 1D sharding and without allowing sequence parallelism. We aim to extend the work to multi-dimensional sharding as well as sequence parallelism. Beyond that, we wish to combine the problem of sharding with fusion, tiling and collectives as well.

58

Tata Digital slashed cloud costs by 80%+ — without slowing down. We fixed costly defaults, trimmed oversized clusters, and optimized storage to build a lean, high-performing data platform at scale. Must-read for anyone serious about cloud cost control over varied platforms, scaling for storage and processing.

30

5 Comments

You are in a system design interview at Amazon for an SDE-3 role and the interviewer has given you the question to Design Netflix. He then asks a follow up to you: How does Netflix knows when to show: Are you still watching? Is it just time-based? If it were time-based, you would see it every time. So what is actually going on under the hood? Both look like a simple pop-up. Underneath, it is a mix of product thinking, client logic, and backend events. Btw, if you’re preparing for system design/coding interviews, check out our mock interview tool. You can use it for free here: https://lnkd.in/gpCn7t2T [1] Start from the product goals The feature is not only about nagging people. - Save bandwidth and CDN cost if the viewer has slept or walked away - Avoid autoplaying potentially sensitive content in an empty room - Protect kids if parents start a show and leave the TV on - Do all this without annoying active binge watchers So the design must answer one question: "When is the user probably not here any more" [2] Naive design - pure timer Simplest idea: - If playback has been running for 2 hours, show Are you still watching - If user clicks "Yes", reset the timer Why this is weak: - Someone can binge 5 episodes in a row and gets interrupted in the middle of an intense scene - Someone can start a show, walk away after 5 minutes, and the platform will keep streaming for the next 2 hours - It ignores how many episodes were auto played, device type, time of day, user habits  [3] Realistic design - session and engagement signals Think in terms of a "watch session" and "engagement events". Signals the client can track: - Play, pause, seek, volume change - Episode finished and next episode auto started - Remote or keyboard input, UI navigation - Screen on or off events from the device - For mobiles: app background or locked state A common heuristic could be: - Count how many episodes have auto played without any user interaction - Track how long it has been since the last button press or navigation - Only trigger the prompt when both are high enough Example rule: - If 3 episodes in a row have auto played - And there has been no interaction for 45 minutes - Then, before starting episode 4, show Are you still watching This feels much smarter: - Active viewers who pause, skip intro, change volume keep resetting the "engagement clock" - Sleeping viewers do not touch anything, so the next episode is blocked by the prompt  [4] Client heavy vs server heavy design You can talk about two design choices. Client first: - All logic runs inside the app on TV, mobile, or web - Client keeps an in memory session model and decides when to show the popup - It still sends telemetry events to backend for analytics and future tuning Pros: - Works even with flaky network - Highly responsive, no extra server round trip Cons: - Logic must be implemented and updated across many platforms - Harder to quickly roll out rule changes Continued ↓

184

12 Comments