About
Specializing in efficient data structures and algorithms for large-scale distributed…
Activity
641 followers
-
Konstantin V. Shvachko shared thisThe Azure Blob Storage team at Microsoft is starting an exciting project to develop a new cloud storage system dedicated to #AI and #ML workloads. If you are passionate about #DistributedStorage at extreme #scale, #reliability, and #performance, and have experience in #VectorSearch and #OpenSource software development, this is a great time to join the team. Please feel free to contact me and Vishwajeet Dusane for more information.Konstantin V. Shvachko shared thisWe are looking to #hire a Principal Software Engineer to Join our Azure Blob Storage Team led by @Vishwajeet Dusane. • Help shape system architecture for AI storage, drive technical innovation across Design & Operations, address complex engineering challenges at hyperscale, and guide teams in building high-quality solutions! • If you currently have extensive experience designing & developing reliable, high-performance distributed systems with a strong focus on Scalability and Resilience #letsconnect! #principalsoftwareengineer #azurecorejobs #azurejobs ##MicrosoftCareers #TechJobs #CloudComputing #DistributedSystems #AzureStorage #Kubernetes #AIEngineering #JoinOurTeam https://lnkd.in/gFXaNDfm
-
Konstantin V. Shvachko shared thisJensen Huang is interviewing the #QuantumComputing industry following his provocative claim that the #technology is 20 years away. Exciting panel on #Quantum #Computing at NVIDIA #GTC25. I liked his explanation for the 20-year estimate - it is how long it took NVIDIA to make GPUs practical for #AI application. It was great to see many key players in the area and learn about different approaches to and the state of the art of Quantum Computing. A replay is anticipated here: https://lnkd.in/gbuCBpRz
-
Konstantin V. Shvachko shared thisJust completed my study/project on #QuantumComputing. Excited to share my notes on the topic, published in USENIX ;login:. I tried to make the content readable by programmers and non-experts in quantum mechanics. Being an enthusiast of quantum computing for a while I was curious how practical the technology is today. I found it increasingly difficult to separate the #computing nature of the subject from "science fiction" and excitement on the one hand, and convoluted terminology of quantum physics on the other. I started collecting insights for myself, eventually realizing it could be useful for others. The article is a rather lengthy read as it touches upon many related topics. It provides essential references supporting the facts and for further education. For better reading experience download a pdf version: https://lnkd.in/gzCJnhhF Big thanks to my supportive family, my former colleagues for discussions, and Rik Farrow for efficient shepherding of the publication. tinyurl.com/shv-quantum #softwareengineering #programmingQuantum Computing Notes: Why Is It Always Ten Years Away?Quantum Computing Notes: Why Is It Always Ten Years Away?
-
Konstantin V. Shvachko posted thisGlad to announce that I recently moved to Colorado from California. Exited to be seeing more of my growing family and climbing more 14ers. Sadly, I was informed that my job role is not "compatible" with the relocation. Taking this opportunity to let my colleagues and friends know that I will be leaving Walmart Global Tech in August.
-
Konstantin V. Shvachko shared thisA fascinating story about the history of the revolutionary transformers paper "Attention Is All You Need” in #generativeai. Turns out the name is a transformed Beatles' song title "All You Need Is Love". It started as a lunch conversation, grew into a side project and culminated in a paper, which was accepted to a conference as a poster and became a key event of the forum. Internally the results were seen as "an interesting #AI project", while OpenAI realized the potential and capitalized on the ideas. All eight authors left Google to build startups. I am sure they all had fun with the project - and that is what counts.8 Google Employees Invented Modern AI. Here’s the Inside Story8 Google Employees Invented Modern AI. Here’s the Inside Story
-
Konstantin V. Shvachko shared thisKudos to Richard Williamson and Ethan Kuehl presenting serverless Spark use cases at Walmart scale. #googlecloudnext
-
Konstantin V. Shvachko shared thisInsightful session at #Googlecloudnext. Urs Hölzle, Eric Brewer, and Jeff Dean (virtually) sharing stories about history of technology from 1999, and predicting changes for the next five years.
-
Konstantin V. Shvachko shared thisSam Guleff in his blog compares popular #datalake formats for building Lakehouse architecture - Apache Hudi, #DeltaLake, and Apache Iceberg. Guessed the winner? Two differentiators from other comparisons: * It is based on Walmart Global Tech production #cloud workloads. * Puts a high focus on cost of ownership. https://lnkd.in/eR74W6eF
-
Konstantin V. Shvachko shared thisBuilding many interesting things at Walmart Global Tech with #bigdata team on #gcp and #azure. Ping me if you want to learn more.Konstantin V. Shvachko shared thisWe're hiring! Our team manages the #bigdata #hadoop , #spark , and #airflow systems at Walmart Global Tech, and are looking for strong technologists with skills in: #bigdata, #spark, #k8s, and #airflow who can take on the role of a Principal Software Engineer! Come help us push the boundaries of technology to make an impact on a global scale!
-
Konstantin V. Shvachko reacted on thisKonstantin V. Shvachko reacted on thisLast week, it was fascinating and inspiring to hear about the initiatives being led by young minds of India, such as Pankhuri Shah and Rakesh Sehgal , to uplift the underprivileged at The/Nudge Institute ’s annual get-together. Their work is commendable and worth learning about. It was an evening well spent with great company. Sowmya atul satija Vasanth Rajamani Jagadish Venkatraman Manohar Paluri Ashwini G. Priya Ramachandran Kalyani Nune Archana Dubey MD Ari Daman
-
Konstantin V. Shvachko liked thisKonstantin V. Shvachko liked thisLinkedIn Data Infra is excited to be collaborating with South Bay Systems (https://lnkd.in/gvdG8z8x) to host Cliff Click next Thursday at our Mountain View HQ for a talk on Simple (https://lnkd.in/g4Q9Kzqn), a teaching language meant to showcase the Sea of Nodes compiler IR, as used in HotSpot's C2 compiler and Sun/Oracle's Graal compiler. As an engineer working mostly in the JVM where performance is completely dependent on JIT compilation, I'm personally looking forward to understanding more about the ideas underpinning this crucial technology. Attendance is free and open to all but requires registration on Luma: https://luma.com/nnq9aq27 Looking forward to seeing you there!South Bay Systems: A Simple Guide to Sea of Nodes · LumaSouth Bay Systems: A Simple Guide to Sea of Nodes · Luma
-
Konstantin V. Shvachko liked thisKonstantin V. Shvachko liked thisAre you prepared for the post-quantum computing world? Unlike Y2K, this transition demands agility. Delaying preparation will only make it more challenging and costly. Mark and I discuss how IBM is preparing for this new era and how IBM can assist you in being ready. For more details, check out the link in the comments. Vishal Jai Dimple Michele Dinesh Ray Koos Alessandro Michael Scott #IBMQuantum #IBMSecurity #PQC
-
Konstantin V. Shvachko reacted on thisKonstantin V. Shvachko reacted on thisA couple of weeks ago, I had the privilege of speaking at The Tech Interactive's National AI Literacy Day event representing IBM . It was inspiring to hear from the passionate individuals dedicated to leveling the playing field as we navigate the world of AI. AI accelerates everything, both good and bad, and the responsibility lies with us. I would like to extend my gratitude to Katrina for including me in this wonderful event and to Kasey (Van Ostrand) for the support. It was an honor to connect with Isabelle, Sarahi, Anupama, the Stanford Accelerator for Learning Create+AI awardees, and many others. #aiforall #ailiteracy #ibmer #ibmlife
-
Konstantin V. Shvachko reacted on thisKonstantin V. Shvachko reacted on thisMatei Zaharia, associate professor of UC Berkeley Electrical Engineering & Computer Sciences (EECS) and a cofounder and CTO of Databricks, has been awarded the ACM, Association for Computing Machinery Prize in Computing! His visionary development of open-source systems has helped enable large-scale machine learning, analytics and AI at a global scale. Learn more: bit.ly/4vbNujK (Photo © Drew Kelly) University of California, Berkeley | UC Berkeley College of Computing, Data Science, and Society
-
Konstantin V. Shvachko reacted on thisKonstantin V. Shvachko reacted on thisA lifelong dream of mine came true around 4am this morning when I stood a few miles away and watched a Falcon 9 take code that I wrote into space. I am truly awestruck. Words fail me. Time for a little sleep, then I'm looking for my next challenge.
-
Konstantin V. Shvachko liked thisKonstantin V. Shvachko liked this🚀 We’re Hiring: Senior Software Engineer – Azure Storage Join Microsoft’s Azure Storage team and help build the foundation of the cloud. You’ll design and deliver high-performance, scalable, and reliable storage systems that power mission-critical AI workloads worldwide. We’re looking for engineers who: Thrive in distributed systems and large-scale infrastructure. Enjoy solving challenges in availability, latency, and fault tolerance. Want to make an impact at global scale. 📍 Location: United States 👉 Apply here: [https://lnkd.in/g3D-Y8Rc] #Hiring #SeniorSoftwareEngineer #AzureStorage #AIStorage #CloudEngineering #DistributedSystems #ScalableSystems #MicrosoftCareers
Experience
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
Education
-
Lomonosov Moscow State University (MSU)
-
-
Activities and Societies: . Faculty of Mechanics and Mathematics, Mathematical Logic and Theory of Algorithms
-
-
-
Licenses & Certifications
Patents
-
Distributed file system using consensus nodes
Issued US US009424272B2, US009361311B2
Projects
-
Apache Hadoop
- Present
Apache Hadoop is an open-source software framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale out from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the framework itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a…
Apache Hadoop is an open-source software framework that allows for the distributed processing of large data sets across clusters of computers using a simple programming model. It is designed to scale out from single servers to thousands of machines, each offering local computation and storage. Rather than rely on hardware to deliver high-availability, the framework itself is designed to detect and handle failures at the application layer, so delivering a highly-available service on top of a cluster of computers, each of which may be prone to failures.
Other creators -
-
Apache Hadoop
-
Apache Hadoop is a framework for distributed processing of large data sets across clusters of computers using simple programming models. It is designed to scale out from single servers to thousands of machines.
Other similar profiles
Explore more posts
-
Michael Bussieck
I’m excited to share a new benchmark, MIPfeas, which shifts the focus in Mixed-Integer Programming from "final optimality" to "early feasibility." Using the Primal Integral metric, we evaluated how solvers like HiGHS, SCIP, and NVIDIA cuOpt deliver high-quality feasible solutions within a short time window. While direct comparison with commercial MIP codes remains difficult, we have grouped the results of three major commercial solvers—COPT, CPLEX, and Xpress—into three synthetic baselines: the Virtual Best, Mean, and Worst Commercial Solvers in this benchmark. A huge thank you to Hans Mittelmann, the NVIDIA team (Burcin Bozkaya, Akif Çördük, and Chris Maes), and ZIB Zuse Institute Berlin for the technical collaboration. Special thanks also to Timo Berthold, Gerald Gamrath, and Ferenc Katai for their expert insights. Check out the full results here: https://lnkd.in/dSfNc58v #Optimization #MIP #GAMS
11 Comments -
Alex Reid
Zig has fantastic interop with C. Excel has a C SDK that's heavy on boilerplate and memory ownership rules, making it tricky to work with. Zig comptime can solve some of those problems with no runtime penalty. So I thought... why not hammer together some new fangled tech with some jurassic-era tech from the 90s? The result is a package called zigxll. This was something of an ADHD segue when working on xllify.com, which is written in C++. No plans to switch that to Zig as it is a moving target that is hard to keep up with. However, comptime and the explicit passing of allocators around is 👌 ... so maybe one day. An example project that uses the package is here: https://lnkd.in/ejnNT63i - see, not much code! The package itself lives here: https://lnkd.in/eF_gw4xg #excel #zig #customfunctions #comptime
-
Marcus Swift
Despite recent hype, most LLMs fall short when it comes to reproducing and resolving real software bugs. 📉 On SWT-Bench (2024–2025) — a benchmark specifically designed for LLM-based bug fixing — even the best systems show underwhelming results: Amazon Q manages just 45% F→P (fail-to-pass) on the verified set. Ensemble models like Otter++ hover at 37% or lower. And even when combining multiple models, the best hybrid systems only reach 72.2%, requiring heavy orchestration and model stacking. These results make it clear: current LLMs alone aren’t cutting it for reliable, autonomous debugging. 🛠️ That’s why we’re introducing Error Solving — a new technique purpose-built to handle error resolution in code, especially where traditional LLM pipelines fail. You can download it now as a modular MCP (Machine-Code Protocol) here: 👉 https://lnkd.in/gxiyFw4b Please comment, star and provide feedback!
1 Comment -
Nishantha Ruwan
The paper identifies a critical performance limitation in multi-turn, agentic large language model (LLM) inference workflows: the key-value (KV) cache I/O from external storage becomes the dominant bottleneck as context lengths grow. In traditional disaggregated inference systems, prefill engines (which generate KV cache data) saturate their storage network interfaces while decode engines sit underutilized, leading to poor throughput and inefficient resource use. To address this, the authors introduce DualPath, a novel inference architecture that adds a second KV-cache loading route directly from storage to the decode engines. Once loaded, the KV cache can be efficiently transferred to prefill engines via high-speed RDMA over the compute network, reducing pressure on the storage network and avoiding congestion with latency-sensitive computation traffic. A global scheduler dynamically balances workload across the prefill and decode engines to optimize utilization and performance. When evaluated on three different models under production-like agentic inference workloads, DualPath significantly improves throughput: up to 1.87× for offline inference scenarios and roughly 1.96× for online serving without violating service-level objectives (SLOs). These results suggest that reorganizing data flows at the system level can meaningfully break storage bandwidth barriers in large-context LLM deployments. https://lnkd.in/gFc7N99w
-
Jonathan Ellis
One of the challenges with the Power Ranking is showing 3 different metrics (Score, Cost, Latency) with different units. Maybe it's as simple as "normalize to 100 and show them all as bars." https://lnkd.in/g9pZmRwY WDYT, better or worse than the current system of graph-per-metric?
5 Comments -
Christoph Pfister
Did anybody else parse the Mag7 reports and find that: -- AWS Trainium chips are on fire -- Google sees "tremendous demand" for their TPUs -- Meta will significantly expand use of their MTIA chip All that on top of the news from a few weeks back that OpenAI will build their own chips with Broadcom. At the same time Nvidia is investing in inference (either directly or indirectly), network infrastructure (Nokia), and even the agentic layer (Poolside). Microsoft does their OpenAI thing, Tesla is Tesla, and who knows what Apple is up to. Bottom line: None of these companies are staying in their lane, and there will be tremendous competition over the next few years. Place your bets. Happy Halloween 🎃
13 Comments -
Boyuan Chen
28.6% resolution rate on real Rust repository issues. That is the current ceiling for LLM-based coding agents. Rust-SWE-bench tests agents on 500 real GitHub issues from 34 Rust projects. The failure analysis is where it gets interesting. 44.5% of tasks fail at the issue reproduction stage. The agent never even gets to write a patch. It cannot set up the environment, cannot reproduce the bug, cannot run the tests. Nearly half the failures have nothing to do with code generation ability. For the other half, compilation errors come from two sources: failure to model repository-wide code structure, and failure to comply with Rust's strict type and trait semantics. When the required patch exceeds 150 lines, the gap between agent architectures widens dramatically. This tells us something important about where the real bottleneck is. We keep optimizing for "can the model write better code." But in strongly-typed, real-world codebases, the harder problem is everything around the code: environment setup, dependency resolution, cross-file navigation, and compiler feedback loops. RUSTFORGER addresses this by adding automated test environment isolation and Rust metaprogramming-driven dynamic tracing. It pushes resolution from 21.2% to 28.6%, and uniquely solves 46 tasks that no other agent could solve across all LLMs tested. For anyone building coding agents: if your evaluation only covers Python, you are measuring the easy part. Strongly-typed languages expose whether your agent actually reasons about code structure or just pattern-matches on syntax. Paper: https://lnkd.in/ea6EmRBs #LLM #CodingAgents #Rust #SWEBench #SoftwareEngineering #AIEngineering
2 Comments
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content