Ming-Chih Hsieh

Yue Yin, Software Engineer at ByteDance, explains how the Gödel scheduler differs from the Kubernetes scheduler through its architecture. She details how Gödel introduces scheduling units for batch processing and implements a two-layer scheduling framework: job-level for filtering nodes, and pod-level for detailed placement. The system uses a distributed architecture with three components — dispatcher, scheduler, and binder — enabling parallel processing through multiple scheduler shards. Watch the full episode: https://ku.bz/lMpNng_33

1

1 Comment

Predicting the breaking point before the first line of code runs. 📉 What if you could run a performance test without an environment? Chun Lin Goh is challenging the status quo of k6 and JMeter. In his latest talk, he demonstrates how to use Discrete Event Simulation to model system behavior at scale. Key Highlights: - Why "Brute-Force" testing is failing modern DevOps teams. - A Case Study: Predicting Amazon RDS T3 throttling via simulation. - How to shift-left performance testing into the design phase. If you’re tired of reactive "break-and-fix" cycles, Chun Lin’s approach is a game-changer. #AWS #SoftwareArchitecture #LoadTesting #DevOps #GohChunLin #TechInnovation #taqelahsg #TaqelahLightningTalks2026

6

3 Comments

☁️ 96% of companies believe that most of their data will be cloud-based within 5 years. Why? Because cloud solutions are no longer just about storage — they’re about: 📊 Real-time analytics for faster decisions 🔒 Stronger security and compliance options ⚡ Scalability without massive infrastructure costs 🤝 Easier collaboration across global teams At Glorium Technologies, we see this shift every day — from healthcare to finance, businesses are moving critical operations into the cloud to stay competitive. The question isn’t if you’ll migrate. It’s how fast you can do it — and how smooth the transition will be. #CloudComputing #DigitalTransformation #DataDriven #GloriumTechnologies

4

1 Comment

Tip: Scale large HNSW vector sets in Redis by splitting your dataset across multiple instances and querying them in parallel with multiplexing, enabling efficient parallel reads and writes that drastically improve performance and manage memory effectively. This approach leverages Redis's scalable vector support to handle hundreds of millions of vectors seamlessly. #Redis #HNSW #VectorSearch #ScalableAI #DataScience

1

Deep Dive: Kafka Messages with KafkaKeeper Discover how KafkaKeeper gives you complete visibility into your Kafka messages - from real-time message browsing and filtering to payload inspection and partition-level insights. No blind spots. No unnecessary complexity. Just real-time message exploration built for modern data teams. Explore how KafkaKeeper transforms message monitoring, debugging, and operational control - all from a single lightweight dashboard. 🎥 Watch the demo to see how you can browse live messages, filter events instantly, inspect payloads, and troubleshoot Kafka pipelines in minutes. 👉 Try on AWS Marketplace: https://lnkd.in/gTuPQKCb 👉 For More Product Details Visit - https://lnkd.in/dwghBmj4 📩 For more details contact us at - info@datatroops.io #KafkaKeeper #DataTroops #ApacheKafka #KafkaMonitoring #Observability #AWS

7

Introducing pgraft: Raft Consensus Protocol for PostgreSQL Achieve true high availability with automatic leader election, zero split-brain scenarios, and crash-safe replication. Built on the same etcd-io/raft engine that powers Kubernetes. Production-ready for PostgreSQL 16, 17, and 18. #PostgreSQL #HighAvailability #RaftConsensus #DatabaseClustering #DistributedSystems #pgElephant #DevOps #SRE #DatabaseEngineering #OpenSource #CloudNative #Kubernetes https://lnkd.in/gcMDmRfi

1

Kafka Performance: Key Tuning Tips In real-time systems, every millisecond matters, and with Kafka, the right tuning can be the difference between smooth streaming and costly lag. From optimizing partition counts to adjusting batch sizes, compression, and retention settings, each tweak impacts speed, scalability, and stability. But manual tuning is time-consuming and error-prone. That’s where #Condense changes the game, giving you a visual, intelligent interface to fine-tune Kafka without deep-diving into complex configs. Faster pipelines, fewer headaches, and performance that scales with you. 🔗Learn more: https://lnkd.in/gEWNJHcf 🔗Explore more: https://lnkd.in/gnXu_bYP #KafkaPerformance #ApacheKafka #RealTimeStreaming #KafkaTuning #DataPipelineOptimization #CondensePlatform #Zeliot

13

Building Near Real-Time Bus Tracking App with Aiven Kafka Tan Ze Han Dylan UPM and I built near real-time pipeline for tracking Rapid KL Bus, powered by Kafka on Aiven #FreeKafka and a Lambda Architecture pipeline. 📍 Kafka (Aiven) for real-time ingestion 🤖 Spark Streaming + Kafka Connector for processing 🚦 AWS S3 + Iceberg for the lakehouse  ⚡Cache & serve near real-time state with Redis ⚙️ Kubernetes + Prefect for orchestration 🗺️ LeafletJS + Next.js for live map visualization This will be our submission for Kafka competition hosted by Aiven. Enjoy the Demo!

225

12 Comments