Tim Callaghan

People often ask me about the differences in architecture between Amazon Dynamo (the 2007 SOSP paper), DynamoDB (the AWS serverless NoSQL database), and Aurora DSQL (the AWS serverless SQL databases). I captured the key points on durability, high availability, programming model, and trade-offs in my blog: https://lnkd.in/gkMG_tVD TL;DR: Despite the shared name, Dynamo and DynamoDB have little in common architecturally, and make very different trade-off decisions.

631

8 Comments

On October 19–20, 2025, AWS services in the us-east-1 (N. Virginia) region suffered a major outage after a race condition in DynamoDB’s DNS system deleted critical DNS records. This disrupted DynamoDB and caused cascading failures across EC2, NLB, Lambda, Redshift, and other services. Full recovery was completed by 2:20 PM PDT on October 20. AWS is fixing the DNS flaw and strengthening recovery, throttling, and failover mechanisms to prevent similar incidents. https://lnkd.in/gZh4A75A

1

2 Comments

Minimizing the dependencies of distributed systems, especially the fundamental building blocks, is one of the hardest but most crucial aspects of system design. It’s too easy to end up with cycles, typically transitive dependencies three or more nodes deep.

4

1 Comment

How Uber works around Apache Kafka's pain points (basically they centralize all Kafka client logic in this uForwarder proxy and have it make gRPC calls to all registered consumers) https://lnkd.in/eCHbDaYC

4

Wittling a retrospective of The Apache CMS as it related to the Apache OpenOffice project… When it’s further along I will share URL, but in the meantime if you have any anecdotal commentary to add please reach out.

Exciting news for Amazon MSK users! AWS Lambda now natively supports Avro and Protobuf formatted events from Apache Kafka, including Amazon MSK, Confluent Cloud, and self-managed clusters—all in Provisioned Mode. This advancement marks a significant leap forward for event-driven architectures, as it eliminates the need for custom deserialization code and manual schema registry management. By integrating directly with AWS Glue Schema Registry, Confluent Schema Registry, or self-managed registries, Lambda can now validate and process Avro or Protobuf events seamlessly, allowing teams to focus more on delivering business value and less on infrastructure plumbing. The benefits extend well beyond simplified integration. Avro and Protobuf are renowned for their compact message sizes and rapid (de)serialization, offering notable performance improvements over JSON. With native support, Lambda users can choose to receive events in their original binary format for direct object mapping or as clean, validated JSON for easy integration across various programming languages. Tools like Powertools for AWS Lambda further streamline this process, making it effortless to work with business objects. Event filtering at the source is another standout feature. Teams can now set up rules to filter out irrelevant events before they reach Lambda functions, reducing costs and improving pipeline efficiency. When schemas evolve, Lambda automatically retrieves and applies the latest versions, ensuring functions always process up-to-date, validated data—with no code changes required. Robust error handling is also enhanced, as problematic events can be routed to #SQS, #SNS, or #S3 for debugging and analysis, keeping workflows reliable and resilient. #AWS #lambda #MSK #ApacheKafka #avro #protobuf https://lnkd.in/gutjCjkD

62

3 Comments