Mr.PlanB Logo

    Newsletter

    Subscribe our newsletter

    Get new infrastructure guides, comparison reports, and migration notes in your inbox.

    Infrastructure notes, guides, and new tools. Unsubscribe anytime.

    Back to Blog
    AWS
    Cloud
    Outage
    DNS
    US-EAST-1
    Infrastructure

    AWS us-east-1 Outage: Why Concentration Risk Still Matters

    October 20, 2025
    9 min read

    Early in the morning on October 20th, engineers across the globe got the kind of wake-up call no one wants. Their monitoring dashboards flatlined, alarms exploded in Slack (or didn't, because Slack was down too), and the AWS Health Dashboard delivered the ever-vague "operational issue" for the US-EAST-1 region. By then, the damage was done. Services that underpin huge swaths of the modern web were toast, and for hours, teams scrambled to figure out what had happened.

    The culprit was DNS, again.

    The irony is painful. The entire AWS infrastructure is supposed to be built around high availability. Spread across multiple Availability Zones (AZs) within regions, it's marketed as resilient by design. However, when US-EAST-1, Amazon's largest and oldest region, face-planted this hard, it exposed something the tech community has whispered for years: US-EAST-1 carries far more weight than any other AWS region.

    How one DNS record wrecked the party

    At the center of the chaos was DynamoDB, Amazon's serverless NoSQL workhorse, which for a stretch of time became unreachable. The reason? DNS resolution for dynamodb.us-east-1.amazonaws.com failed. That single point of failure had a domino effect, triggering outages in at least 82 AWS services according to community updates, from IAM to Lambda to EC2.

    Calling this a "glitch" undersells it; it was a full-blown architecture gut punch. IAM (Identity and Access Management) being disrupted meant other services couldn't:

    • Authenticate
    • Orchestrate
    • Communicate with one another

    Without DNS, services that rely on dynamic endpoint discovery just… stopped. If you couldn't resolve a hostname, you couldn't do anything, and no, retrying your requests didn't help.

    East-1 isn't optional, and that's the problem

    AWS promotes multi-region redundancy as the gold standard. Ironically, though, many global services, including the control planes for Route 53 (their own DNS service) and IAM, are tightly coupled with the US-EAST-1 region. So even if your application spans us-west-2, eu-west-1, or ap-southeast-2, you're stuck if the brains of the operation live in us-east-1.

    Users in dev communities pointed this out quickly:

    "AWS: for maximum resiliency you need redundancy across multiple regions. Also AWS: many of our services have a control plane which has a single point of failure in us-east-1"

    Another joked:

    "us-east-1 has been fucked for years. I guess they just admitted it and turned it off."

    Humor aside, the dependency on this one region is systemic as well as ironic. Some even speculate that AWS, by offering the lowest prices and fastest latency in us-east-1, is subtly incentivizing developers to bet big on a single region, even when they shouldn't.

    All the AZs in the world won't save you from bad design

    Availability Zones were supposed to prevent this kind of thing. Each AZ has:

    • Separate power
    • Cooling systems
    • Networking infrastructure

    The idea is that even if a meteor strikes one data center, the others will carry the load. That model only works when issues are physical.

    When the problem is logical, like a botched DNS update or a bad deployment, it doesn't matter how many AZs you've sprinkled around. If the failure point is in software and it propagates across the entire region, you're going down. One commenter nailed it:

    "AZ's reduce risk of physical issues. They aren't foolproof to software issues that are deployed to the entire region."

    That's exactly what happened here. Whether due to deployment automation, propagation of misconfiguration, or a dependency flaw, the issue didn't stay in one AZ. It spread like wildfire across all six.

    The fall of Slack, DockerHub, Ring, and friends

    The carnage wasn't confined to AWS customers running niche apps. Major services were hit:

    ServiceImpact
    SlackCommunication down
    DockerHubContainer registry unavailable
    Quay.ioImage pulls failed
    DatadogMonitoring alerts silent
    RingSecurity cameras offline

    Users couldn't log into the AWS console, check security camera feeds, or even get monitoring alerts. Many only learned about their own outages because Reddit and X (formerly Twitter) were still up.

    A particularly meta moment came when users joked, "It's always DNS," followed by "No way it's DNS again." And then: "It was DNS." The comment section read like a meme graveyard, filled with disbelief and gallows humor. A few people joked about hot coffee spills in the server room or haunted infrastructure. Behind the laughter, though, was a growing distrust in the resilience of supposedly redundant cloud systems.

    What should have been done, and still can be

    AWS didn't break any SLAs, since their legal guarantees account for this kind of stuff. Trust is another matter, and it's wearing thin.

    So what now?

    Some teams nailed it. A handful of engineers proudly reported their systems failed over cleanly to us-east-2 or other regions, thanks to active-active setups or at least asynchronous replication. DynamoDB's Global Tables, when configured properly, do support multi-region writes, a rare bright spot in the chaos. But it takes effort, architecture planning, and, yep, money.

    And there's the rub. One of the most upvoted comments summed it up:

    "The cost of making an app resilient to regional outages doesn't always make business sense. The regions and services are incredibly stable and a large amount of customer apps can legitimately lose availability briefly with no harm done."

    In other words, risk tolerance is a business decision. You can build for regional redundancy, but for most teams it comes down to trade-offs.

    A bigger problem in disguise

    This outage was more than a fluke. It reflects cloud architecture at scale: hidden interdependencies, opaque control planes, and design decisions that pile up over a decade until they become invisible bottlenecks.

    It's also a reminder that no matter how far we go with automation, machine learning, and cloud-native best practices, the basics still matter. DNS is ancient tech, and it's also essential. When it fails, everything breaks.

    Amazon will patch this up. Services are already back online, and a formal post-mortem will follow, probably dense with engineering jargon and heavy on lessons learned.

    The next time us-east-1 wobbles, and it will, ask yourself what you learned the last time. The cloud's most popular region isn't going away, but maybe, just maybe, your single point of failure should.