AWS · Architecture · Best Practices

What a Well-Architected review actually finds

Two weeks of reading someone else's AWS account, and the same handful of problems keep showing up: single-AZ databases, IAM nobody has audited since 2021, and instances paid for at on-demand rates for three years.

By Hardik ShahDecember 15, 2024Updated August 18, 20268 min read

A Well-Architected review sounds like an audit. In practice it is two days of reading someone else's account with fresh eyes, and it almost always turns up the same handful of things. Here is what we keep finding.

What the review actually is

AWS publishes a framework with six pillars: operational excellence, security, reliability, performance efficiency, cost optimisation, and sustainability. A review walks a workload through the questions under each pillar and writes down where the answer is "we never got to that."

The framework is not the interesting part. The interesting part is that answering the questions out loud, with the people who built the thing in the room, surfaces problems nobody had a reason to raise before.

The five findings that come up every time

Nobody knows what the bill is for

Cost tags exist but are inconsistent, so the finance team gets one number and engineering gets no breakdown. Before you can cut spend you have to be able to attribute it. That usually means fixing tagging and turning on Cost Explorer allocation tags, which is a week of unglamorous work and the highest-leverage thing on the list.

Once the attribution is there, the savings tend to sit in three places: instances sized for a launch that happened three years ago, dev environments running through the weekend, and storage that was never lifecycled to a cheaper tier.

Access is broad because narrowing it once broke something

Someone hit a permissions error at 11pm during an incident, widened a policy to make it go away, and never came back to it. Multiply by four years. IAM Access Analyzer will show you which of those permissions have never actually been used, which makes the cleanup an evidence-based conversation instead of an argument.

Backups exist. Restores have never been tried

Snapshots are scheduled, retention is set, and no one has ever restored one into a working environment. A backup you have not tested is a hope, not a control. The fix is cheap: restore one, time it, write down the number. That number is your actual RTO, and it is usually a surprise.

The alerts fire, and nobody reads them

Alarm fatigue is the normal end state of monitoring that was added incident by incident. If an on-call engineer routinely ignores a channel, that channel is decoration. Cut alerts down to the ones that mean "a human must act now," and move the rest to a dashboard.

One person knows how the deploy works

Not written down, not automated, holds the whole release in their head. This is the finding that gets the least attention in the report and causes the most damage when it lands.

What to do with the report

The output is a list of findings with a severity against each. The mistake is treating it as a backlog to burn down in order. Pick the two that reduce the most risk, and the one that saves the most money, and schedule those. The rest is context for the next review.

We run these on our own managed accounts every six months, because the answers drift. A workload that was well architected in March has had eleven changes since.

If you want one run on your account

We do these as a fixed-scope engagement: two days on the account, a findings document you own, and a call to walk through it. No obligation to have us do the remediation.

Questions

Questions this note gets asked

Anything missing, email [email protected].

How long does a Well-Architected review take?

Two weeks for a single account of ordinary size. Longer if there are several accounts or nobody left who can explain how a workload got that way.

What does a Well-Architected review find most often?

The same four things: a production database in one availability zone, IAM policies nobody has read since they were written, on-demand instances that have been running for years, and a backup that has never been restored.

Is the AWS Well-Architected Tool enough on its own?

It finds what a machine can find and it is worth running. What it cannot do is tell you which of its findings matter to your product, which is most of the value in a review.

Do we have to fix everything the review finds?

No. The output is ranked with cost, effort and blast radius on each line so you can draw a line partway down it. Most teams do the top three items and leave the rest until something changes.

Send us your AWS bill

Thirty minutes with an engineer, no slide deck. We will tell you what we would change first and what it would cost to have us do it.

Talk to an engineer