Automated security scorecard

The problem

Security posture is easy to assert and hard to prove. Anyone can say “I follow least privilege” or “my headers are tight.” The interesting version is a system that checks itself, on a schedule, and shows its work — so the claim is a live graded artifact, not a bullet point.

The constraint is cost and honesty. AWS has a managed answer for continuous compliance — Config — but it bills per configuration item recorded and per rule evaluation, which lands around $2+/month for even a tiny account: more than the rest of this platform combined. So the design goal was a homegrown, transparent, read-only auditor that costs effectively nothing and that a reader can fully understand by looking at one Lambda.

The architecture

A daily EventBridge schedule invokes a Lambda that runs a fixed set of read-only checks and writes scorecard.json to the shared S3 data bucket, served same-origin through the existing CloudFront /data/* behavior. The /security/ page fetches it first-party and renders a grade plus per-check pass/fail badges. If any check flips from pass to fail versus the previous run, the Lambda publishes a descriptive message to the same SNS topic the observability canary uses.

The controls

Account — the proof that the deploy story holds up:

• No root access keys. The account root should never carry a long-lived key — it’s the highest-blast-radius credential there is.
• Deploys use GitHub OIDC, verified by the presence of the GitHub Actions OIDC provider. That’s positive evidence CI assumes a role with short-lived tokens instead of a static access key. (One accepted, documented exception: a single local-admin IAM user holds an access key for hands-on Terraform. It’s never on the CI path, so the check grades the deploy path rather than pretending the key doesn’t exist.)
• S3 account-level Public Access Block fully enabled (all four settings), so no bucket can accidentally be made public regardless of its own policy.
• No security group open to 0.0.0.0/0, catching the single most common cloud misconfiguration even though this account runs no servers.

Live site — graded by fetching andruxa.dev exactly as a browser would:

• CSP, HSTS, X-Frame-Options, Referrer-Policy, Permissions-Policy all present (HSTS additionally required to carry a real max-age).
• TLS certificate valid ≥ 30 days, via acm:DescribeCertificate on the site cert.

The first run found three real things

The point of building this was to catch what I’d otherwise miss — and on its very first run it did, grading the account a D:

• Two orphaned security groups left over from an old ALB/EC2 experiment, one with SSH (port 22) open to 0.0.0.0/0. Nothing was attached to them, but they were live world-open rules sitting in the account. Deleted.
• No account-level S3 Public Access Block. Every individual bucket was locked down, but the account-wide guardrail — the one that catches the bucket you haven’t created yet — was missing. Enabled it in Terraform.
• The original “zero IAM users” check was too absolutist: it flagged my own local-admin user. The honest reframe grades what actually matters (OIDC deploys, no root keys) and documents the one admin key as an accepted exception, rather than gaming the check or pretending the key isn’t there.

Fixing those took the grade to A / 100. That’s the whole value proposition: a scorecard that only ever says “all green” isn’t auditing anything. This one found real gaps the first day it ran.

Key decisions & trade-offs

Homegrown instead of AWS Config. Config is the “correct” managed answer and the right call at organizational scale — conformance packs, history, drift timelines. At this scale it’s ~$2+/mo to grade a handful of things a 120-line Lambda can check for free, and the Lambda is fully legible: the controls are the code. The trade-off is that I own the check logic and there’s no historical timeline — acceptable for a portfolio, and an honest thing to say in an interview rather than pretending Config is always the answer.

Read-only, least privilege — the auditor practices what it grades. The role gets exactly the inspection actions it needs. Most (iam:GetAccountSummary, iam:ListUsers, s3:GetAccountPublicAccessBlock, ec2:DescribeSecurityGroups) don’t support resource-level scoping, so they’re granted on *; the ones that do are pinned — acm:DescribeCertificate to the site cert, s3:PutObject/GetObject to the data/ prefix, sns:Publish to the one topic. It can read posture and nothing else; it can’t change a single resource.

Alert on the flip, not the state. A naive design pages every day a check is red, which trains you to ignore it. This Lambda compares against the previous scorecard.json and only publishes when a check goes pass → fail — a genuine regression — with a message naming exactly what broke. Steady-state red (a known, accepted finding) stays quiet.

Cost breakdown

What I’d do differently at scale

For a real account this graduates to AWS Config with managed + custom rules and conformance packs (CIS, PCI), Security Hub to aggregate findings across accounts, and an AWS Organizations SCP layer so the controls are enforced, not just observed — you can’t open a public bucket if the SCP forbids it. The scorecard pattern still earns its place as the public, human-readable summary on top; the enforcement moves underneath it. I’d also push findings through the same incident tooling as everything else rather than raw SNS once there’s a team to route them to.

Links

• Live scorecard: andruxa.dev/security
• Repo: github.com/andruxap/resume-website