Privacy-first analytics from CDN logs

The problem

I wanted to know which pages people actually read. Every conventional answer to that costs something I wasn’t willing to pay: a third-party script is a new origin my CSP would have to allow, a cookie is a consent banner, and both mean shipping a tracker to the visitor to measure the visitor.

But the measurement had already happened. Every request to this site passes through CloudFront, which can write down what it served. The data was there — the question was only whether I could get at it without any of it becoming surveillance.

The architecture

Four steps, none of them in the request path:

1. CloudFront standard logging (v2) delivers access logs to S3 as Parquet, partitioned Hive-style by distribution and date.
2. A Glue table with partition projection makes those files queryable without a crawler.
3. A nightly Lambda runs a handful of Athena queries and writes one aggregate document.
4. /traffic/ fetches that document same-origin through the existing /data/* behavior.

The visitor’s browser does nothing. There is no script to block, no request to a tracker, and the page you’d read the numbers on is itself measured exactly the same way as any other page — by the CDN, after the fact.

Key decisions & trade-offs

Standard logging v2, not the legacy log format. The legacy option writes gzipped text to a flat prefix. v2 writes Parquet into partitioned paths and — critically — lets me choose which fields are delivered. That third capability is the entire privacy design, and legacy logging simply cannot do it.

The privacy property is structural, not procedural. c-ip, x-forwarded-for, c-port, cs(Cookie) and the viewer’s network operator are excluded at delivery. They are not collected and then deleted; they never arrive. That distinction matters because “we delete IPs after 30 days” is a promise you have to trust, while “the field is not in the delivery configuration” is a fact you can verify — and I verified it against the live delivery rather than against my own Terraform.

The one honest compromise: user agents. I do deliver cs(User-Agent), and it’s the only field here capable of identifying anything. Without it, crawler traffic is indistinguishable from human traffic, and a page proudly publishing “traffic” that is 90% bots is worse than no page at all. So it’s kept, used only to classify and exclude bots, and never published — the aggregate contains no user agent string anywhere. With no IP, no cookie and no ASN to join it against, a user agent alone is a weak identifier. I’d rather state that trade plainly than quietly claim a purity the system doesn’t have.

Partition projection instead of a Glue crawler. A crawler costs money per run and always lags new partitions. Projection computes partition locations from the query predicate itself, so a query for one day reads one prefix and never lists the bucket. The result: a real query in production scanned 397 bytes.

A scan ceiling the code cannot opt out of. Athena bills per byte scanned, so the failure mode of a scheduled query isn’t a wrong number — it’s a surprise invoice after someone drops a WHERE clause. The workgroup sets bytes_scanned_cutoff_per_query with enforce_workgroup_configuration, so a query that loses its partition filter is killed by AWS rather than billed, and no client can override the limit. The guardrail is the design, not a monitoring alert after the fact.

Publish the real numbers, including the small ones. This is a personal site; the traffic is modest. Publishing it anyway is the point — an analytics page you’d only show when the number flatters you isn’t measurement, it’s marketing.

What the logs are not

They can’t tell me who you are, and they can’t follow you between visits. There is no identifier that survives a request. What they can tell me is which pages get read, what fraction is served from cache, and roughly where in the world requests come from — which is genuinely all I wanted.

Three things that were not guessable

Worth writing down, because each fails silently rather than loudly:

Field names aren’t the names in the docs’ prose. CloudFront accepts the W3C forms — cs(Referer), cs(User-Agent), cs(Host) — not the hyphenated cs-referer I’d have written from memory. A wrong name isn’t a validation error; the field just never shows up.

The partition path takes variables only. My first attempt at a suffix path, distributionid={distributionid}/{yyyy}/..., was rejected outright: with Hive-compatible paths enabled, AWS generates the key= segments itself, so writing one is a conflict. The list of allowed variables isn’t in the prose documentation at all — it comes from describe-configuration-templates, which is where I got it after the first failure.

Parquet column names are sanitized, and the types are all strings. cs(Referer) lands as cs_Referer, mixed case preserved, and sc_status is a string, not an integer. Declaring it as int in Glue would have produced HIVE_BAD_DATA at query time — long after the table was created and looked fine. I read the schema out of a delivered file before writing the table rather than after debugging it.

Cost breakdown

The honest note: Parquet conversion is a real charge, and I’d initially scoped this without it. It’s still cents at this volume, but “free” would have been wrong.

What I’d do differently at scale

At meaningful traffic the nightly full-window rebuild stops making sense — you’d aggregate each day once into a compact summary table and read only the summaries, rather than re-scanning thirty days every night. The queries would move behind a proper orchestration step so a partial failure retries one day instead of the whole window, and I’d add a bot-classification list maintained as data instead of a tuple of substrings in the Lambda.

What wouldn’t change is the shape: measure at the edge where the request already is, exclude identifying fields at the point of collection rather than in post-processing, and put a hard, service-enforced ceiling on anything that bills per byte.

Links

• The artifact: andruxa.dev/traffic
• It also counts clicks the link shortener structurally cannot count itself
• Repo: github.com/andruxap/resume-website