pickuma.
Meta

How We Keep 490 Published Reviews From Going Stale

A look at the systems we use to stop nearly 500 tool reviews from quietly rotting: staleness scoring, automated link checks, and visible changelogs.

7 min read

A review is a snapshot. You publish it, the tool ships a redesign three weeks later, doubles its free tier, or quietly kills the feature you spent two paragraphs praising — and now your snapshot is wrong. One wrong review is an embarrassment. Nearly 500 of them, accumulating drift in parallel, is a structural problem. This is how we manage it on a catalog that currently sits at 490 published reviews.

The honest starting point: you cannot re-read 490 articles on a schedule and expect it to happen. Manual review is the thing that always loses to whatever is on fire this week. So the goal is not discipline. The goal is to make staleness visible and cheap to act on, so that the work that survives a busy week is the work that matters most.

Staleness is a score, not a feeling

Every post carries an updatedAt field in its frontmatter alongside publishedAt. That gap — days since last meaningful edit — is the raw signal. But raw age is a bad proxy. A review of a stable Unix utility can sit untouched for a year and stay correct. A review of an AI tool shipping weekly can be wrong in a month.

So we weight age by volatility. Each review inherits a volatility tier from its category. AI and dev-tool reviews decay fast; infrastructure and “dev knowledge” explainers decay slowly. The staleness score is roughly days-since-update × category-volatility-weight, and the catalog is sorted by it descending. What surfaces at the top isn’t the oldest article — it’s the one most likely to be lying to a reader right now.

That single reordering changes the economics. Instead of asking “which of 490 should I check,” the question becomes “here are the 10 most likely to be wrong — start at the top.” A 30-minute pass on a Friday clears real risk instead of nibbling at the alphabet.

Reviews rot in two distinct ways, and each needs its own machine.

Dead links are the mechanical failure. Affiliate redirects break, vendors rename their docs, a /pricing page becomes a /plans page. We run a link-rot audit across the whole catalog on a schedule — every outbound URL gets a HEAD request, redirects are followed once, and anything returning 404, 410, or a redirect chain longer than one hop gets flagged. Affiliate links get special handling: a paused program returns 410 by design from our /go/[slug] redirect, so the audit distinguishes “intentionally retired” from “silently broken.” This is fully automatable and runs without a human in the loop until something flags.

Drifted facts are the harder failure, because no HTTP status code tells you that a tool’s free tier dropped from 1,000 records to 200. A live link can point at a page that contradicts your review. We don’t pretend this is fully solved. What helps: every review’s frontmatter lists the specific tools it covers, so when we learn a tool changed — a pricing email, a changelog, a release note — we can query which reviews reference it in one command instead of grepping prose. The blast radius of any single tool’s change is a known, queryable set, not a mystery.

Failure modeDetectionHuman needed?
Dead outbound linkScheduled HEAD-request auditOnly on flag
Retired affiliate410 from /go redirectNo (by design)
Pricing / tier driftTool→review reverse lookupYes
Feature removedManual, triggered by changelogYes

The pattern across both: automate detection completely, accept that correction of factual drift still needs judgment, and shrink the search space so that judgment is spent on the right twelve paragraphs.

Make the update visible, not silent

When we do correct a review, we don’t quietly overwrite it. Each post supports a changelog array in frontmatter — short, dated notes that render at the article. “2026-04: updated pricing after free tier dropped to 200 records.” “2026-05: removed section on the export feature, deprecated by vendor.”

This is partly an E-E-A-T signal — a reader can see the page is maintained, and search engines reward freshness that is real rather than a date bump with no edit. But the bigger reason is internal accountability. A changelog is a diff you can’t hide from. It forces the update to be a specific, describable change rather than a vague “refreshed.” If you can’t write the changelog line, you didn’t actually fix anything.

After any substantive edit, the post re-enters the syndication loop: the sitemap gets re-pinged via IndexNow so Bing and Yandex re-crawl the corrected version quickly, instead of serving the stale one for weeks. A correction nobody re-indexes is half a correction.

Notion

We track the volatility tier, last-reviewed date, and open drift flags for the whole catalog in a single Notion database — sorted by staleness score so the riskiest review is always row one.

Free for personal use; team plans from $10/user/mo

Try Notion

Affiliate link · We earn a commission at no cost to you.

None of this makes a review immortal. Tools will always move faster than any audit cadence, and some drift slips through until a reader emails us. But the difference between a catalog that decays gracefully and one that quietly becomes a liability is whether staleness is measured. A number you can sort by beats good intentions every time.

FAQ

How often do you run the full audit?+
The link-rot check runs on a schedule across all 490 reviews and only escalates to a human when something flags. The factual-drift review is event-driven — triggered when we learn a covered tool changed — rather than calendar-driven, because a fixed cadence either wastes effort on stable content or lags behind volatile content.
Why weight staleness by category instead of just using age?+
Because age is a weak proxy for wrongness. An infrastructure explainer can be correct for a year; an AI-tool review can be wrong in a month. Weighting days-since-update by a category volatility tier surfaces the reviews most likely to be misleading a reader right now, not just the oldest ones.
What stops a fixed review from going stale again immediately?+
Nothing stops it permanently — that's the point of treating staleness as a continuous score rather than a one-time cleanup. A corrected review re-enters the same sorted queue, just with a fresh timestamp that lowers its score for now. When its tool moves again, it climbs back up the list.

Related reading

See all Meta articles →

Get the best tools, weekly

One email every Friday. No spam, unsubscribe anytime.