How We Fact-Check a Tool Review Before We Publish It

Most tool reviews on the internet are rewritten press releases. Someone reads the vendor’s landing page, paraphrases the feature list, drops in an affiliate link, and hits publish. We have done the boring version of this enough times to know it does not survive contact with a reader who actually uses the tool. So every review here goes through a fixed pass before it ships. This is that pass, written down, so you can hold us to it.

Every claim gets a source or it gets cut

The first read of a draft is not for tone. It is a hunt for unsourced claims. We go line by line and flag anything that asserts a fact about the world: a price, a feature, a limit, a version number, a comparison. Each flagged line needs one of two things attached to it — a primary source we can link, or a test we ran ourselves. If it has neither, it does not get softened. It gets deleted.

The failure modes are predictable, so we look for them specifically:

• Stale pricing. A plan that was $8 per seat last quarter is $12 now, or the free tier dropped from 5 projects to 3. Pricing is the single most-changed and most-trusted number in any review, so we re-check the vendor’s pricing page on the day we publish, not the day we drafted.
• Feature drift. “Tool X does not support SSO” is true until the week it ships SSO. We check the current docs and changelog, because a feature gap is the kind of claim that ages into a lie without anyone touching the article.
• Borrowed numbers. If a benchmark or adoption figure came from someone else, we trace it to its origin. A stat that has been quoted three blogs deep has usually mutated on the way. If we cannot find the original, the number is gone.

The rule we keep coming back to: a claim you cannot link or reproduce is an opinion wearing a fact’s clothes. Readers can tell the difference, and so can search engines.

We use the tool we are reviewing

Reading the docs is not testing. For any review where we make a usability or performance claim, someone on the team runs the actual product — sets up an account, pushes a real workload through it, and hits the edges. The notes from that session are where the specific, checkable details come from: how long onboarding took, where the free tier actually caps out, which advertised feature is behind a sales call rather than a signup.

This is also how we avoid the worst category of error, which is repeating a feature that exists on the marketing site but not in the product yet. “Coming soon” on a vendor page is not a feature. If we cannot click it, it does not go in the review as something the tool does — at most it goes in as something the tool says it will do, dated and clearly marked.

We also record what we did not test. If a review covers a tool’s free and pro tiers but we never touched the enterprise plan, the article says so. Pretending to broad coverage you do not have is the same sin as fabricating a number — it just hides better.

The numbers have to reproduce

When a review includes a measurement — cold-start time, build duration, query latency, export size — we write down how we got it, not just the result. The environment, the input, the number of runs, the version of the tool. The test for whether a number stays in the article is simple: could a reader with the same setup get close to the same figure? If the answer is no, the number is decoration, and we cut it.

This is why you will not see round, suspiciously clean metrics here. “10x faster” and “trusted by thousands of developers” are the tells of a review that measured nothing. A real measurement is awkward: 2.3 seconds, not “lightning fast”; a free tier that allows 1,000 monthly active users, not “generous limits.” Awkward numbers are the ones that came from a stopwatch instead of a thesaurus.

We keep the test notes for each review in a shared workspace so a second person can re-run a check before we publish, and so we can revisit the numbers when the tool ships a major version. That paper trail is what lets us update an old review with confidence instead of guessing whether the original figure was ever real.

None of this makes a review unbiased — we run affiliate links, and we say so on every page. What it does is make the review checkable. You should be able to take any factual sentence in one of our tool reviews, follow it to a source or a reproducible test, and confirm it yourself. If you ever can’t, that is a bug, and we want to hear about it.