What a Daily LLM Digest Agent Costs to Run for a Year: $166 and 34 Hours

We have run a daily LLM digest agent in production for a full year. Once a day it pulls candidate items from a handful of public sources — a freelance marketplace’s public listing feed, a public procurement portal, a couple of newsletter sites — drops the noise, summarizes what survives, and assembles one digest.

Before we built it, the cost question had no straight answer anywhere. Estimates were either “pennies” or a pricing calculator with no failure modes in it. So we instrumented every run and kept the receipts. Here is the actual year.

Where the money went

The agent fires once a day at 06:10 UTC and does three model stages.

Triage. Roughly 120 candidate items arrive per run. They go to a small model in batches of 40 — title, source, one-line excerpt — which returns keep/drop plus a one-word reason. That is about 18,400 input and 1,900 output tokens a day. On the small tier we used ($1 per million input, $5 per million output), $0.028 a day.

Summarize. The ~12 items that clear triage each get their own call with the fetched page text attached. About 47,000 input and 3,400 output tokens a day. On the mid tier ($3 / $15 per million), $0.19 a day.

Assemble. One editorial pass over the stitched draft: ordering, collapsing near-duplicate stories, writing the subject line. About 9,200 input and 1,300 output tokens, $0.05 a day.

Daily model spend: $0.27. Across 365 runs, including re-runs after failures, that was roughly 30M input and 2.6M output tokens, or $106 for the year.

The only other hard cost was the runner: a $5/month VM that also hosts two unrelated cron jobs, which we did not prorate. Postgres and object storage stayed inside free tiers — the entire year of state, including cached source HTML, is under 400 MB. Delivery went through a newsletter platform we never outgrew.

That last row is the one that matters.

The 34 hours

41 of 365 runs — 11% — needed a person. Four incidents account for most of the time:

Nine days of empty digests. One source started returning HTTP 200 with an empty result array instead of an error. Nothing threw. The agent triaged zero items, summarized zero items, and sent a digest with two entries instead of twelve. Our health check asked “did the run throw?”, and the answer was no, every day, for nine days. Token cost of the incident: under $2. Time to notice, diagnose, and add a floor assertion on item count: about three hours.

An uncapped retry loop. A malformed model response failed schema validation, and the retry wrapper had no ceiling. It ran 214 attempts overnight before the run’s wall clock killed it. Cost: $6.10.

Silent input bloat. A source changed its markup and our extractor started handing the summarizer full page chrome — nav, footer, comment threads. Input tokens per item went from 4,100 to 18,600 and stayed there for six days. No error, slightly worse summaries, $4.80 in extra spend. We only caught it because we chart tokens-per-item daily.

Duplicate sends. A mid-run kill landed between “summarized” and “marked as sent.” The next run re-summarized and re-sent twelve items. Cheap in tokens, embarrassing in the inbox.

None of these cost more than $10. Every one cost between 40 minutes and three hours of attention. That ratio held for the entire year: the token bill was predictable and small, and the failures were slow, silent, and expensive in time.

Cutting both numbers

Five changes moved the needle, in rough order of impact.

Triage before you summarize. Running the mid-tier summarizer over all 120 candidates would cost about $2.06 a day — roughly $750 a year. The small-model triage pass costs $0.028 and removes 90% of the volume, and the digest is indistinguishable. This one decision is the difference between a $100 year and a $750 year.

Cache the stable prompt prefix. Our system prompt plus scoring rubric measured 2,100 tokens, repeated on every summarize call, twelve times a day. Uncached, that line alone would add about $24 a year. With prefix caching it is under $5. Free money, one config flag.

Cap retries and set a per-run token ceiling. Three attempts with exponential backoff, plus a hard budget the run refuses to exceed. The 214-attempt night becomes a three-attempt failure with a clear alert.

Cache fetched source HTML to object storage, keyed by URL and date. Re-runs after a crash then cost nothing in network calls, and you can replay a bad day against fixed inputs to test a fix. This is what turned “reproduce the bug” from an hour into five minutes.

Write state before you send, not after. Mark items as processed in the same transaction that records the send, and make the send itself idempotent on a run key. Kills mid-run duplicates permanently.

If you are pricing one of these agents, budget the tokens at roughly what a coffee subscription costs and budget three hours a month of your own time. The second number is the one that decides whether the agent is worth running.