Why We Moved Search, Tagging, and Recommendations Out of the Cloud

Most "AI-powered" browser extensions ship everything to a server — including the parts that have nothing to do with AI at all. Search, tag cleanup, "find similar items": these are decades-old, well-understood techniques. They don't need a language model, and they definitely don't need a network round-trip. We went through AI Summary Helper feature by feature and asked, for each one: does this actually need the cloud, or are we just defaulting to it because that's the easy architecture?

Most of them didn't need it.

What actually moved on-device

Search. The extension originally matched only on article title and tags — fast, but blind to anything in the article body. Rather than brute-force scanning full text on every keystroke, or shipping every query to a server, we built a small TF-IDF index from the user's own saved archive. Cheap fields — title, tags — get checked first and instantly, the same as before. Longer queries get widened against the index, which is built once per session and updated incrementally, not re-scanned per keystroke.

Duplicate detection. Before a save completes, a tiered check runs: exact normalized URL match first (cheap, O(1)), then title similarity on the handful of candidates that pass that filter, and only then — for the few remaining candidates — a full TF-IDF cosine-similarity comparison. Most saves never reach the expensive tier at all, because the cheap tiers already resolved them.

Tag normalization. Tags accumulate drift over time — "ML," "Machine Learning," "machine-learning" all meaning the same thing to a human and nothing alike to a filter. A small alias table catches the obvious cases; a fuzzy edit-distance merge across the whole tag vocabulary catches the rest, picking whichever spelling the user actually used most often as the canonical form rather than forcing a "correct" one on them.

Readability and tone. Flesch–Kincaid grade level is pure arithmetic on sentence and syllable counts — no model required. Sentiment is a bundled AFINN wordlist with a small negation check on the two preceding tokens. Neither of these needed to leave the browser.

"Similar articles" and Digest. Both reuse the exact same TF-IDF vectors search already built. It would have been easy to build three separate similarity systems for three separate features. Building one index and letting search, recommendations, and digest curation all read from it kept the total system smaller and meant improving the index once improved all three features at once.

What we deliberately left in the cloud

The actual summarization step — the part that genuinely requires a language model — still needs a network call. There's no getting around that with current on-device model sizes if you want a cloud-tier model's output quality. Users bring their own API key for that step (OpenAI, Gemini, Mistral, DeepSeek), or point the extension at a local Ollama instance and skip the network for that step too. That's the one place where "on-device" isn't the default — it's opt-in, because the tradeoff there is real: a local model is slower and lower-quality than a hosted one, and we didn't want to pretend otherwise.

Why this is more than a privacy talking point

The privacy angle is real — an extension with broad host permissions sending page content to a server for things that don't need it is a legitimate trust question, not just optics. But the more mundane reason to do this is that it's usually just the correct engineering call. TF-IDF search doesn't need 200ms of network latency. Tag deduplication doesn't need an API bill attached to it. Matching the tool to the actual job, instead of routing everything through "call the cloud" by default, tends to produce something faster, cheaper to run, and more resilient to being offline — the privacy benefit comes along for free, not the other way around.