Cloud AI vs Local AI

The cloud-vs-local question is the single most consequential AI tooling decision an operator makes. The default answer is "use cloud" — and for most users most of the time, that is the right answer. But there are specific situations where local AI is the only honest choice, and those situations are growing.

This page lays out the actual tradeoff so you can decide deliberately rather than by default.

What cloud AI is

Cloud AI is the API or app run by a lab: Claude (Anthropic), ChatGPT (OpenAI), Gemini (Google), Grok (xAI). You type in a chatbox or call an API endpoint; the model runs on the lab's GPUs in some data center; the response comes back. The lab sees everything you typed.

What you get: state-of-the-art quality (these are the smartest models on the planet), large context windows (200k+ tokens), regular updates (new model releases every few months), no hardware requirements (any laptop works), easy integration (chat app, API, mobile app).

What you give up: privacy (lab reads your inputs by default on consumer tiers · enterprise tiers offer zero-retention contracts but most humans don't have those), dependency (if the lab is down, your work stops · if the lab raises prices, you pay), trust (a small set of companies in California or Beijing now own the cognitive infrastructure most operators rely on).

What local AI is

Local AI is a model that runs entirely on your own machine. The two leading consumer-grade tools are Ollama (CLI · simple model management · runs on Mac, Windows, Linux) and LM Studio (GUI · model browser · same model support). The leading open-weights model families in 2026 are Llama (Meta), Mistral, Qwen, DeepSeek, and Gemma — each shipping models in size tiers from ~1B to ~70B+ parameters.

What you get: total privacy (nothing leaves the machine · works without internet), no rate limits (you can run it all day), no subscription cost (one-time hardware investment, then free), full control (you choose the system prompt, the refusal posture is yours to set), ownership (you can run the same model in 5 years even if the lab disappears).

What you give up: quality (open-weights models in 2026 are roughly 12–18 months behind frontier on hardest benchmarks · this gap is closing fast), speed (depends on your hardware · a recent MacBook Pro runs a 14B model usably; a desktop with a real GPU runs 30–70B models smoothly), setup curve (one terminal command for the basic case, but harder if you want quantization, fine-tuning, or specific runtime), context window (smaller for most local models · 8–32k typical, vs 200k+ on cloud).

The quality gap, calibrated

In May 2026, here is roughly the quality ladder, from best to "good enough for most tasks":

GPT-5 / Claude Opus 4.8 / Gemini Ultra 3 — frontier cloud, best at everything. Llama 4 70B / Qwen 3 72B / DeepSeek V4 — open-weights frontier, roughly Claude 3.7 / GPT-4 era quality. Llama 4 8B / Mistral Nemo / Phi-4 — small open-weights, surprisingly good for narrow tasks.

For writing first drafts, summarizing, code completion, brainstorming, and general chat, the open-weights 70B class is now indistinguishable in blind testing from frontier cloud for most users. For frontier reasoning, long-context analysis, agentic multi-step tasks, and the hardest benchmarks, cloud still wins by a meaningful margin.

The gap is closing about 12–18 months per generation. Open weights in 2026 = closed cloud in 2024–2025.

Privacy posture, calibrated

This is where the tradeoff is starkest. Cloud labs in 2026, on consumer tiers, train on your inputs by default unless you opt out. (Anthropic changed this default in August 2025 — Claude Free/Pro/Max are now default-opt-in. ChatGPT Free has always been default-opt-in.) Enterprise contracts can be zero-retention, but most users don't have enterprise contracts.

Local AI is the only honest "nothing leaves your machine" option. For confidential client work, journalism with sensitive sources, medical or legal drafting, therapy-style journaling, and any prompt you wouldn't want screenshot — local is the only path that respects the boundary.

Hardware requirements

Cloud: any laptop with a browser.

Local: a recent Mac (M1 or newer), or a Windows/Linux machine with at least 16GB RAM and ideally a discrete GPU. The smallest models (1–3B params) run on phones. 7–14B runs on most modern laptops at usable speed. 30–70B needs a real GPU or an Apple Silicon machine with 64GB+ unified memory.

If you have a 5-year-old laptop with 8GB RAM and no GPU, local AI is going to feel slow. If you have a recent machine, it feels surprisingly snappy.

When to use each

Use cloud when: - The task needs frontier reasoning (hard analysis, complex code, agentic multi-step) - You're working with public information (no privacy concern) - Speed matters more than control - You need the largest context windows (long documents, codebases)

Use local when: - Confidentiality is non-negotiable (medical, legal, journalism, therapy, sensitive client work) - You want offline capability (travel, unreliable internet, air-gapped environments) - You're running it all day every day (rate limits or per-token costs would hurt) - You're building something you want to control end-to-end (your refusal posture, your system prompt)

Use both when: - You're an operator running real work — frontier cloud for the hard 20% of tasks, local for the high-volume 80% where privacy and rate-limits matter

Decision framework

If you have not yet tried local AI, install Ollama, pull llama3.2:3b (~2GB download), run one real task. Even if you never use it again, having an opinion is the operator move. The local-AI lesson on /learn walks through this in detail.

If you are paying for two or more cloud subscriptions, evaluate whether local could replace one. Most operators discover that a recent Mac runs a 14B model fast enough for 60–70% of daily work, with the remaining 30–40% routed to cloud for the hard tasks.

If your work involves confidential material and you are using consumer cloud chatbots, you are leaking that material to the labs. That is the conversation worth having with yourself this week. Local is the easiest mitigation.

The cloud vs local question is not "which is better." It is "which is right for which task." The operator who runs both, calibrated by job, gets the quality of cloud and the privacy of local at the same time. That is the upgrade.

For per-task model routing, see /tools — every job is mapped to the recommended AI with one-sentence routing reasoning.