Technology & privacy
Private AI on a Mac: what actually runs locally
Some AI on a Mac genuinely runs on the Mac. Some of it says it does. Here is the inventory, and how to tell the difference for any app you are considering.
Last updated
The inventory
| Task | Runs locally on a Mac? | Notes |
|---|---|---|
| Speech recognition (dictation) | Yes, well | Whisper and Parakeet families. Close to cloud quality on clean audio |
| Transcribing recorded files | Yes | Slower than real time on long files, but entirely local |
| Small language models for writing tasks | Yes | Tidying, summarising, rewriting. Good at bounded jobs |
| Frontier-scale language models | No | Too large. This is the honest boundary |
| Translation | Yes, for many pairs | Apple's own, and open models |
| Image generation | Yes | Slower than a service, fully local |
| Optical character recognition | Yes | Built into macOS and very good |
| Semantic search over your own files | Yes | Embedding models are small |
| Most "AI" in web apps | No | A browser tab is almost always calling a server |
| Apple Intelligence | Partly | Some on-device, some via Private Cloud Compute. Not the same thing |
Where the line actually is
It is not a mystery and it is not about capability in the abstract. It is memory.
A model has to fit in RAM to run. A 16 GB Mac can comfortably hold a model of a few gigabytes alongside everything else you are doing. Frontier models are hundreds of gigabytes. No amount of optimisation bridges that, and anyone promising a frontier-quality model running on your laptop is describing something else.
The useful consequence: local AI is excellent at bounded tasks — recognise this speech, tidy this paragraph, translate this sentence, find the file that means this — and not a substitute for a large general model doing open-ended reasoning. Choosing tools by that distinction rather than by a privacy label gets you further.
Apple Intelligence is two things
Worth separating carefully, because it is the most common source of confusion about what is local on a Mac.
- On-device models. Some Apple Intelligence features run on your Mac. These are genuinely local.
- Private Cloud Compute. Other requests go to Apple servers running on Apple silicon, with a published architecture and verifiable server builds. It is a serious design and materially better than an ordinary API call — and it is still a request leaving your machine.
If your requirement is "nothing leaves the device", the second category does not meet it, however good the engineering. If your requirement is "processed by a party I trust under a design I can inspect", it may. Know which requirement you have.
How to check any app
- Does it download a model? A few hundred megabytes to several gigabytes on first run. No download means no local model.
- Does the feature work with the network off? The definitive test. Run it on each AI feature separately, not just the main one.
- Does it ask for an API key? Usually a hosted service — though some apps use a key only for optional extras, so check which.
- Watch the traffic while the feature runs Little Snitch, or
nettop -p <pid>. A local feature is quiet. - Test each stage separately This is the one people skip. An app can recognise speech locally and then send the text to a cloud model to tidy it. Two stages, two answers.
Building a local-first setup
If the goal is to do most of your work without sending it anywhere, this is roughly what is achievable in 2026:
- Dictation: a local speech app. Genuinely solved.
- Writing assistance: a local language model for tidying and summarising, via Ollama or an app that embeds one. Good at bounded tasks.
- Transcription: a local transcription tool for recordings.
- Search: local semantic search over your own documents.
- Translation: macOS's own, for common pairs.
- Open-ended reasoning: this is where you will still reach for a hosted model, and the honest advice is to be deliberate about what you put in it rather than to pretend a local alternative exists.
Questions
Can I run a large language model on my Mac?
Small and mid-size models, yes, comfortably — through Ollama or apps that embed one. Frontier-scale models, no: they need hundreds of gigabytes of memory. Local models are very good at bounded tasks and are not a substitute for a large general model doing open-ended reasoning.
Is Apple Intelligence private?
Partly on-device and partly via Private Cloud Compute, which is a well-designed system with verifiable server builds. If your requirement is that nothing leaves the machine, the cloud half does not meet it however good the design.
Is local AI worse?
For bounded tasks — speech recognition, tidying text, translation, search — often indistinguishable, because the task does not need scale. For open-ended reasoning, yes, and by a clear margin.
How do I check whether an app's AI is local?
Turn off the network and use the feature. Test each AI feature separately: an app can recognise speech locally and still send the text to a cloud model to tidy it.