A local LLM writing assistant that lives where you type — not in a chat window

You've run Ollama. You know what a GGUF is. And yet when you write an email, the local model you so carefully set up is sitting in a terminal or a chat window, three ⌘Tabs away from your cursor. This guide is about closing that last gap: getting local-LLM writing help at the caret, in every Mac app.

The gap every “run an LLM locally” guide stops at

The tutorials all end at the same place: model downloaded, server running, chat interface open. That's genuinely useful for coding — editors already know how to talk to a completion backend. For writing, though, the story stops short. Your email lives in Mail, your messages in Slack, your notes in Obsidian; the model lives in a chat box. So you copy text out, paste answers back, and the “assistant” is really a destination you commute to. What's missing isn't the model — it's the plumbing between the model and the text field you're actually typing in.

What a small model is honestly good at

Second honest point: for writing help at the caret, you don't want a 70B model, and you shouldn't expect 70B behavior from a small one. A Gemma-1B-class model is genuinely good at three things:

• Continuation. Given your last sentences as prefix, predicting the next few words is exactly what a base model does natively. That's autocomplete — the ghost text that appears in grey and that you accept with Tab.
• Translation. Short conversational messages translate well at this size, especially when the target language is fixed per conversation.
• Tone. Rewriting one sentence more formally or more lightly is a small, well-bounded transformation.

What it is not good at is drafting an essay from a one-line prompt. Souffleuse doesn't ask it to. The design bet is that a small model doing the right small jobs, instantly, beats a big model in a window you have to visit.

How Souffleuse wires llama.cpp into the caret

Souffleuse is that plumbing, packaged: llama.cpp running on Metal, embedded in a menu-bar app, connected to whatever text field has focus through the macOS Accessibility API. That API is how the app reads your prefix (the text before the cursor) and knows where the caret is, in Mail, Notes, Slack, Safari, Chrome, Pages, Bear, Obsidian… The suggestion is drawn as grey ghost text at the cursor; Tab accepts (word by word or in chunks), Esc dismisses, and just typing on makes it vanish.

A few engineering details that matter to the feel, since you'll ask:

• KV-cache reuse between keystrokes. The prompt isn't re-evaluated from scratch at every character; the cache carries over, which is why suggestions feel instant instead of “fast for an LLM”.
• Cancel-on-keystroke. Every keystroke cancels the in-flight generation, so stale suggestions never flash on screen.
• Engine unloads at idle. The model stays warm while you write and is released when you stop — the RAM-and-battery reality of keeping an LLM resident is handled for you.
• Zero config. No server process, no port, no Modelfile. Fields macOS marks as secure (passwords, card numbers) are ignored by construction.

Setup, all of it

1. Download the DMG (~7.5 MB, notarized by Apple, requires a Studio license) and drag it into Applications.
2. Grant the Accessibility permission it asks for — that's the API doing the caret work.
3. Let it download a GGUF model once (0.4–1.5 GB depending on the model). This is the only network connection the app ever makes.
4. Type anywhere. Grey text appears; Tab takes it, Esc clears it. // summons translation or a tone rewrite.

Privacy by architecture, not by promise

You already know why local matters — it's probably why you installed Ollama in the first place. Souffleuse takes the same stance to its logical end: there is no server in the design at all. Not “we don't log” — there is nothing to send to, no account to create, no telemetry. It even learns your style from an encrypted history that stays on your Mac, readable by you alone. The privacy guarantee is structural, which is the only kind that survives a change of ownership or terms of service. More on that in the offline AI writing guide.

Full disclosure: the app is made in France and its interface is currently in French. The model's output follows whatever language you type — English in, English out — and if you can configure a llama.cpp sampler, you can survive a French Preferences window.

Frequently asked questions

Do I need Ollama or llama.cpp installed to use Souffleuse?

No. Souffleuse ships llama.cpp with Metal support inside the app. You download the app (~7.5 MB), it fetches a GGUF model once (0.4–1.5 GB depending on the model), and that's it — no server to run, no terminal, no config files.

Is a 1B-class model good enough for writing help?

For the right jobs, yes. A small model is genuinely good at continuing your own sentence, translating a message, and adjusting tone — tasks where your text supplies most of the context. It is not the tool for generating essays from scratch, and Souffleuse doesn't pretend otherwise.

Does it eat RAM and battery all day?

The engine stays warm while you're writing and unloads when idle, so it doesn't hold memory hostage all day. Between keystrokes the KV cache is reused instead of recomputing the prompt, which is what makes suggestions feel instant.