---
name: maluos-gpu-mac
description: Serve the models on an Apple Silicon (M-series) Mac to MaluOS machines. Install Ollama on the Mac, bind it to the LAN, keep the Mac awake, pull models, and add the Mac as an ollama_local backend of a MaluOS gateway with tier aliases. Use when the user wants a Mac mini, Mac Studio or MacBook with Apple Silicon to act as a GPU or model server for MaluOS, or asks why a Mac backend is unhealthy or a model is missing.
---

# A GPU server on an M-series Mac

Human guide: https://maluos.com/howto/gpu-mac/ . Two machines: the **Mac** (runs Ollama under
macOS; nothing MaluOS is installed) and the **MaluOS host** (server or desktop whose gateway
uses the Mac). Keep track of which one a command runs on.

## What is true today (say this to the person if they expect more)

- MaluOS does not run on Apple Silicon.
- The MaluOS node service for non-MaluOS machines (queue, per-caller keys, run history, the
  Sovereign LLM Manager's node API; repository `maludb/maluos-local-llm-server`) is approved
  and **not yet built**. Until it ships, the Mac is a plain Ollama backend of a MaluOS gateway.
  The gateway still provides queueing, policy, budgets and the ledger for every MaluOS caller.
- Ollama has **no authentication**. The LAN or VPN and the macOS firewall are the boundary.
  Never point callers outside the MaluOS gateway at the Mac.

## Procedure

### 1. Ollama on the Mac

Install from https://ollama.com/download or `brew install ollama`. Check:

```sh
ollama --version
curl http://127.0.0.1:11434/          # "Ollama is running"
```

`ollama signin` is optional; it adds Ollama's cloud models to the daemon, and the gateway's
policy on the MaluOS side decides whether callers may use them.

### 2. Bind to the network (Mac)

Ollama app:

```sh
launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
# quit Ollama from the menu bar, open it again
```

Homebrew service: put `OLLAMA_HOST=0.0.0.0:11434` in the service's environment and
`brew services restart ollama`; or test in the foreground with
`OLLAMA_HOST=0.0.0.0:11434 ollama serve`.

Verify **from the MaluOS host**: `curl http://mac-mini.lan:11434/` answers. If not: macOS
firewall (System Settings → Network → Firewall → allow Ollama) or the variable not applied.

### 3. Models and sleep (Mac)

```sh
ollama pull qwen3:32b          # unified memory sizes the model: ~24 GB for a 32B model
ollama pull llama3.1:8b
ollama pull nomic-embed-text
ollama ps
sudo pmset -a sleep 0 disksleep 0
sudo pmset -a womp 1
```

### 4. Backend on the MaluOS host (operator)

```sh
sudo maluctl gateway backend add mac-mini --kind ollama_local --endpoint http://mac-mini.lan:11434 \
    --models qwen3:32b,llama3.1:8b,nomic-embed-text --max-concurrent 1 --keep-alive 30m
maluctl gateway backends                                   # mac-mini healthy with the models
sudo maluctl gateway alias set strong   --tier strong   --backend mac-mini --model qwen3:32b        --position 0
sudo maluctl gateway alias set standard --tier standard --backend mac-mini --model llama3.1:8b      --position 0
sudo maluctl gateway alias set embed    --tier embed    --backend mac-mini --model nomic-embed-text --position 0
sudo maluctl gateway catalog
```

Model management through the gateway: `sudo maluctl gateway ollama pull MODEL --backend mac-mini`,
`warm MODEL --backend mac-mini`.

### 5. Test (MaluOS host)

```sh
maluctl gateway chat "Say hello from the Mac." --model strong
maluctl gateway routes            # route row names mac-mini
maluctl gateway usage --by backend
```

`scripts/verify.sh mac-mini.lan:11434 mac-mini` runs these checks from the MaluOS host.

## Facts that shape advice

- `--max-concurrent 1` is right for one model's worth of memory; raise only with room for two.
- An `ollama_local` backend is treated as local by the gateway: the sensitivity (egress) gate
  does not apply. If the person's data policy needs the gate, pair the Mac with one MaluOS GPU
  server and let desks use that server as a `remote` backend instead.
- Several MaluOS hosts can add the same Mac; each has its own queue, there is none across them.
- The first call after a restart loads the model (can take a minute); warm it.

## Stop and ask before

- changing the Mac's sleep settings (it affects the person's machine);
- opening the macOS firewall to networks beyond the LAN;
- pulling models that exceed free disk or free unified memory.

## Never

- expose the Mac's port 11434 to the internet;
- put a provider API key on the Mac for MaluOS callers; cloud keys belong in the MaluOS vault;
- claim the node service, keys or a shared queue exist on the Mac until the repository ships them.

## Troubleshooting map

| Symptom | Where | Fix |
|---|---|---|
| backend unhealthy | MaluOS host: `curl http://MAC:11434/` | bind address (step 2), macOS firewall |
| `model_missing` | Mac: `ollama list` | pull it (on the Mac, or via the gateway with `--backend`) |
| first call very slow | Mac: `ollama ps` | model loading; warm; longer `--keep-alive` |
| worked yesterday | Mac | slept, or an update restarted Ollama without `OLLAMA_HOST` |