Installing a GPU server on an M-series Mac

Before you start

You need an Apple Silicon Mac (M1 or later) on the same network as at least one MaluOS machine, server or desktop, whose gateway will use it. The Mac keeps running macOS; nothing MaluOS is installed on it.

1Install Ollama on the Mac

Download Ollama for macOS from ollama.com/download and open it, or with Homebrew:

brew install ollama

Check it from a terminal on the Mac:

ollama --version
curl http://127.0.0.1:11434/        # "Ollama is running"

You do not need an ollama.com account for local models. Signing in (ollama signin) adds Ollama's cloud models to the same daemon; whether MaluOS callers may use them is then decided by the gateway's policy, not by the Mac.

2Bind Ollama to the network

By default Ollama listens on the loopback address only. Make it listen on all interfaces, then restart it. For the Ollama app:

launchctl setenv OLLAMA_HOST "0.0.0.0:11434"
# quit Ollama from the menu bar and open it again

If you run it as a Homebrew service instead, set the variable in the service's environment and restart the service: brew services restart ollama after adding OLLAMA_HOST=0.0.0.0:11434 to the plist's environment, or run it in the foreground with OLLAMA_HOST=0.0.0.0:11434 ollama serve while you test.

From the MaluOS machine:

curl http://mac-mini.lan:11434/      # "Ollama is running"

If it does not answer, the macOS firewall is blocking incoming connections (System Settings → Network → Firewall: allow Ollama), or the variable was not picked up by the restarted app.

3Pull models and keep the Mac awake

Pull the models the Mac will serve. Apple's unified memory means a Mac Studio with 64 GB runs models that need a 48 GB GPU elsewhere.

ollama pull qwen3:32b
ollama pull llama3.1:8b
ollama pull nomic-embed-text
ollama ps                            # what is loaded and how much memory it takes

A Mac that sleeps is a GPU server that is down. For a Mac mini or Studio that serves the network, stop it sleeping:

sudo pmset -a sleep 0 disksleep 0
sudo pmset -a womp 1                 # wake on network access, as a safety net

System Settings → Energy, "Prevent automatic sleeping when the display is off", does the same.

4Add the Mac as a backend of a MaluOS gateway

On the MaluOS server or desktop that will use the Mac, as an operator. Name the backend after the machine; the kind is ollama_local, which is the gateway's native Ollama adapter pointed at an address instead of the local daemon:

sudo maluctl gateway backend add mac-mini --kind ollama_local --endpoint http://mac-mini.lan:11434 \
    --models qwen3:32b,llama3.1:8b,nomic-embed-text --max-concurrent 1 --keep-alive 30m
maluctl gateway backends

What you should see: mac-mini healthy with the three models. --max-concurrent 1 keeps the Mac to one generation at a time, which is what one model's worth of memory runs well; raise it only on a machine with room for two. --keep-alive is how long a model stays loaded after a call.

Now put the Mac's models into the tiers your callers ask for. Position 0 is tried first:

sudo maluctl gateway alias set strong   --tier strong   --backend mac-mini --model qwen3:32b        --position 0
sudo maluctl gateway alias set standard --tier standard --backend mac-mini --model llama3.1:8b      --position 0
sudo maluctl gateway alias set embed    --tier embed    --backend mac-mini --model nomic-embed-text --position 0
sudo maluctl gateway catalog

Models can also be managed on the Mac from the MaluOS side through the gateway: sudo maluctl gateway ollama pull MODEL --backend mac-mini and warm MODEL --backend mac-mini.

5Test it

maluctl gateway chat "Say hello from the Mac." --model strong
maluctl gateway routes                 # the route row names mac-mini
maluctl gateway usage --by backend     # tokens per backend

On the Mac, ollama ps shows the model loaded. Every caller on the MaluOS machine now reaches the Mac through the same socket, class and policy as any other rung; a long wait on the Mac overflows to the next rung of the ladder (another host, or the cloud if the caller's policy allows) exactly as it would between local and cloud.

6Keep the boundary honest

• Ollama has no authentication. Anyone who can reach port 11434 on the Mac can use it. Keep the Mac on the LAN or a VPN, and let the macOS firewall allow Ollama only from local networks. The MaluOS gateway is where identity, policy and budget are applied; do not point outside callers at the Mac directly.
• The sensitivity gate does not apply to an ollama_local backend: the gateway treats it as local. If that is wrong for your data, pair the Mac with one MaluOS GPU server that applies the gate rather than with every desk.
• Several MaluOS machines can share one Mac by each adding it as a backend. There is then no queue across them, only inside each; the node service will add that.

If something is wrong

• The backend is unhealthy. curl http://mac-mini.lan:11434/ from the MaluOS machine. If it fails, step 2: the bind address or the macOS firewall.
• A call fails with model_missing. The alias names a model the Mac has not pulled. ollama list on the Mac, or pull through the gateway with --backend mac-mini.
• The first call takes a minute. That is the model loading into memory. Warm it after a restart, and keep --keep-alive long enough for your usage.
• It worked yesterday. The Mac slept, or a macOS update restarted Ollama without the environment variable. Step 3 and step 2.
• Still stuck. Open an issue at github.com/maludb/maluos/issues.

What next

• When the node service ships, install it on the Mac beside Ollama. It adds the queue, per-caller keys and the run history on the Mac itself, speaks the same node API as a MaluOS GPU server, and the gateway then adds the Mac with backend add NAME URL --key as a remote backend.
• Give the Mac to the office. Pair it with the application server's gateway, and every desk enrolled with the office reaches it through --model office with the office's ledger and budget.