---
name: maluos-gpu-nvidia-arm
description: Serve the models on an NVIDIA DGX Spark or another arm64 NVIDIA Linux machine to MaluOS machines. Install Ollama on DGX OS, bind it to the LAN with a systemd drop-in, firewall it, pull large models, and add the machine as an ollama_local backend of a MaluOS gateway with tier aliases. Use when the user wants a DGX Spark, Grace Blackwell or any arm64 NVIDIA box to act as a GPU server for MaluOS, or asks whether MaluOS runs on arm64.
---

# A GPU server on NVIDIA ARM (DGX Spark)

Human guide: https://maluos.com/howto/gpu-nvidia-arm/ . Two machines: the **Spark** (DGX OS,
arm64; nothing MaluOS is installed) and the **MaluOS host** whose gateway uses it.

## What is true today

- MaluOS has **no arm64 build**; it is on the roadmap as an idea, not a plan. Do not try to
  install MaluOS or its packages on the Spark.
- The MaluOS node service for non-MaluOS machines is approved and **not yet built**. Until it
  ships, the Spark is a plain Ollama backend of a MaluOS gateway; the gateway provides the
  queue, policy, budgets and ledger for MaluOS callers.
- Ollama has **no authentication**; LAN or VPN plus a firewall is the boundary.

## Procedure

### 1. Ollama on the Spark

```sh
nvidia-smi                                  # driver must work first
curl -fsSL https://ollama.com/install.sh | sh      # arm64 supported; creates user, unit, /usr/local binary
ollama --version
systemctl status ollama
```

Read the script before running it if the person's policy requires. Expected: the installer
reports the NVIDIA GPU; the `ollama` service is active.

### 2. Bind to the network (Spark)

```sh
sudo systemctl edit ollama
```
```ini
[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
```
```sh
sudo systemctl restart ollama
ss -ltnp | grep 11434                       # 0.0.0.0:11434
sudo ufw allow from 192.168.1.0/24 to any port 11434 proto tcp    # if ufw is active; LAN only
```

Verify **from the MaluOS host**: `curl http://spark.lan:11434/` answers "Ollama is running".

### 3. Models (Spark)

```sh
ollama pull qwen3:32b
ollama pull llama3.3:70b                    # 128 GB unified memory fits ~70B at default quantisation
ollama pull nomic-embed-text
ollama ps
```

### 4. Backend on the MaluOS host (operator)

```sh
sudo maluctl gateway backend add spark --kind ollama_local --endpoint http://spark.lan:11434 \
    --models qwen3:32b,llama3.3:70b,nomic-embed-text --max-concurrent 1 --keep-alive 30m
maluctl gateway backends
sudo maluctl gateway alias set strong   --tier strong   --backend spark --model llama3.3:70b     --position 0
sudo maluctl gateway alias set standard --tier standard --backend spark --model qwen3:32b        --position 0
sudo maluctl gateway alias set embed    --tier embed    --backend spark --model nomic-embed-text --position 0
sudo maluctl gateway catalog
```

### 5. Test (MaluOS host)

```sh
maluctl gateway chat "Say hello from the Spark." --model strong
maluctl gateway routes                      # route row names spark
maluctl gateway usage --by backend
```

`scripts/verify.sh spark.lan:11434 spark` runs these checks from the MaluOS host.

## Facts that shape advice

- `ollama_local` backends are treated as local: the sensitivity gate does not apply. For a
  gated path, pair the Spark with one MaluOS GPU server and let desks use that as `remote`.
- Several MaluOS hosts can share one Spark; each queues on its own.
- A Spark that was installed before the NVIDIA driver worked runs Ollama on the CPU; fix the
  driver and reinstall Ollama.

## Stop and ask before

- running a vendor install script as root on the person's machine;
- opening the firewall beyond the LAN;
- pulling models larger than the free disk.

## Never

- install MaluOS packages or the MaluOS apt source on an arm64 machine;
- expose port 11434 to the internet;
- claim the node service, keys or a shared queue exist on the Spark until the repository ships them.

## Troubleshooting map

| Symptom | Where | Fix |
|---|---|---|
| backend unhealthy | MaluOS host: `curl http://SPARK:11434/` | `systemctl show ollama -p Environment`; firewall |
| Ollama on CPU | Spark: `nvidia-smi`, install log | fix driver, reinstall Ollama |
| `model_missing` | Spark: `ollama list` | pull it |