Installing a GPU server on NVIDIA ARM (DGX Spark)

Before you start

You need the Spark, or another arm64 Linux machine with an NVIDIA GPU and working drivers, on the same network as at least one MaluOS machine whose gateway will use it. The Spark keeps running DGX OS; nothing MaluOS is installed on it.

1Install Ollama on the Spark

Ollama's installer supports arm64 Linux and detects the NVIDIA driver. On the Spark, as a user with sudo:

nvidia-smi                                   # the driver works before you start
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
systemctl status ollama                      # the installer created a systemd service

What you should see: the installer reporting that it found the NVIDIA GPU, and the ollama service active. Read the script first if your policy requires it; it installs a binary under /usr/local, a system user and a unit.

2Bind Ollama to the network

By default the service listens on the loopback address only. Add the bind address to the unit with a drop-in and restart:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"

sudo systemctl restart ollama
ss -ltnp | grep 11434                        # 0.0.0.0:11434

If the machine runs a firewall, allow the port from the LAN only:

sudo ufw allow from 192.168.1.0/24 to any port 11434 proto tcp

From the MaluOS machine, curl http://spark.lan:11434/ answers Ollama is running.

3Pull the models

The Spark's unified memory is the point of the machine: pull the large models that no desk can run.

ollama pull qwen3:32b
ollama pull llama3.3:70b
ollama pull nomic-embed-text
ollama ps                                    # what is loaded and its memory

If you want Ollama's cloud models beside the local ones, ollama signin on the Spark adds them to the same daemon; whether MaluOS callers may use them is then the gateway policy's decision.

4Add the Spark as a backend of a MaluOS gateway

On the MaluOS server or desktop that will use it, as an operator. The kind is ollama_local, the gateway's native Ollama adapter pointed at an address:

sudo maluctl gateway backend add spark --kind ollama_local --endpoint http://spark.lan:11434 \
    --models qwen3:32b,llama3.3:70b,nomic-embed-text --max-concurrent 1 --keep-alive 30m
maluctl gateway backends

What you should see: spark healthy with its models. Then give its models the tier positions you want; position 0 is tried first:

sudo maluctl gateway alias set strong   --tier strong   --backend spark --model llama3.3:70b     --position 0
sudo maluctl gateway alias set standard --tier standard --backend spark --model qwen3:32b        --position 0
sudo maluctl gateway alias set embed    --tier embed    --backend spark --model nomic-embed-text --position 0
sudo maluctl gateway catalog

Models on the Spark can be managed from the MaluOS side through the gateway: sudo maluctl gateway ollama pull MODEL --backend spark, warm MODEL --backend spark.

5Test it

maluctl gateway chat "Say hello from the Spark." --model strong
maluctl gateway routes                       # the route row names spark
maluctl gateway usage --by backend

On the Spark, nvidia-smi shows the Ollama process on the GPU while it answers. A long wait on the Spark overflows to the next rung of the ladder, another host or the cloud, under the caller's policy.

6Keep the boundary honest

• Ollama has no authentication. Anyone who can reach port 11434 can use the Spark. Keep it on the LAN or a VPN and firewall the port; the MaluOS gateway is where identity, policy and budget are applied.
• The sensitivity gate does not apply to an ollama_local backend; the gateway treats it as local. If that is wrong for your data, pair the Spark with one MaluOS GPU server, which applies the gate for the desks that use it, rather than with every desk.
• Several MaluOS machines can share one Spark by each adding it as a backend; there is then no queue across them, only inside each. The node service will add the shared queue.

If something is wrong

• The backend is unhealthy. curl http://spark.lan:11434/ from the MaluOS machine; if it fails, the drop-in was not applied (systemctl show ollama -p Environment) or the firewall blocks the port.
• Ollama runs on the CPU. nvidia-smi must work before the install; reinstall Ollama after fixing the driver so it detects the GPU.
• model_missing on a call. The alias names a model the Spark has not pulled: ollama list on the Spark.
• Still stuck. Open an issue at github.com/maludb/maluos/issues.

What next

• When the node service ships, install it on the Spark beside Ollama: it adds the queue, per-caller keys and the run history on the machine itself, speaks the node API a MaluOS GPU server speaks, and the gateway then adds the Spark as a remote backend with a key.
• Give the Spark to the office. Pair it with the application server's gateway, and every enrolled desk reaches it through --model office.