---
name: maluos-gpu-server
description: Turn a MaluOS Server with an NVIDIA RTX card into a shared GPU server: install the signed nvidia component, run local models through the gateway, set tiers, turn sharing on, issue per-client inference keys, expose the node API on port 11435, and pair desks to it as remote backends. Use when the user wants to share a GPU on the MaluOS network, pair a desk with a GPU host, issue or revoke gateway keys, or debug not_shared, unhealthy remote backend or model_missing errors.
---

# Installing a MaluOS GPU server

Human guide: https://maluos.com/howto/gpu-server/ . Two machines are involved: the **GPU host**
(serves) and each **desk** (pairs). Keep track of which one a command runs on.

## Scope check

- The host must be **x86-64 with an NVIDIA RTX card** and MaluOS Server installed
  (`maluos-first-server` skill). An NVIDIA DGX Spark is arm64 and is not a MaluOS device
  (`maluos-gpu-nvidia-arm` skill); an Apple Silicon Mac is `maluos-gpu-mac`.
- Chassis rule for sharing: a server shares always; a dedicated desktop opts in; a laptop
  opts in and shares only while idle. `maluctl gpu status` prints the rule.

## Procedure

### 1. Driver (host, operator)

```sh
sudo apt install maluos-nvidia          # signed modules for the generic kernel, driver 595 (580 alternative), nvidia-smi, nvtop
sudo reboot
maluctl gpu status                      # the card, driver, VRAM, temperature, power, chassis rule
nvidia-smi
```

If the GPU is missing after the reboot: `dmesg | grep -i nvidia`; with Secure Boot on, the
package `linux-modules-nvidia-595-generic` (signed) must be what apt installed, not a DKMS build.

### 2. Models (host, operator)

```sh
sudo maluctl gateway ollama install
sudo maluctl gateway ollama pull llama3.2:3b          # fast
sudo maluctl gateway ollama pull llama3.1:8b          # standard
sudo maluctl gateway ollama pull qwen3:32b            # strong, if VRAM allows (rule of thumb: VRAM > download size)
sudo maluctl gateway ollama pull nomic-embed-text     # embed
sudo maluctl gateway ollama status
sudo maluctl gateway alias set strong --tier strong --backend ollama-local --model qwen3:32b --position 0
sudo maluctl gateway ollama warm qwen3:32b
sudo maluctl gateway catalog
```

Verify: `maluctl gateway chat "hello" --model standard` answers; `nvidia-smi` shows the
Ollama process holding VRAM.

### 3. Sharing (host, root)

```sh
maluctl gateway share status
sudo maluctl gateway share on
```

With sharing off, a network caller's inference is refused 503 `not_shared` while the node API
still answers status.

### 4. Keys (host, root) — one per client

```sh
sudo maluctl gateway key issue --name ed-desk --class interactive --cloud-allowed no
sudo maluctl gateway key issue --name build-box --class batch --tiers standard,embed --max-concurrent 2
sudo maluctl gateway key list
```

The token (`mg_…`) is shown **once**. Hand it to the desk's owner over a trusted channel; do
not paste it into the transcript. A key is the principal `key:<name>` with its own policy on
the host. `--kind node` is for a manager, not needed for pairing.

### 5. Expose (host, root)

```sh
sudo maluctl gateway expose --bind 0.0.0.0     # TCP 11435; refuses without at least one key
maluctl gateway status
sudo ufw allow from 192.168.1.0/24 to any port 11435 proto tcp   # if ufw is active; LAN only
```

Verify from a desk: `curl http://gpu-1.lan:11435/node/v1/health`.

### 6. Pair a desk (desk, root)

```sh
sudo maluctl gateway backend add gpu-1 http://gpu-1.lan:11435 --key mg_… --models qwen3:32b,llama3.1:8b
sudo maluctl gateway alias set strong --tier strong --backend gpu-1 --model qwen3:32b --position 0
maluctl gateway backends                     # gpu-1 healthy with its models
maluctl gateway chat "hello" --model strong
maluctl gateway routes                       # route row names gpu-1
```

The key goes into the desk's vault as `system/remote-gpu-1` for the gateway's own principal.
Calls are ledgered on the desk (under the caller) and on the host (under `key:<name>`), joined
by ids. `remote_allowed` in a policy governs LAN nodes separately from `cloud_allowed`; the
sensitivity gate applies as for the cloud; no forwarding chains.

### 7. Operate (host or desk as noted)

```sh
maluctl gateway node status | queue | runs --limit 20 | models      # host
maluctl gateway node cancel ID | priority ID CLASS                   # host, operator
maluctl gateway usage --since 7d --by principal                      # either side
maluctl gateway alerts
sudo maluctl gateway key revoke ed-desk                              # host
```

`scripts/verify.sh` checks the host; `scripts/verify.sh --desk HOST:PORT` checks a pairing.

## Stop and ask before

- rebooting the host (step 1) if others use it;
- exposing the gateway (step 5): it opens a network port; confirm the bind address and the
  firewall rule with the person;
- issuing a key with `--cloud-allowed yes`: the client could then spend the host's cloud
  budget;
- revoking a key: in-flight calls finish, the client loses the backend.

## Never

- expose on `0.0.0.0` on a host with a public interface and no firewall;
- paste a key token into the transcript or a file;
- set `--max-concurrent` above 1 on a backend unless VRAM holds two generations of the model;
- pair a desk to a host over the public internet without a VPN; the node API is HTTP.

## Troubleshooting map

| Symptom | Where | Fix |
|---|---|---|
| 503 `not_shared` | host | `sudo maluctl gateway share on`; laptop host must be idle |
| `expose` refuses | host | issue a key first |
| backend unhealthy on desk | desk → host | `curl http://HOST:11435/node/v1/health`; firewall or bind address |
| `model_missing` run | host | pull the model on the host, or fix the desk's alias |
| slow, VRAM full | host | one generation at a time is by design; keep one model warm, smaller models for other tiers |