Add boxes

One box runs Garrison with a capable model. More boxes let you run larger models, which give better investigations and summaries. Boxes are added from the console in a few minutes; models then move or download on their own.

How boxes combine

Boxes are joined directly with a QSFP cable; no network switch is needed. That allows three layouts:

Layout	Cabling	What it gives you
Pair (2 boxes)	One QSFP cable between the two	Models too large for one box run across both
Box plus a pair (3 boxes)	The two new boxes cabled to each other; the first box on the LAN only	The first box keeps Garrison and a fast model; the pair runs a large model. Complex questions go to the pair, simple ones stay local
Standby box (2 boxes)	LAN only	A second complete appliance that takes over if the first fails (planned)

  ┌─────────┐                ┌─────────┐ port 1 ══QSFP══ port 1 ┌─────────┐
  │ box 1   │                │ box 2   │                        │ box 3   │
  └────┬────┘                └────┬────┘                        └────┬────┘
       └─────────────────── LAN ──┴──────────────────────────────────┘
                    box plus a pair

Cable port 1 to port 1. Port 0 is often already in use; the console detects which ports are actually connected.

Join a box

1. Rack the new box, connect it to the LAN, and cable it if it will be part of a pair. It needs DGX OS as delivered, like the first box.

2. In the console, open Platform → Cluster and click Add boxes. Choose how many boxes you are adding. You get one command, valid for one hour and for that many boxes.

   

3. Run that command on each new box (over SSH, as a user with sudo). It checks the box, enrols it with the appliance’s certificate authority and starts the node agent. It downloads everything from the first box, so the new box needs no internet access.

4. The box appears as pending, with its hardware and its cabled ports. The console works out which ports are cabled to which by sending a short test over each cable.

5. Pick the layout the console recommends. It shows the change it will make (the new boxes, the cable addresses, the model to place, how questions are routed) for you to review.

6. Apply. The console brings up the cable links, checks their speed, copies the model files across the cable, starts the model and runs the acceptance checks.

The new boxes also appear in Systems once you enrol their Garrison agents (Linux systems); give them the Log collector role so their metrics show on the Garrison Cluster dashboard.

Headless boxes

DGX OS starts a desktop on each box, which holds about 2 GiB of memory a model could use. On the Cluster page, Desktop GUI → headless turns it off (and Bring the desktop back turns it on). SSH and the console are not affected; nothing reboots.

Placing models

After boxes join, choose which model runs where under AI Gateway → Models: add a model, pick the box or pair, and approve. See AI Gateway.