5x A100 setup finally complete
gpu4.pics
via BreakIt-Boris
NVIDIA A100 80GB
5× NVIDIA A100 40GB · 200 GB total VRAM
80 GB VRAM NVIDIA 400 W TDP
running Goliath 120B· GGUF8-BIT
12
tokens / sec
Generation
12tok/s
higher is better
Prompt / prefill
—tok/s
higher is better
Time to first token
—
lower is better
Efficiency
0.03 tok/s/W
Electricity cost
$1.77/Mtok
Power draw
450 W
All benchmarks on this rig
| Model | Quant | Gen | Prompt | TTFT | Power | Perf/W | $/Mtok | Score |
|---|---|---|---|---|---|---|---|---|
| Goliath 120B | GGUF8-BIT | 12 | — | — | 450 W | 0.03 tok/s/W | $1.77/Mtok | — |