gpu4.pics
data via LocalScore
NVIDIA GeForce RTX 4060 Ti
llamafile 0.9.1
16 GB VRAM NVIDIA
running Meta Llama 3.1 8B Instruct· Q4_K_M
49.3
tokens / sec
565 · Strong
Generation
49.3tok/s
higher is better
Prompt / prefill
2,247tok/s
higher is better
Time to first token
612 ms
lower is better
Efficiency
0.37 tok/s/W
Electricity cost
$0.127/Mtok
Power draw
133 W
All benchmarks on this rig
| Model | Quant | Gen | Prompt | TTFT | Power | Perf/W | $/Mtok | Score |
|---|---|---|---|---|---|---|---|---|
| Meta Llama 3.1 8B Instruct | Q4_K_M | 49.3 | 2,247 | 612 ms | 133 W | 0.37 tok/s/W | $0.127/Mtok | 565 |