Compute hardware
A cluster consists of multiple nodes connected to one or more central filesystems. A node is basically a single computer, roughly comparable to a powerful desktop machine. Some nodes are networked together with fast connections (Infiniband) that enable efficient communication between nodes, allowing large jobs to run in parallel on multiple nodes. Finally, some nodes include GPUs (graphical processing units), which can accelerate certain compute jobs.
The different types of nodes available on Roar are:
| Resource | Count | Cores | Memory (GB) |
CPU | CPU Family |
Network |
|---|---|---|---|---|---|---|
| Basic (4 GB/core) |
120 | 64 | 256 | Gold 6430 | sapphirerapids | Ethernet |
| Standard (8 GB/core) |
140 35 |
48 64 |
512 384 |
Gold 6342 EPYC 9354 |
icelake AMD Genoa |
Infiniband |
| 2xGPU A100 (40GB) |
38 | 48 | 384 | Gold 6248R | cascadelake | Infiniband |
| GPU V100 (32GB) |
2 | 24 | 512 | E5-2680v3 | haswell | Ethernet |
| 4xGPU V100 (32GB) |
2 | 24 | 512 | Gold 6132 | skylake | Ethernet |
| GPU A40 (48GB) |
12 | 36 | 1024 | Gold 6354 | icelake | Ethernet |
| GPU P100 (12 GB) |
60 | 28 | 256 | E5-2680v4 | broadwell | Infiniband Ethernet |
| High Memory (20 GB/core) |
25 | 48 | 1024 | Gold 6342 | icelake | Infiniband |
| Interactive | 16 | 28 | 512 | E5-2680v4 + P100 GPU |
broadwell | Infiniband Ethernet |
Note: While cluster nodes are configured with the memory described above, some of the memory is reserved for the operating system and system processes, so not all memory is available for user jobs. We normally reserve less than 10GB of memory for the operating system.