CUBIC (HPC Cluster)
The primary computing resource at CBICA is a RedHat Enterprise Linux-based HPC cluster.
The cluster is connected to the University of Pennsylvania Hospital System network (HUPNet), secured by the hospital system firewall.
The cluster is housed in secure datacenter, with 24x7 monitoring, redundant power, cooling and network connections.
The cluster is used for interactive work and software development, as well as batch processing of image data. Image processing jobs in the lab use the SoGE queuing system to efficiently manage resources for both parallel and serial computing jobs.
The CUBIC cluster consists of 168 compute nodes, providing a total of 4804 CPUs with and 58TB of RAM. In support of applications that require large amounts of memory, 3 nodes have 3TB of RAM each. The nodes have a total of 162TB of local storage for installed software, temporary data, and network filesystem caching.
The cluster has 54 batch nodes that contain 104 NVidia P100 and V100 GPU processors.
Compute nodes communicate directly via a 10Gb/sec network to a 40Gb/s network core, and have access to approximately 600TB of highly-available storage.
The cluster has nodes for interactive work that can be used for software development, visualization of results, and submission of jobs to the batch computing nodes. Each interactive node has an NVidia GPU for prototyping GPU jobs.
Several highly-available infrastructure servers manage data backups, a source code revision repository, an internal network information system, the CBICA public website, and internal documentation systems.