Cray XC40 system is the latest entry to SERC’s HPC class of systems. This Cray XC40 is a system that combines the capabilities of Intel’s latest Xeon Haswell processors for the CPU cluster and Nvidia’s K40 series of GPU cards and Intel’s Xeon-Phi 5120D cards for the accelerator cluster connected using Crays’s own Aries high speed interconnect on a dragonfly topology with DDN’s high performance storage units.
CPU only cluster: Intel Haswell 2.5 GHz based CPU cluster with 1376 nodes; each node has 2 CPU sockets with 12 cores each, 128GB RAM and connected using Cray Aries interconnect.
Accelerator based clusters: Two accelerator clusters one with Nvidia GPU cards (44 nodes) and the other with Intel Xeon-Phi cards (48 nodes) for the accelerator clusters. Each node on the accelerator cluster has Intel IvyBridge 2.4 GHz based single CPU socket with 12 cores and one GPU or the Phi card with 64GB RAM. The GPU card is Tesla K40 card with 2880 cores and 12GB device memory. The Intel Xeon-Phi Coprocessor 5120D has 60 cores with 8GB device memory.
High Speed Storage: 2 PB usable space provided by high speed DDN storage unit supporting Cray’s parallel Lustre filesystem.
Software environment: The entire system is built to operate using Cray’s customized Linux OS, called Cray Linux Environment, for the cluster supporting architecture specific compilers from Cray as well as Intel and open-source based Gnu compilers. System also hosts architecture specific parallel libraries like OpenMP, MPI, CUDA and Intel Cluster software. Extensive range of parallel Scientific and Mathematical libraries like BLAS, LAPACK,Scalapack, fftw, hdf5, netcdf, PETSc, Trilinos etc. are also available on the system. To facilitate users with parallel program development latest version of the DDT parallel debugger and profiler is enabled for use on the system.
The system is accessible using login nodes with the domain name xc40.serc.iisc.ernet.in and can be used for launching jobs in batch execution mode. User is expected to ssh into the login nodes and create appropriate job scripts and submit these scripts to the PBSPro batch scheduling software. The parallel file system is accessible to users through job scripts and is meant for using during job execution.
OEM – Cray Inc
Ongoing Training Sessions
Accessing the system
The XC40 has login nodes, through which the user can access the machine and submit jobs. The machine is accessible for login using ssh from inside IISc network (ssh firstname.lastname@example.org). The machine can be accessed after applying for Cray XC40, for which:
Fill the online HPC application form here and submit at Room no:117, SERC.
HPC Application form must be duly signed by your Advisor/Research Supervisor.
For any queries, email to helpdesk_serc or please contact System Administrator, #109,SERC.