DEISA Infrastructure and Resources
The DEISA research infrastructure is constituted by leading national supercomputers in Europe interconnected with a high bandwidth point to point network provided by GEANT2 and the National Research Networks (NRENs). High bandwidth network connectivity is required to guarantee the high performance of the distributed services, and to avoid performance bottlenecks.
DEISA incorporates several different platforms and operating systems (IBM AIX on Power5-6, IBM Linux on PowerPC, SGI Linux on Itanium, Cray XT, and NEC vector systems), and the consortium has deployed middleware that enables the transparent access to distributed resources, high performance data sharing at European scale, and transparent job migration across similar platforms.
Schematic of the DEISA Infrastructure based on a dedicated network provided by GEANT2 and the National Research and Education Networks (gray colored).
The DEISA network provided by GEANT2 and the National Research and Education Networks offers dedicated bandwidth connecting the supercomputers which are spreading across Europe. This internal DEISA network has been built in addition to the standard Internet connectivity offered by each national supercomputer centre. Nearly all DEISA sites are now connected via 10 Gb/s lines. The current network uses dedicated wavelengths on NRENs and GEANT2 fibre links footprint. It is managed by DEISA itself and operates at 10 Gb/s between nearly all DEISA sites.
Schematic of the DEISA dedicated network provided by GEANT2 and the National Research and Education Networks.
The portfolio of computing resources incorporates several platforms and operating systems.
- Cray XT5 (Linux)
- Intel Xeon Nehalem (Linux)
- IBM Power6 (AIX, Linux)
- IBM BlueGene/P (Linux)
- IBM PowerPC (Linux)
- SGI ALTIX 4700 (Linux)
- NEC SX-9 vector systems (Super UX)
The aggregated power of the DEISA supercomputing resources exceeds two PetaFlop/s.
Each supercomputing system has a common data space, a shared file system which is transparently accessible from every compute node in the cluster at one site. Such a cluster-wide shared file system offers a single system view to compute jobs which are running locally on the cluster.
The DEISA Global File System extends this site-local data sharing model to a wide-area multi-cluster shared file system that is available within the DEISA HPC infrastructure. It supersedes the necessity for users to stage data explicitly from site to site before their compute jobs starts execution at a remote site.
Schematic of the DEISA global shared file system based on IBM MC-GPFS (gray colored) and a dedicated network provided by GEANT2. The computing resources shown can be replaced of upgraded according to the table of provided resources. Some sites also have more than one cluster integrated in the DEISA Global File System.
Starting from a proof of concept utilizing IBM's Multicluster General Parallel File System (MC-GPFS) that was deployed on several AIX systems, DEISA has nowadays reached a status of providing a European-wide shared file system based on MC-GPFS that is available on various compute platforms of different vendors.
The extension of MC-GPFS from a homogeneous operating system environment to a heterogeneous multi-vendor environment has been enabled by adaptating IBM's GPFS client software specfically to the PowerPC-Linux platfom, the SGI Altix system and the Cray XT system. Thus, the DEISA Global File System integrates the IBM AIX-based supercomputers with Power6 processors, the Linux-based systems, the SGI-Altix systems with Itanium processors and the Cray XT systems.
New developments in the area of global shared file systems are monitored, and alternative technologies and file systems, such as Lustre and NFSv4, are subject of further investigation.
As an alternative solution with special regard to supercomputers for which the appropriate MC-GPFS client is not yet available or which have the DEISA GPFS temporarily not mounted, DEISA provides the possibility to stage data from site to site via Globus GridFTP. A corresponding infrastructure of GridFTP servers that are accessible via the DEISA internal network has been established.