Project Structure
Work Packages
The DEISA2 project is structured into nine work packages. Two management and communication-related work packages, five work packages categorised into three types of Service Activities (SA), and two Joint Research Activities (JRA). The work packages are structured into tasks and subtasks. Each work package is managed by a work package leader.
The following table lists all the Work Packages of the DEISA2 Project.
| Package | Scope | Description |
|---|---|---|
| WP1 | Management | Management of all aspects of the project, coordination of the integration of the work packages, quality control, financial control, communication with the EU, support of the scientific management of resources. |
| WP2 | External Relations, Dissemination and Training | Initiation and development of external relations with partners, organisations and communities, as well as dissemination that will target investors, partners, stakeholders, science communities and the public. Furthermore the training activities are carried out to enable potential users to utilise the DEISA infrastructure efficiently. |
| WP3 (SA1) | Operations | Operating the infrastructure including all services, adopting approved new technologies from SA2, advancing the operation of the DEISA HPC infrastructure to a turnkey solution for the future European HPC ecosystem by improving the operational model and integrating new sites. |
| WP4 (SA2) | Technologies | Monitoring technologies already in use in the project, identifying and selecting technologies of relevance for the project, evaluating technologies for pre-production deployment, planning and designing specific sub-infrastructures to upgrade existing services or deliver new ones based on approved technologies. |
| WP5 (SA3) | Applications enabling | Enhancing scientific applications from the DEISA Extreme Computing Initiative, Virtual Communities and EU projects to cope with a production infrastructure characterized by an aggressive parallelism on heterogeneous HPC architectures at a Europe-wide scale. |
| WP6 (SA3) | Environment and User Related Application Services | Maintenance and improvement of the DEISA application environment and interfaces, DEISA-wide user support in the applications area. |
| WP7 (SA3) | Extreme Computing Projects and Benchmark Suite | Coordination and organisational handling of the DEISA Extreme Computing Initiative (annual calls for HPC applications), and the provision and maintenance of a European Benchmark Suite for supercomputers. |
| WP8 (JRA1) | Integrated DEISA Development Environment | Provision of an integrated environment for scientific application development, based on a software infrastructure for tools integration, which provides a common user interface across multiple computing platforms. |
| WP9 (JRA2) | Enhancing Scalability | Enabling supercomputer applications for the efficient exploitation of current and future supercomputers. |
Management and Communication-related activities
The management and communication-related tasks are grouped into two work packages. WP1 concerns the overall project management. WP2 comprises external relations, dissemination and training activities.
Service Activities
The tasks of the Service Activities are structured into five work packages according to three different categories of services:
-
Operations: Operation of the DEISA HPC infrastructure (SA1)
Objective: advancing the persistent distributed European HPC environment to a turnkey operational solution for the European HPC ecosystem (SA1)
-
Technologies: Exploration and adoption of new technologies (SA2)
Objective: reinforcing the technology roadmap
-
Applications: Applications- and user-related services fostering the exploitation of the DEISA HPC infrastructure (SA3)
Objective: continuing and advancing computational sciences in Europe
As DEISA is establishing and operating a distributed HPC environment on a continental scale, European-wide operating counterparts of the three departments "Operations", "Technology", and "Applications" which usually constitute the major departments of an HPC centre have to be
organized and coordinated. Correspondingly, three European-wide operating teams are being established
- the Operations Team operates the whole DEISA HPC infrastructure
- the Development and Technology Team ensures and advances the technological standards across the distributed HPC infrastructure
- the Applications Support Team supports users and user communities to optimize their benefit from the DEISA infrastructure
Together, these three service activities advance the existing DEISA infrastructure towards an operations model which can be adopted as a turnkey solution by the future European HPC ecosystem. The following figure illustrates the relation and the communication flow between the three Service Activities.
The Operations Service Activity (WP3)
WP3 deals with the day-to-day operation of the overall DEISA HPC infrastructure. The corresponding Operations Team is continuing its collaborative work that started in the previous DEISA1 and eDEISA projects.
| Task |
Description |
|---|---|
| Network-Related Service |
Connecting the different HPC systems of the principal partner sites with a dedicated high-performance network (currently mostly with 10Gbit/s) in order to form and maintain a tightly coupled, distributed HPC infrastructure, using the network provided by GÉANT2 together with the local National Research Network provides is the basic means of networking for DEISA. Operation and maintenance of the network is one fundamental task. |
| Data-Related Services |
Based on the networking infrastructure, a global file system infrastructure (Multi-Cluster GPFS) has been deployed as one of the core components of the DEISA infrastructure. This global file system has to be operated and kept highly available. Another duty is the operation of facilities for data transfer between the DEISA global filesystem and external data storages. |
| Compute-Related Services |
The global file system allows to execute compute jobs at any DEISA site without any need to stage data from site to site. The operation of adequate job submission and job management facilities which work across the DEISA HPC clusters, also allowing for job rerouting and job migration in a homogeneous multi-cluster system environment, is one aspect of this task. Other duties are the operation and maintenance of Grid middleware (e.g., UNICORE) as well as the monitoring of the status of the infrastructure components using the Resource Monitoring and Information System (RMIS) provided by the Technologies Work Package. |
| AAA-Related Services |
The duty of this task is the day-to-day operation and maintenance of authentication, authorization, and accounting services. This covers the support of the user administration and account management including the support of the PKI infrastructure withing DEISA and a security management. |
| User-Related Services |
Duties are the provision and maintenance of user interfaces and science gateways (portals) as well as the provision of the first-level help desk (using a trouble-ticket-system). The direct support of users, especially applications-specific support is the duty of WP7 -WP9 (SA3: “Application and User Support”). |
| Integration Support for Associate Partners |
With regard to the envisaged HPC ecosystem, DEISA aims at opening its infrastructure for non-DEISA HPC centres. Exploring and offering ways to connect external HPC centres to the DEISA infrastructure is the major duty of this support task. Initially, operational support is provided to a few HPC centres, on request. During the process of connecting new HPC sites with the DEISA infrastructure, "best practices" will be developed resulting in a
corresponding checklist which will help to advance the operational model for the future European
HPC ecosystem. |
The smooth operation of the DEISA infrastructure requires a close active cooperation between the sites' staff members working on the first five tasks.
The local operation teams at each DEISA site are responsible for the DEISA-specific daily work. These teams are led by a site representative who will in turn coordinate all activities between all the DEISA sites. The site representatives ensure that any actions (e.g., local maintenance actions) performed at their site are not interfering with the overall DEISA services and that the quality of service within the HPC infrastructure is not significantly affected.
These site representatives for operations (in the figure they are referred to as “Deputys”) form the Operations Team. The figure above illustrates the services and relations of the DEISA OperationsTeam.
The Technologies Service Activity (WP4)
The service activtiy in WP4 comprises the roadmap and tasks for exploring new technologies.
The objectives are
- monitoring technologies already used by DEISA
- identifying and selecting relevant new technologies of potential interest to the project,
- evaluating technologies for pre-production deployment purposes,
- planning and designing specific
sub-infrastructures using
the selected technologies in order to foster existing services or to deliver new services,
- deploying and testing the new sub-infrastructures on the DEISA HPC infrastructure in production.
| Tasks |
Description |
|---|---|
| Network-Related Technologies |
This task aims at exploiting GÉANT2’s future network services for DEISA at its optimum as well as providing enhanced technologies for DEISA network performance and security monitoring. |
| Data-Related Technologies |
The evaluation and advancement of the shared multi-cluster storage and data management technologies is one fundamental task of this service activity. It comprises the exploration and evaluation of new or existing technologies for file management (e.g., wide area network shared file systems, file transfer and file staging facilities, hierarchical storage management) as well as data management facilities (e.g. distributed data bases). With regard to a further technology evolution in this area, this task aims at providing the most effective and easy-to-use file and data management facilities for DEISA. |
| Compute-Related Technologies |
Another fundamental service is the user-based submission, control, and monitoring of supercomputing applications that are running on distributed HPC systems. This technology domain comprises job management (via batch schedulers such as PBSpro, Loadleveler, LSF), middleware providing a consistent abstraction layer over the underlying job management systems (e.g. UNICORE) as a seamless interface in an heterogeneous environment, and both job execution and system monitoring tools. |
| AAA-Related Technologies |
Services concerned with authentication, authorisation and accounting (AAA) are the key to any distributed HPC infrastructure. They allow to identify end entities (servers and users) securely (authentication), to enforce access policies for users and servers (authorization), and to gain information about the resource usage that allows to operate an accounting-based access control system. This task examines existing and new AAA technologies, and, at the same time, empower the available AAA infrastructure of DEISA in order to provide advanced services and to foster scalability (with regard to the prospective integration of non-DEISA sites) as well as to guarantee a high degree of security. |
| User-Related Technologies |
This task aims at simplifying the user access to the DEISA HPC infrastructure, both regarding data and compute aspects. The ability to exploit different HPC platforms in an heterogeneous environment requires significant effort from the users. Learning all the operating system, job management, parallel programming techniques and software environments is not trivial. For this reason, many users need interfaces which hide as many details of the technical environment as possible. With these facilities, often referred to as portals, rich or thin clients, the users can focus efforts on their applications without having to bother with the underlying system-specific issues. In addition, tools for the visualisation of data can increase the user's productivity significantly. Therefore, this task will investigate and evaluate portals and portal technologies, user interfaces on the client side, and visualisation tools in order to hide the technical complexity of the heterogeneous DEISA infrastructure as much as possible. |
In fact, these five tasks are not independent and partly build upon each other. The following figure illustrates that each objective (A-E) mentioned above is concerned with all the five tasks.
The Applications Service Activity (WP5-7)
The Applications- and User-related Service Activitiy (SA3) is structured into three work packages: WP5 (Applications Enabling), WP6 (Environment and User Related Application Services), and WP7 (Extreme Computing Projects and Benchmark Suite).
Two work packages are classified as Joint Research Activities that are relevant for the project: WP8 is dedicated to the development of an Integrated DEISA Development Environment, which is a tool that supports developing applications across the heterogeneous DEISA HPC infrastructure. WP9 specifically supports WP5 on enhancing applications scalability.
