his is the README file for the NAMD application benchmark, distributed with the DEISA Benchmark Suite: http://www.deisa.eu/science/benchmarking/ Last modified by the DEISA Benchmark Team on 2009-12-03. ----------- NAMD readme ----------- Contents -------- 1 General description 2 Code structure 3 Building and Executing 3.1 Charm++ 3.2 NAMD 4 Input Data 5 Output Data 6 References 1 General Description ===================== NAMD is a parallel, object-oriented molecular dynamics code designed for high-performance simulation of large biomolecular systems. NAMD uses a variant of spatial decomposition that uses cubes whose dimensions are slightly larger than the cutoff radius. Thus, atoms in one cube need to interact only with their neighboring cubes; there are 26 such neighboring cubes. For each pair of neighboring cubes, NAMD assigns a non-bonded force computation object, which can be independently mapped to any processor. The number of such objects is therefore 14 times (26/2 + 1 self-interaction) the number of cubes. Forces due to covalent bonds within biomolecules are represented via a sum of 2-body (bond), 3-body (angle), and 4-body (dihedral and improper) terms which follow the topology of the molecule. In order to minimize communication while avoiding redundant calculations, a force computation object is created for each cube and its upstream neighbors, these being the (at most) 7 neighboring cubes at equal or greater coordinates along all three axes. Bonded forces among sets of (2, 3, or 4) atoms are calculated by this object if and only if the base cube coordinates are equal to the minimum of the cube coordinates for all constituent atoms along each axis. NAMD works with AMBER and CHARMM potential functions, parameters, and file formats. For workstation clusters and other massively parallel machines with special high-performance networking, NAMD uses the system-provided MPI library (with a few exceptions) and standard system tools such as mpirun are used to launch jobs. Since MPI libraries are very often incompatible between versions, you will likely need to recompile NAMD and its underlying Charm++ libraries to use these machines in parallel (the provided non-MPI binaries should still work for serial runs.) The provided charmrun program for these platforms is only a script that attempts to translate charmrun options into mpirun options, but due to the diversity of MPI libraries it often fails to work. Charm++ is an object-oriented message passing parallel programming model which has mainly been used in realm of HPC and scientific computing. It can be used to write parallel applications that run on a variety of machines ranging from a single notebook/desktop computer to computer clusters containing tens of thousands of processors. NAMD web site: http://www.ks.uiuc.edu/Research/namd/ Charm++ web site: http://charm.cs.uiuc.edu/ For the DEISA benchmarking we are using version 6.1 of Charm and the version 2.7b1 of NAMD. 2 Code structure ================ Here you will find a brief description of the files structure for this code. README.txt this file. input/ folder containing the inputs for the benchmark (apoa1, 1GND, 1CQY, nucleosome_146K_atoms and membrane_716K_atoms). run/ folder where executable should be after the compilation. src/charm-6.1 source files of charm++ (needed to compile NAMD). buildCharm.in This file configures the way that charm++. have to be compiled using the 'build' script. Don't touch this file, It acts as a template file. build Script to build Charm++. src/ Source files of the program. doc/ Documentation about charm++. src/NAMD_2.7b1_Source source files of NAMD. arch/ folder containing all known architectures customizations for the compiling. Each configuration file provides architecture-depending compilers and the flags compiler must use. It also provides the location of FFTW and TCL include and libraries. buildNamd.in This script is used to configure the environment for each platform where benchmark is going to be compiled. It is similar to 'buildCharm.in' so you should not modify it. config This script creates a folder with all needed files for the compilation (lib, include, soft-links ...). src/ All source files needed to compile NAMD. 3 Building and Executing ======================== - Charm++ First, you need to decide which version of Charm++ to use. The "build" script in Charm++'s source directory takes several command line arguments to compile Charm++. The command line syntax is: build [options ...] [charmc-options ...] For detailed help messages, use -h or --help to the build script, i.e. ./build --help REQUIRED: --------- specifies the parts of Charm++ to compile. The most often used is "charm++", which will compile the key Charm++ executables and runtime libraries. Other common targets are "AMPI" and "FEM". defines the CPU, OS and Communication layer of the machines. See "How to choose a " below for details. Running the build script, a directory with the name as a combination of "version" and "options" like "---..." will be created and the build script will compile Charm++ under this directory. For example, on an ordinary Linux PC: ./build charm++ net-linux will build Charm++ in the directory: net-linux/. The communication defaults to UDP packets and the compiler to gcc. When Charm++ is built successfully, the directory structure under the target directory will look like: net-linux/ | --- bin/ # all executables | --- doc/ # documentation | --- include/ # header files | --- lib/ # libraries | --- lib_so/ # dynamic libraries | --- examples/ # all example programs | --- tests/ # all test programs | --- tmp/ # Charm++ build directory - NAMD Building a complete NAMD binary from source code requires working C and C++ compilers, Charm++/Converse, TCL, and FFTW. NAMD will compile without TCL or FFTW but certain features will be disabled. Set up build directory and compile: ./config [tcl] [fftw] cd make For example: ./config fftw Linux-ia64-icc cd Linux-linux-ia64-icc make Within JuBE, some XML configure files: one for each platform that this version of NAMD has been ported: - Babel, IBM BlueGene/P Proc: PowerPC 450 850 MHz 2GB RAM per node - HECToR, Cray XT4 Proc: AMD Opteron 2.3 GHz quad-core 2 GB memory per core - Jump, AIX Proc: IBM Power4 1.7 Ghz - Louhi, Cray XT4 Proc: Dual-core AMD Opteron processor 2,6 GHz - LRZ-HLRB2, SGI's Altix 4700 Proc: Intel Itanium2 Montecito Dual Core - MareNostrum, PowerPC cluster Proc: JS21 Blades: Two Dual-core chips PPC-970MP 2.3 GHz - VIP, IBM eServer 575 Proc: Power6 4.7 GHz dual-core - JuGENE, IBM BlueGene/P Proc: PowerPC 450 850 MHz 2GB RAM per node - SP6, IBM pSeries 575 Proc: Power6 4.7 GHz dual-core Those files contain the description for each possible input with the parameters that must be configured for the benchmarking. The parameters are number of cpus, tasks per node, name of the benchmark, its input file location ... There are also two extra XML files in order to compile and execute the benchmark. compile.xml file describes all the steps that system have to do for compile Charm and NAMD source code depending on each platform. execute.xml file describes the format and parameters for the submitting file on each platform. Both compile and execute files get needed data from XML file for each platform. About the known problems: A couple of problems were found when porting the code to these three platforms. In Louhi, the default C and C++ compiler version is 7, but there were some problems and known bugs compiling NAMD with that version, those error were fixed switching the C and C++ compilers to version 6. In the Altix, the problem was that initial source code of NAMD is prepared to the use the mpich library. The problem was solved replacing the -lmpich flag for -lmpi. 4 Input Data ============ In first place, we selected the structure of ApoA1, which models a bloodstream lipoprotein particle involving 92.000 atoms. ApoA1 is one of the most used data sets for benchmarking using NAMD. By the other hand, the other two structures selected were chosen among a list of 30 structures prepared for the MoDEL project (1). This list of proteins was built to simulate the 30 most populated folds according to SCOP (30, 31), CATH (32), Dali (33, 34), and Dagget’s databases (35) (see SI Data Set). When several good three-dimensional models exist in a class, we favor structures without bound ligands. For the purpose of benchmarking, we selected a small one (<100 residues), a big one (>400 residues). This allowed us to see how the change in system size affects the scalability of the application and which is the best number of processors for a certain amount of atoms in the simulation. Two additional datasets were added subsequently (nucleosome_146K_atoms and membrane_716K_atoms); corresponding to a medium (146K atoms) and large (716K atoms) sizes dataset. This datasets are tunned to run in a reasonable time-frame to avoid wasting cycles and at the same time allow to explore the effect of different dataset sizes in scalability. Example of input file for NAMD: (apoa1.namd) ----------------------------------------------------------------------- cellBasisVector1 108.8612 0.0 0.0 cellBasisVector2 0.0 108.8612 0.0 cellBasisVector3 0.0 0.0 77.758 cellOrigin 0.0 0.0 0.0 # initial config coordinates apoa1.pdb temperature 300 seed 74269 # force field params switching on switchdist 10 cutoff 12 pairlistdist 13.5 margin 0 stepspercycle 20 PME on PMEGridSizeX 108 PMEGridSizeY 108 PMEGridSizeZ 80 structure apoa1.psf parameters par_all22_prot_lipid.xplor parameters par_all22_popc.xplor exclude scaled1-4 1-4scaling 1.0 timestep 1.0 fullElectFrequency 4 # protocol params numsteps 500 outputtiming 20 # output params outputname ./apoa1-out ----------------------------------------------------------------------- This input instructs NAMD to do 500 steps of conjugate-gradient energy minimization to remove steric clashes within the guessed geometry. 5 Output Data ============= Browse trough .out to see if the simulation remains stable (energies remain roughly constant). 6 References ============ 1. Rueda et al., "A consensus view of protein dynamics", 104 (3) 796, PNAS. http://www.pnas.org/cgi/reprint/104/3/796 30. Andreeva A, Howorth D, Brenner SE, Hubbard TJ, Chothia C, Murzin AG (2004) Nucleic Acids Res 32:D226–D229. 31. Murzin AG, Brenner SE, Hubbard T, Chothia C (1995) J Mol Biol 247:536–540. 32. Pearl F, Todd A, Sillitoe I, Dibley M, Redfern O, Lewis T, Bennett C, Marsden R, Grant A, Lee D, et al. (2005) Nucleic Acids Res 33:D247–D251. 33. Dietmann S, Park J, Notredame C, Heger A, Lappe M, Holm L (2001) Nucleic Acids Res 29:55–57. 34. Holm L, Sander C (1995) Trends Biochem Sci 20:478–480.