Rules for DEISA Benchmark Suite
The following DEISA Benchmark Suite rules have to be adhered to so that a result of an execution becomes official and as such can be published on the DEISA website.
- Only minor changes of the source code, especially due to portability issues, are allowed. If any other changes were introduced, they must be reported with the results in order to be checked and approved.
- If a code requires a library, its name and version number is mentioned on the related README file. Replacement of the named libraries with alternative libraries is not allowed.
However, the version number of each library is provided only as a recommendation - the version of a library can be substituted for a more recent release. In such a case, the version number of the library has to be clearly mentioned when submitting the results.
- Running the complete Suite is recommended but not mandatory; partial results are accepted but will be less relevant.
- There is no restriction on the usage of the compile-line options. Nevertheless, for each code, the compile-line options used must be reported with the final results.
- Use of the C preprocessor is allowed only if it is specifically mentioned in the README file of an application. In such a case, preprocessing must be limited to those directives (couples of keys/values) described in the README file.
For each application and each dataset, the benchmark should be run on the
number of cores, and with the number of iterations/timesteps, that are
defined in the JUBE configuration file of each application. For each execution, the number of MPI processes has to be equal to the real number of cores used. Any information concerning non-standard execution (underutilised nodes, user-defined MPI topologies, MPI task affinity, etc.) must be reported.
- For each code and for each execution, the numerical results of a run must pass the automatic checking in order for them to be considered valid.