Grid Computing with Relativistic Quantum Chemistry Software

J Grid Computing (2014) 12:681–690 DOI 10.1007/s10723-014-9309-4

Grid Computing with Relativistic Quantum Chemistry Software Case Study of Code Adaptation and Effective Deployment Miroslav Iliaˇs · Miroslav Dobrucký

Received: 22 August 2013 / Accepted: 13 August 2014 / Published online: 23 August 2014 © Springer Science+Business Media Dordrecht 2014

Abstract We describe the adaptation of the scientific software DIRAC for the grid environment and provide the study of the code’s performance. The most effective approach for effective utilizing of the grid computing power is not relying upon program’s specific dependencies installed on grid servers, but rather in precise, local preparation of all necessary static executables and related scripting launchers. Running few functionality and performance tests before carrying out any production calculations with the code is recommended. Keywords DIRAC program suite · Adaptation for grid computing · Relativistic quantum chemistry · Effective grid utilization · Static executable

1 Introduction Grid is a collection of heterogeneous computing resources (single computers or clusters) from various M. Iliaˇs () Department of Chemistry, Faculty of Natural Sciences, Matej Bel University, Tajovského 40, SK-97400 Banská Bystrica, Slovakia e-mail: [email protected] M. Dobrucký Institute of Informatics, Slovak Academy of Sciences, Dúbravská cesta 9, SK-84507 Bratislava, Slovakia e-mail: [email protected]

administrative domains, equipped with appropriate middleware. The main idea behind grid network is the sharing of computational resources capable for massive distribution of independently running execution tasks and the data sharing and the cooperation, i.e. software code sharing. The reader can learn about grid from number of textbooks, let us mention few [1–4]. Scientific grid computing [5] has gained popularity as application software from various scientific fields can be executed in the grid computing network. Computational quantum chemistry software is non-separable part of the manifold of scientific software being executed daily on various grids in the world. Many published works, however, deal with top-layer software for managing the workflow of a particular software. For example, Nishikawa and coauthors [6] proposed intelligent scheduler for Gaussian quantum chemistry code tasks. Bardridge et al. [7] discussed prototype environment for computational chemistry, mentioning the GAMESS program suite. Another work [8] enlightened general workflow infrastructure for computational chemistry on the grid. Molecular dynamics and visualisation in the Grid was highlighted in the work by Constantini et al. [9]. The aim of this paper is, however, not preparing an intermediating tool communicating between the grid environment and the quantum chemistry code of choice. Rather, we focus on the software’s adaptation and execution in diverse Grids. To achieve

682

that, one has to adapt compilation parameters for the source code accordingly, and test the generated binary executable on accessible grid computing elements (denoted as CE, plural CEs). In order to execute an externally prepared, dynamically linked binary program on a distant grid CE, the CE must have installed all necessary libraries for execution runs. For such an executable file all dynamic objects the code is referring to must be present there. These are usually specific compilers and mathematical libraries. Also, script languages, the code is using, are to be installed on a remote CE. A more challenging way - contrary to the traditional choice of dynamical linking - is to prepare own static executable, which is carrying all objects inside the binary file and is thus totally independent on external resources. There is, however, associated risk that such prepared static executable may not work on all distant grid computers. There is some uncertainty regarding correctness of the static binary file obtained on one Linux kernel and being executed on another computer with different (Linux) operating system or other system parameters. In the electronic book [10], for example, authors claim that “...static linking causes far more problems than it solves, and should be avoided at all costs.” Yet, we choose this route here. In the context of the present work we did not strive for adjusting grid CEs for the purpose of running our specific software. We assumed here that assigned computing elements are offered to grid users as such, without possibility to demand installation of additional software. This means preparing the static executable code on our side, on a local computer, and test it on a variety of distant CEs available in the grid. One of the reasons is that it is not always possible to compile the executable on any CE without user intervention. We believe that external (what means out of a CE) generation of an executable file applicable for certain architecture is the best way for maximum usage of resources offered by a grid computing network. The application software DIRAC [11] for relativistic quantum chemical calculations has been selected for the purpose of this work. Readers interested in this topic can find out more about relativistic effects in chemical systems and about their computational treatment in standard textbooks and review papers, like [12–15]. The DIRAC code contains more than million lines of the source code written in programming

M. Iliaˇs, M. Dobrucký

languages Fortran 77, Fortran 95 with addition of C and C++ and is altogether linked with the Fortran compiler. DIRAC has one production executable, dirac.x, and all effort here is about making it properly working on as many grid CEs as possible. Finding a platform for compilation of the static executable, which will work properly - in the most optimistic scenario - on all grid CEs, is not easy, but tractable. The DIRAC web manual has a dedicated web page [16] informing users and developers about peculiarities of static buildups. Indeed, among listed cases of compilation and execution failures we report Linux distributions where the static executable behaves correctly; and, as we found, also when deployed to other servers. In the past it has been experimented with the DIRAC code in the grid environment by Gomes et al. [17]. Authors described the framework for the execution of computational chemistry codes in a grid network. Concerning the heterogeneous grid authors prepared dynamically linked binary codes for different operating systems. Contrary to their work, here, substantial effort was invested in preparing functional static binary code and in testing it on accessible grid CEs. This work is oriented for the praxis of productive scientific calculations. Its core is simple: prepare static executable suitable for distant grid computers, and provide a simple scripting framework for running it. Our aim is learning, how to create an usable executable with the best executions performances working on certain architecture and test it through practical calculations in the grid environment at hand. Regarding the usability of the DIRAC software, since one of us - M.I.- is code developer, our experience is that users tend to run their calculations on computer clusters, what involves preparing the dynamically linked binary executable locally at the cluster machine. For the DIRAC suite, we want to open and explore the new, promising field of grid computing to enable more production calculations of interesting chemical systems containing heavy elements. We also studied possibilities of the DIRAC code automated buildup and tests execution at CEs. Whenever possible on a distant grid CE, we try to perform the software configuration, compilation and execution in the same way as being done on a local computer. In cases where DIRAC code compilation on a CE

Grid Computing with Relativistic Quantum Chemistry Software

683

was not possible, static DIRAC binaries were prepared on a local computer, but sole test executions were carried out on a remote grid CE. Last, but not least, we compared execution performance of locally compiled executable against the imported, externally compiled static executable on one cluster CE, we have access to. The article is structured as follows. In the second section we provide all computational details about the adaptation of DIRAC and of related software for grid computing. In the third section we present and discuss numerical results and other achievements. Conclusions are given in the fourth part of the paper.

and with the standard open-source GNU compiler set (version 4.4.7). Scientific Linux 5 with 2 CPU Intel Pentium (x86 64 Linux architecture) was taken as the server for compilation of the DIRAC software and of other necessary programs as well. There are other hardware and software details associated with this server but we believe they are not important for the scope of this present work. If some specific details turn to be relevant for compilation or running of the static DIRAC in the Grid they shall be included in the maintained DIRAC web-manual [16]. We have adopted the so-called ILP64 integer data model what means working with integer variables of the default size of 8 bytes. This allows using wider index range, allowing thus larger amount of indexed data in calculations. On x86 64 Linux platforms, for which our static executable is suited, however, this integer model requires adapted mathematical libraries. The GNU Blas and Lapack libraries do not offer the possibility to link against 8-bytes integers, as these are designed by default for 4-bytes integers only, so in this case the Blas and Lapack library source codes have to be compiled together with the executable sources. The Intel MKL library, on the other hand, is adapted for the ILP64 data model, both for dynamic and static linking. The same feature should hold for other commercial mathematical libraries, but was not tested here. Significant speed up is achieved through the parallelization. The DIRAC software is for the most part threaded through the Open MPI standard [19]. We succeeded in preparation of the static version of the Open MPI suite with Intel compilers [18]. This brings forth the static mpif90, mpicc and mpiCC compilers, together with the mpirun launcher. Using these Open MPI compilers we generated the static parallel executable dirac.x, which can be launched with the standalone mpirun program in presence of some Open MPI files. Note that also the Intel-MKL library can be employed as multi-threaded according to the Intel internet manual [18]. This was the default choice for the serial static dirac.x generated with the Intel compilers. Usually, the simplest grid jobs consist of a submission of a mere executable, what is the easiest way for running grid computational tasks. However, scientific software’s executables are accompanied by various kind of wrapper files, what are scripts written

2 Computational Details The development version of the DIRAC program suite [11] was taken for testing in the grid environment. Progress of this code is a vibrant process. Various fixes, improvements, additions of functionalities and other multiple source code changes are appearing during the time. For obtaining stable results with respect to the development version we employed the git repository version of “84b6f4d5” with the latest change from 30.3.2013. The typical DIRAC software buildup is with dynamical links to compiler, system, mathematical and other libraries. This type of code creation is suitable for concrete machine only, usually for a computing cluster. The default configuration and compilation of the dirac.x executable, which is with dynamical linking, is thus machine dependent. The demand for the grid environment is to have ”all in one”, what means independent and functional static executable. As we already mentioned in Section 1, we decided to compile the DIRAC executable statically to avoid dependencies on dynamic library files. We found, in fact, that creating static executable of a such massive and complex source code as DIRAC is not easy task. Though the DIRAC-for-Grid compilation involves static linking, the resulting executable might not perform correctly everywhere. We may got either crash already at the beginning of the code’s run, or substantial portion of control tests may be hurt due to erroneously made static executable. Despite mentioned difficulties, we mastered the DIRAC’s static buildup with the Intel compiler suite and with the MKL library (version 12.1.3) [18]

684

in some interpreted language. For the sake of interarchitectural portability DIRAC is using scripts written in the Python language [20]. Concerning the need of the Python interpreter, we decided to rely completely on the equipment of remote CE and did not try to prepare own static Python code from its source. Likewise we fully depend on the bash-script interpreter to be at disposal on CEs. The CMake [21] cross-platform, open-source build system is employed in depth for the DIRAC buildup. The set of written CMake macro files is inseparable part of the DIRAC suite and is ensuring proper selection of compilers, their flags, mathematical libraries and of other buildup parameters. Consequently, the CDash [22] system is used for displaying status of DIRAC configurations, compilations and test runs on the dedicated web page [23]. We managed to generate the static cmake, ctest executables and got them used with the static DIRAC suite on some CEs for buildup and tests. As every properly managed software also DIRAC contains set of tests designed to verify many of code’s functionalities. Those are launched with the Python script runtest, which runs selected tests from their directories and is checking obtained output files against stored reference output files. The CMake also ensures that runtest brings outputs also onto the CDash web [23], so that developers can quickly react and made fixes. The runtest script is calling the pam script which handles individual calculations. Another important Python script in DIRAC is setup, which is setting variables and directory for the DIRAC buildup with the CMake system. We have been given access to these three virtual organizations (VO): VOCE [24], COMPCHEM [25–27] and to the recently created one, SIVVP (with the full name sivvp.slovakgrid.sk) [28], in Slovakia. Practically, it is possible to work in three different terminal windows with each having designed variable pointing to the certificate file with the extension for a given virtual organization. In Fig. 1 we present the structure of the DIRAC-forGrid suite, designed specially for grid computations. The static dirac.x executable was prepared in three different ways - as parallel with the Open MPI-IntelMKL suite and as serial with Intel-MKL and GNU compilers. Corresponding sizes of the dirac.x binary were approximately 74 MB, 57 MB and 92 MB, respectively. The full optimization was employed for


GNU-static dirac.x

Intel-MKL static dirac.x

OpenMPIIntel-MKL static dirac.x

Static mpirun executable and related files

Static cmake, ctest executables

DIRAC Python scripts pam, runtest and setup

Reference input and output files

DIRAC source code files for buildup

Fig. 1 Schematic structure of the DIRAC-for-Grid suite

the compilation of the static executables with Intel and GNU compilers, and preserved throughout all the work. The optimization with predefined set of flags for each compiler in CMake source files is the default choice for code’s buildup. The DIRAC-for-Grid contains the Open MPI and CMake/CTest static executables as mentioned above. The whole package was placed on dedicated grid data storage servers (known as storage elements - SEs), each associated with its own virtual organization. The strategy in this work is as follows: (i) prepare the DIRAC-for-Grid suite containing all necessary components as shown in Fig. 1, (ii) launch predefined set of testing calculations, (iii) carry out computationally more demanding production runs on CEs selected from the step (ii). This workflow is schematically shown is Fig. 2. The EMI middleware [29] is used nowadays in the EGI Grid infrastructure for launching of all DIRAC calculations. Upon finishing a grid job we got the logfile of both stderr and stdout streams for analysis, and also we extracted DIRAC output files. The log output file contains important data from utility bash-script called from the jdl-script [30], like timing, number of used processors, size of disk, size of memory, results of tests, and other characteristics of CEs, which are discussed in this paper. The bash interpretative language was used as it is common for all CEs. The fresh short term proxies have been downloaded periodically from the designated “MyProxyServer” thus enabling long running calculations. In our computations we reserved all CPU cores at one node of a CE (this is


At the local server compile dirac.x executables statically, prepare the package DIRAC-for-Grid of Figure 1.

Upload the DIRAC-for-Grid package onto chosen SE of the VO.

Download the DIRAC-for-Grid package onto CE, unpack, query CE and perform own testing (and production) calculations.

After finishing DIRAC-in-Grid computations download and analyze output files.

Fig. 2 Workflow chart of the DIRAC software in the Grid

through the “HostNumber = 1; WholeNodes=True” commands in the corresponding jdl-script).

3 Results and Discussion Using the prepared static DIRAC suite (see Fig. 1) we carried out several testing and afterwards productions calculations in line with the workflow scheme shown in Fig. 2. Each CE has certain hardware (and software) limitations. There are several parameters specifying suitability of a given CE for a particular production calculation. To run DIRAC on a distant CE, the hosting hardware has first to accept the externally compiled dirac.x executable. If code’s execution is correct on the CE, we are half-winners. Then further demands are appearing, like the size of the available memory, the amount of the disk space, the maximum allowed wall clock time and, optionally, the number of CPU cores per processor. The first task in our grid calculations was to find out characteristics of accessible CEs i) by running simple bash-scripts printing out desired characteristics, and, ii) through passed testing calculations. Our test runs were split into three categories: first, those carried out with the serial static code obtained with GNU compilers and internal mathematical library, second, with the serial executable generated with Intel compilers plus the MKL library and,

685

third, with the parallelized static executable produced by the (static) Open MPI Intel compilers with the MKL library. For testing calculations we demanded at least 2 hours of wall time; as to the memory, at least 4 GB was sufficient. These needs were satisfied on almost all CEs. The total memory included the virtual memory what brought additional space, but also possible problems due to swapping. Our findings are summarized in Table 1. We gathered information about several CEs relevant for those who want to perform scientific calculations - number of core CPUs in the processor unit, total memory size, maximum wall clock time, current free disk space. Unfortunately for grid users, the size of the accessible disk space can not be determined by predefined parameter. Rather, it was found out during preliminary test runs directly through simple command df -h. Missing variable specifying the size of the free disk space of CE appears to be a drawback, but can be partly compensated by internal agreements within the VO specifying minimum size of working disk at each CE. We found almost all CEs were provided with sufficient version of Python and of other utility programs, like tar, bash-shell. Therefore we were able to carry out the testing calculations. Concerning the quality of some published grid parameters of CEs, we found discrepancies between real and presented values for Memory, VMemory attributes of some CEs. For that reason we preferred to determine the memory size manually through the Linux command free -m -t performed by the script at the CE. Another issue dealt in the paper is code’s automatic buildup and testing. On top of it, whenever possible, the CE gives results of standard test suite runs and own buildups and put them on the CDash web. This requires not only programs cmake, ctest [21], which we prepared in advance as static binaries and delivered them onto CE as we can not rely completely upon CE’s software, but also the presence of a C++ compiler. Two CDash examples are shown in Fig. 3. These involve i) carrying out separate DIRAC tests with statically prepared executables and ii) the complete DIRAC buildup from the source code on the CEs with appropriate compilers present on given CE. In Fig. 3 we see that the ce.scope.unina.it CE, contrary to the ce4.ui.savba.sk CE, can not get the static OpenMPI-Intel-MKL dirac.x executable working (see also Table 1), what is - by having closer look - evident by almost all tests failed. Though at least one

686


Table 1 Characteristics of selected CEs in three VOs

Memory (GB)

Number of CPUs

Disk size(GB)

Max wall time (hours)

Static dirac.x obtained with GNU Intel OpenMPI MKL Intel-MKL

Own build

CDash report

VO=COMPCHEM ce.scope.unina.it cert-37.pd.infn.it cream01.grid.uoi.gr grid0.fe.infn.it gridce2.pi.infn.it grisuce.scope.unina.it prod-ce-01.pd.infn.it wario.univ-lille1.fr

16 50 12 64 8 16 50 24

8 24 2 24 4 8 24 8

48 392 55 422 54 48 825 114

72 72 72 72 48 72 72 120

yes yes yes yes yes yes yes yes

yes yes yes yes yes yes yes yes

no yes yes yes no no yes yes

yes2 no yes1 no yes1 yes2 no yes1

yes no yes no yes yes no yes

VO=SIVVP ce3.ui.savba.sk ce-sivvp.ui.savba.sk

8 64

4 12

622 404

72 72

yes yes

yes yes

yes no

yes1 no

yes no

VO=VOCE ce.irb.egi.cro-ngi.hr ce2.ui.savba.sk ce4.ui.savba.sk cream02.grid.cyf-kr.edu.pl hephygr.oeaw.ac.at hepx4.uibk.ac.at

20 4 40 32 24 14

8 2 16 8 8 4

221 137 212 70 51 44

168 72 72 unlim 72 72

yes yes yes yes yes yes

yes yes yes yes yes yes

no yes yes yes yes yes

no no yes1 yes1 no no

no no yes yes no yes

CE name

1 2

GNU compilers suite Intel-MKL compilers suite

test failed in other displayed cases in Fig. 3, the static executable can be considered as functional since displayed failed tests can be easily fixed in the used DIRAC development code.

Note that the possibility to display test results or even the whole code buildup on the CDash web is of minor importance as this part is done at the top level of carrying out DIRAC tests. At least, this informs us

Fig. 3 CDash reports of tests and own buildups of two selected CEs. Picture is assembled from print-screens of the DIRAC CDash web [23]. For characteristics of used CEs see Table 1.

In the first column there is the name of the employed working node of the CE cluster


687

about this specific CE’s capability. We reject the idea of complete buildup on the CE including downloading the source code from the repository as this requires presence of user’s private ssh-key at the distant CE.

payload. This way would be suitable for large number of CEs available in the Grid where maintaining long list of individual CEs would be ineffective. Corresponding execution times on three available CEs (belonging to SIVVP and VOCE virtual organization, respectively) are given in Table 2. We observe that the Intel compilers with the MKL library show better code performance than the GNU compilers suite. Note that we report wall times without considering additional time spent for the job deployment. The reason is that while wall times of job execution are almost constant for a given CE (assuming there are identical working nodes within CEs), deployment times vary depending on the current Grid workflow. In Table 3 we demonstrated that static external executable distributed onto grid CEs is fully comparable in performance with the locally compiled one. This is important finding confirming the reliability and efficiency of the externally prepared static executable. Also, by combining local cluster computing with grid computing one can benefit from more computing power. Nowadays, modern clusters have portion of their computing resources dedicated for grid usage. Unfortunately, with CEs at our disposal we were not able to perform parallel OpenMPI-Intel-MKL calculations of such computationally demanding chemical system. We had to resort to serial Intel compilers with the internally threaded MKL library. The bottleneck shown to be the size of working disk, and also the available memory. This is because computational chemistry methods like the IHFSCC one need transformed integrals over atomic orbitals, and this step is very disk and memory consuming. Note that values of measured quantities - execution wall times - can not be considered as exactly reproducible. Computers connected in grid are evolving in time and the user may not always be informed about all software and hardware changes of a specific CE.

3.0.1 Tl2 Molecule As an example from the praxis of relativistic quantum chemical calculations with DIRAC we have chosen the heavy Tl2 molecule in the internuclear distance ˚ We employed the uncontracted basis set of 3 A. [24s22p16d10f2g] by K.Fægri Jr. for constructing molecular spinors in the self-consistent field procedure with the two-component infinite-order Hamiltonian [31]. Correlation effects were treated with the Intermediate Hamiltonian Fock space Coupled Cluster (IHFSCC) method [32]. A reasonable space, restricted between −15 a.u. an 100 a.u., was used for transformation of molecular spinors. With the IHFSCC method, we did calculations, starting from the Tl2+ 2 closed shell system, involving (0,0), (0,1) and (0,2) sectors. Converged IHFSCC calculations ended up with ground and plethora of excited energy states of the Tl2 molecule. Within available VOs we were interested in gaining maximum computing performance for our chemical systems. The simple rule of thumb in quantum chemical calculations is the bigger system is treated the more memory, disk space and the CPU time it consumes. First, in the corresponding jdl-script for the Tl2 molecule we specified demands on CEs hardware: size of the memory, number of free CPUs, maximum wall-clock time. As turned out later, it was simpler for us to select directly known CEs, which have enough disk space and memory, from previous calculations in Table 1. However, detecting the free disk space directly in the test jobs can not be used as a perfect criteria for the choice of a CE as this is volatile parameter. For instance, worker free disk space varies depending on the current workload and on a disk cleaning status. Concerning the memory, since we demanded whole worker nodes, our jobs should get the full amount the worker node has at disposal. Presented approach for selecting suitable CEs can be further improved. More advantageous than keeping static lists of suitable CEs (as is in Table 1) is considering pilot job based workload scheduling systems, allowing evaluation of the execution environment before executing own user

Table 2 Execution performance (wall time) of the molecule calculations CE name (VO)

Tl2

Intel-MKL-static GNU-static

ce-sivvp.ui.savba.sk (SIVVP) 9h20m ce.irb.egi.cro-ngi.hr (VOCE) 15h26m ce4.ui.savba.sk (VOCE) 9h56m

1d5h39m 2d9h2m 1d1h53m

The characteristics of used CEs are given in the previous Table 1

688


Table 3 Execution performance (wall time) of the Tl2 molecule calculations obtained at the ce-sivvp.ui.savba.sk cluster, belonging to SIVVP VO dirac.x

Serial compiler

Executable

Intel-MKL

GNU

local, dynamic local, static external, static1

9h18m 9h11m 9h25m

1d4h46m 1d4h38m 1d5h5m

1

Performance of this grid-distributed executable on the same CE is given in Table 2. For characteristics of the CEs see Table 1

Not mentioning the DIRAC software which is evolving as well. Nevertheless, presented wall times serve as a clue for the choice of the optimal CE for specific computational tasks. Also, we had to repeat some of our calculations several times in order to pass through “windows of opportunity” when most of CEs in VOs were working correctly. Quantum chemists (or, in general, scientific grid users) are interested in obtaining maximum computing performance. How to select most suitable CEs for computational tasks within given VO? According to the findings here, first, list all accessible CEs and try running DIRAC (or other software) test suite at them, making notes similar to Table 1. For subsequent and more demanding production calculations choose most powerful CEs, according to selected criteria, from those CEs already verified with test runs as we did in Table 2. Another strategy for selecting CEs, as mentioned above, would be employing pilot jobs based workload scheduling systems.

4 Conclusions and Perspectives We prepared the static DIRAC package for computations in the grid environment on Linux x86 64 servers. The DIRAC-for-Grid distribution does not rely upon installed compilers and math libraries on CEs. This strategy was proven suitable as it does not put any extra burden on CEs administrators. In opposite, distribution and maintenance of pre-compiled software over various CEs in the Grid consumes qualified man power. We advocate this simple approach also in order to exploit maximum capacity offered by the grid computing network and we believe that

we have achieved the best effectiveness for our calculations. As to the Dirac-for-Grid package preparation, we succeeded to produce working static executables on one local server. If other Linux distributions are suitable for that, and on what parameters does it exactly depend, will be investigated and published elsewhere [16]. DIRAC users shall be provided with web tutorial for the grid computing. In the paper we have been limited by three virtual organizations and also by two sets of compilers - GNU and Intel. Not mentioning other VOs, there are more commercial compilers available, like Portland Group (PGI), Lahey Fortran and others, which are good candidates to try. We found that the best compiler available to the public under non-commercial licence is the Intel suite. For time consuming jobs the full Open MPI parallelization would be suitable. However, we did not succeed here in utilizing Open MPI threaded DIRAC for our production example. We were restricted in wider usage of the Open MPI static version of DIRAC due to unknown bug occurring on some powerful, multi-CPU CEs with large disk space at disposal. Though this problem is not related to the grid environment as such, it is part of our effort to adapt properly the static executable for parallel OpenMPI calculations in the Grid. We feel that this issue has to be addressed here as our aim is to bring platform independent, parallelized static binary code. Nevertheless, the Intel serial code with the internally parallelized MKL library proved to be good option instead. In this work we likewise stress importance of launching test runs with the distributed static executables of the CE of choice. Especially before carrying out more time-consuming calculations one should verify correctness of code’s functionalities through quick tests. As to the choice of a suitable CE in the framework of the virtual organization, it can be either user’s direct selection of CE or an automated middleware’s assignment. We also report that some of CEs have some of their grid attributes assigned incorrectly. For that reason we recommend to obtain certain CEs parameters directly from CE by simple bash-scripts before running any production calculations. Another unfortunate feature was occasional inaccessibility of several CEs due to various errors in the grid network, requiring intervention of CEs administrators.


689

All efforts presented here have practical consequences for further production calculations with the DIRAC code. Note, that the majority of quantum chemists prefer to execute their calculations on clusters (cluster computing) with locally compiled codes. But with massive amount of calculations waiting in batch queues users may choose the grid computing as a feasible alternative. Keeping this in mind, our future plans with DIRAC for the grid infrastructure involve i) studying further possibilities of preparing statically linked and multi-threaded executable (by combining both Open MPI and MKL parallelization) and, ii) carrying out hardware demanding production calculations of properties of various chemical systems with heavy elements. Additional useful aspect is for the distribution of DIRAC for users: scientists can choose to receive static executables to avoid sometimes lengthy compilation from source files.

Bhatia, K., Taufer, M.: The computational chemistry prototyping environment. Proc. IEEE 93(3), 510 (2005). doi:10.1109/JPROC.2004.842747 Sudholt, W., Altintas, I., Baldridge, K.: In: Alexandrov, V.N., van Albada, G.D., Sloot, P.M.A., Dongarra, J.J. (eds.) Computational Science ICCS 2006, vol. 3993, pp. 69–76. Springer, Berlin Heidelberg (2006) Costantini, A., Gutierrez, E., Lopez Cacheiro, J., Rodriguez, A., Gervasi, O., Lagana, A.: Int. J. Web Grid Serv. 6(2), 141 (2010). doi:10.1504/IJWGS.2010.033789 Domingo, D., East, J. (eds.): Red Hat Enterprise Linux 6 Developer Guide - An Introduction to Application Development Tools in Red Hat Enterprise Linux 6. Red Hat, Inc. and others. Electronic book (2010) DIRAC: A relativistic ab initio electronic structure program, Release DIRAC12. written by Jensen, H.J.Aa., Bast, R. ,Saue, T. and Visscher, L. with contributions from Bakken, V., Dyall, K.G. , Dubillard, S., Ekström, U., Eliav, E., Enevoldsen, T.,Fleig, T.,Fossgaard, O., Gomes, A.S.P., Helgaker, T., Lærdahl, J.K., Lee, Y.S., Henriksson, J., Iliaˇs, M., Jacob, Ch.R., Knecht, S., Komorovský, S., Kullie, O., Larsen, C.V., Nataraj, H.S., Norman, P., Olejniczak, G., Olsen, J., Park, Y.C., Pedersen, J.K., Pernpointner, M., Ruud, K., Sałek, P., Schimmelpfennig, B., Sikkema, J., Thorvaldsen, A.J., Thyssen, J., van Stralen, J., Villaume, S., Visser, O., Winther, T., Yamamoto, S. (see http://www. diracprogram.org) (2012) Dyall, K.G., Faegri, Jr., K.: Introduction to relativistic quantum chemistry. Oxford University Press, London (2007) Reiher, M., Wolf, A.: Relativistic quantum chemistry: The fundamental theory of molecular science. Wiley, New York (2009). http://www.google.sk/books?id=YwSpxCfsNsEC Iliaˇs, M., Kellö, V., Urban, M.: Relativistic effects in atomic and molecular properties. Acta Phys. Slovaca Rev. Tutorials 60(3), 259 (2010). doi:10.2478/v10155-0100003-1 Autschbach, J.: Perspective: relativistic effects. J. Chem. Phys. 136(15), 150902 (2012). http://scitation.aip.org/ content/aip/journal/jcp/136/15/10.1063/1.3702628 DIRAC- web manual (search for static linking). www. diracprogram.org Gomes, A., Merzky, A., Visscher, L.: In: Alexandrov, V., van Albada, G., Sloot, P., Dongarra, J. (eds.) Computational Science ICCS, Lecture Notes in Computer Science, vol. 3993, pp. 97–104. Springer, Berlin Heidelberg (2006) Intel Compilers (Intel Developer Zone). https://software. intel.com/en-us/intel-compilers Open MPI: open source high performance computing. http://www.open-mpi.org/ Python programming language official website. http:// www.python.org/ CMake - cross platform make. http://www.cmake.org/ CDash - an open source, web-based testing server. http:// www.cdash.org/ DIRAC - dashboard. https://repo.theochem.kth.se/cdash/ index.php?project=DIRAC EGEE - VOCE. http://egee.cesnet.cz/en/voce/ COMPCHEM VO. https://www3.compchem.unipg.it/ compchem/

Acknowledgments Authors thank Prof. Trond Saue and Dr. Radovan Bast, who are main authors of DIRAC, for valuable discussions. This work was done under Slovak-French Research and Development Cooperation (Grant No. SK-FR-00180-11). M.I. is grateful for the financial support from the Slovak Research and Development Agency (Grant No. APVV-005910) and from the Agency of the Ministry of Education, Science, Research and Sport of the Slovak Republic for the Structural Funds of EU (ITMS 26110230082). M.D. thanks the Slovak Grant Agency (VEGA 2/0054/12) for support. The authors also acknowledge the use of resources provided by the European Grid Infrastructure. For more information, please reference the EGI-InSPIRE paper (http://go.egi.eu/pdnon).

References 1. Magoules, F., Pan, J., Tan, K.A., Kumar, A.: Introduction to Grid Computing. Taylor & Francis, New York (2009) 2. Wang, L., Jie, W., Chen, J.: Grid Computing: Infrastructure, Service, and Applications. Taylor & Francis, New York (2010) 3. Wilkinson, B.: Grid Computing: Techniques and Applications. CRC Press, Boca Raton (2011) 4. Preve, N.P.: Grid Computing. Springer, Berlin Heidelberg New York (2011) 5. Coveney, P.V.: Scientific grid computing. Philos. Trans. R. Soc. A Math. Phys. Eng. Sci. 363(1833), 1707 (2005). doi:10.1098/rsta.2005.1632 6. Nishikawa, T., Nagashima, U., Sekiguchi, S.: Computational science ICCS 2003. In: Sloot, P.M.A., Abramson, D., Bogdanov, A.V., Gorbachev, Y.E., Dongarra, J.J., Zomaya, A.Y. (eds.) Lecture Notes in Computer Science no. 2659, pp. 244–253. Springer, Berlin Heidelberg 7. Baldridge, K.K., Greenberg, J.P., Sudholt, W., Mock, S., Altintas, I., Amoreira, C., Potier, Y., Birnbaum, A.,

8.

9.

10.

11.

12. 13.

14.

15.

16. 17.

18. 19. 20. 21. 22. 23. 24. 25.

690 26. Laganà, A., Riganelli, A., Gervasi, O.: Computational science and its applications - ICCSA. In: Gavrilova, M., Gervasi, O., Kumar, V., Tan, C., Taniar, D., Lagan, A., Mun, Y., Choo, H. (eds.) Lecture Notes in Computer Science, vol. 3980, pp. 665–674. Springer, Berlin Heidelberg (2006). doi:10.1007/11751540 70 27. Laganà, A., Costantini, A., Gervasi, O., Lago, N., Manuali, C., Rampino, S.: COMPCHEM: progress towards GEMS a Grid Empowered Molecular Simulator and beyond. J. Grid Comput. 8(4), 571 (2010). doi:10.1007/s10723-010-9164-x 28. SIVVP. http://www.sivvp.sk/home/

M. Iliaˇs, M. Dobrucký 29. EMI - European Middleware Initiative. http://www.eu-emi. eu/ 30. JDL - Job description language. http://diracgrid.org/files/ docs/UserGuide/GettingStarted/UserJobs/JDLReference/ 31. Iliaˇs, M., Saue, T.: An infinite-order two-component relativistic Hamiltonian by a simple one-step transformation. J. Chem. Phys. 126(6), 064102 (2007). doi:10.1063/1.2436882 32. Visscher, L., Eliav, E., Kaldor, U.: Formulation and implementation of the relativistic Fock-space coupled cluster method for molecules. J. Chem. Phys. 115(21), 9720 (2001). doi:10.1063/1.1415746