Application of Parallel Ensemble Monte Carlo ...

Sains Malaysiana 37(1)(2008): 89-93

Application of Parallel Ensemble Monte Carlo Technique in Charge Dynamics Simulation (Aplikasi Teknik Monte Carlo Ensembel Selari pada Simulasi Dinamik Cas) A. P. OTHMAN, R. UMAR

& G. GOPIR

ABSTRAK

Pada masa yang lepas, simulasi dinamik cas bagi peranti pepejal seperti mobiliti arus dan halaju hanyut arus fana dijalankan pada sistem kerangka utama atau pada sistem komputer berprestasi tinggi. Perkara ini berlaku kerana simulasi seperti ini sangatlah membebankan jika dijalankan pada komputer peribadi (PC) yang berasaskan pemprosesan tunggal. Bagi tujuan simulasi dinamik cas, himpunan zarah yang banyak, melebihi 40,000 zarah biasanya diperlukan bagi menghasilkan keputusan yang memuaskan. Jika simulasi yang melibatkan zarah sebanyak ini dijalankan pada PC berpemprosesan tunggal dengan kaedah Monte Carlo ensemble atau Monte Carlo zarah tunggal yang lazim, masa komputasi yang diperlukan adalah terlalu lama, walaupun jika kita menggunakan PC dengan pemprosesan 2.0 MHz. Akhir-akhir ini, kaedah simulasi yang lebih cekap, mudah dibangunkan dan menjimatkan telah wujud, iaitu dengan penggunaan rangkaian PC yang dijalankan dalam aplikasi sejajar. Kaedah ini dilakukan dengan pembangunan jaringan kluster dalam model pelayan-klien. Makalah ini melaporkan pembangunan satu rangkaian kluster LINUX bagi mensimulasikan pemodelan peranti keadaan pepejal dengan kaedah Monte Carlo Ensembel secara sejajar. Penggunaan piawai Parallel Virtual Machine (PVM) telah dicadangkan bagi menjalankan algorithma yang dibangunkan. Hasil simulasi yang jalankan pada kluster LINUX yang telah dibangunkan juga disertakan. Kata kunci: Monte Carlo; dinamik cas; algorithm selari ; kluster LINUX ABSTRACT

In the past, simulating charge dynamics in solid state devices, such as current mobility, transient current drift velocities are done on mainframe systems or on high performance computing facilities. This is due to the fact that, such simulations are costly in terms of computational requirements when implemented on a single processor-based personal computers (PCs). When simulating charge dynamics, large ensembles of particles are usually preferred, such as exceeding 40,000 particles, to ensure a numerically sound result. When implementing this type of simulation on a single processor PCs using the conventional ensemble or single particle Monte Carlo method, the computational time is very long even on the fast 2.0 MHz PCs. Lately, a more efficient, easily made available tools and cost effective solution to this problem is the application of an array of PCs employed in a parallel application. This is done using a computer cluster network in a master-slave model. This paper report the development of a LINUX cluster for the purpose of implementing parallel ensemble Monte Carlo modelling for solid states device. The use of Parallel Virtual Machine (PVM) standards when running the parallel algorithm of the ensemble MC simulation been proposed. Some results of the development are also presented in this paper. Keywords: Monte Carlo; charge dynamics; parallel algorithm; LINUX cluster INTRODUCTION The Ensemble Monte Carlo (EMC) (Jacoboni & Reggiani 1983), is an old method used in device simulation, especially to simulate semi-classical charge transport in bulk semiconductor materials and devices where the device geometry is typically larger than a de Broglie wavelength (> 0.1 µm). However after over thirty years, the EMC method is coming back and once again becomes popular because of the availability of inexpensive and powerful computing machines. The difficulty with the method has been its large computational burden especially when incorporating the full band structure model of the semiconductor system (Fischetti et al. 1988; Karl 1991).

Briefly, The EMC method is usually referred to as a stochastic technique, which uses random number generation. In the case of simulations involving particle transport phenomena in solids, ECM method is used to solve the usual Boltzmann transport equation !F (k , r , t ) !t

e r + v" r F (k , r , t ) # " k F (k , r , t ) h

$ !F (k , r , t ) ' =& ) . % (scat !t

90

directly using a statistical numerical approach, without making assumptions on the electron distribution function F(k, r, t) which is a function of wave vector k, space r and time t. As it was, EMC is a powerful method in solving problems such as modelling high-field transport and ultra fast devices. In such devices, operation in the high field regime is always desirable so as to achieve the desirable high-speed or high-frequency performance. In such device modelling, many assumptions usually employed in the conventional techniques (Boltzman transport equation or the reduced forms of this, drift-diffusion and balance equation) are no longer valid. The central idea of EMC is the Monte Carlo Simulation itself. The terminology ensemble refers to the successive and simultaneous calculation of many particles. It is an approach to represent the solution of a mathematical problem by a parameter of a true or hypothetical distribution function and to estimate the value of this parameter by taking samples from this distribution. This idea provides a direct tie to the ensemble concept found in statistical mechanics by Gibbs and the probabilistic interpretation of wave functions in quantum mechanics (Ferrenberg et al. 1992; Vattulainen et al. 1995; Rubinstein 1981). The availability of a wide variety of computational platforms, from vectors, multitasking and parallel processing, to RISC workstations and personal computers have made the choice of implementing EMC method on the PCs a formidable task. This difficulty is coupled with the evolution of programming languages. However in this

paper, we are not going to indulge in any specific programming language even though in the forth-coming discussion the author is referring to the use of C/C++ programming language. Briefly, the algorithm of EMC applied on a LINUX cluster as shown in Figure 1. ENSEMBLE MONTE CARLO SIMULATION

Since the distribution function F(k, r, t) is not used in the Ensemble Monte Carlo simulation, the carrier transport in semiconductor is analysed based on following time evolution of an ensemble of particles through the material in both real and momentum space. It is done with the successive and simultaneous calculation of the motions of many particles during a small time increment *t - this is where the word ensemble comes in. The key word of calculation of the motions here simply means the calculation of mean carrier free flights and the calculation of scattering events of each particle in the ensemble. The motion of each particle in the ensemble has to be simulated in turn, for the full duration of the simulation. This method is essentially dynamics and is therefore suitable for the analysis of transient carrier motion. The typical MC algorithm, as applied to semiconductor simulation is well established and has been dealt with thoroughly in a number of literatures (Jacoboni & Lugli 1989; Kurosawa 1966). The method is best visualised in the diagram shown in Figure 2. Each horizontal line shows the trace of events of each particle. The tick ‘x’ on each line indicates the scattering event at various time. The vertical line shows sampling time, such as when the poisson solver equation is solved.

4 Units user control 1. Hardware schematic for multiple computers (eight units of slaves) in LINUX cluster. The communication is mediated via a fast Ethernet 10/100 mbps switch and the interaction is mediated via keyboard, video and mouse (KVM) switch

FIGURE

91

FIGURE

2. The flowchart of ensemble Monte Carlo simulation

A BRIEF REVIEW OF THE PARALLEL VIRTUAL MACHINE (PVM) PLATFORM OF THE CLUSTER

The PVM system was originally developed at the Oak Ridge National Laboratory. The current version of PVM are being jointly developed and maintained by a number of research teams (Bookman 2003). Under the PVM parallel processing model, heterogeneous computer resources are made available as one large distributed memory computer as networked as in Figure 1. The overall computational problem is divided into a number of co-functional, possibly identical processors. Each node of virtual machine executes one or more of these processes. The operating process communicates via a massage passing protocol defined by the PVM software ( the alternative one is MPI). A message passing server is installed on each node and handles all communications among the nodes of the virtual machines. The client code makes use of PVM Application Program Interface (API) calls in order to send and receive massages from other processors running on the virtual machine (Bookman 2003). The implementation of the virtual machine used for the simulations discussed in this paper utilizes 9 Pentium IV (2.0GHz) machines. The node machines are linked via a standard Ethernet switch (as shown in Figure 1). The random nature of the network parameters, due to the nature of Ethernet, defines the nature of the virtual machine obtained. As implemented, the virtual machine is most suited to coarse grained parallel processing, where variations in communication performance have minimal impact on the total system performance.

via command ping. When the networking was running, configure secure shell hosts (ssh) of the system. Set up network file systems NFS and NIS for the communication between the slaves and the master. Remote login hosts (rlh) was also configured for the master to login into slaves. Then, finally it was the installation of the PVM software on each of the nodes. PVM- HOW TO SETUP AND COMPILE THE APPLICATIONS

Step-1: To set up up the working environment for PVM. a.

Firstly, a valid account is required on every computer(host) that is available for PVM use. The main important thing to remember is to do all activities via the master.

b.

After succeeding in creating an account, setup the working environment into the account by checking the presence of two files .bashrc and bashrc.stub. Edit the .bashrc file by copying the bashrc.stub file into the .bashrc file.

Step-2: To create working directory After Logging into the LINUX Shell via the master PC a.

Create the pvm3 directory inside a home directory (such as puaad) [puaad@masterpuaad]$mkdir pvm3 [puaad@masterpuaad pvm3]$cd pvm3

b.

Create the bin directory inside the home /puaad/ pvm3/ directory [puaad@masterpuaad]$mkdir bin [puaad@masterpuaad bin]$cd bin

c.

Create the LINUX directory inside /puaad/pvm3/ bin/ directory [puaad@masterpuaad]$mkdir LINUX [puaad@masterpuaad]$cd LINUX [puaad@masterpuaad LINUX]$

SETTING UP THE CLUSTER TO IMPLEMENT PARALLEL ENSEMBLE MONTE CARLO

Outlined are some steps to setup the cluster. The main step is to configure the IP address of each node. Choose either a PC or a node to be a master and the rest will be slaves. Then configure the IP configuration. The Netmask chosen for all nodes was 255.255.255.0. The default gateway (IP) was the IP for the master. After installation and restarting the machines, check the functionality of the networking

92

d.

Go back to /puaad/pvm3/ directory [puaad@masterpuaad LINUX]$cd [puaad@masterpuaad pvm3]$

c.

Copying the Makefile.aimk program from /usr/share/pvm3/other/makefile.aimk. The PVM package includes the aimk shell script to help you to compile PVM applications. [puaad@masterpuaad pvm3]$cp/usr/ share/pvm3/ other/Makefile.aimk .

d.

Open and modify the ’Makefile.aimk’ file [puaad@masterpuaad pvm3]$vi Makefile.aimk

RUNNING AND TESTING PVM INSTALLATION FROM CONSOLE

After logging in into the LINUX Shell via the master PC, type ‘pvm’ from the console. The following command prompt of PVM will be shown: pvm>

type this command to compile hello.c and hello_other.c [puaad@masterpuaad pvm3]$aimk hello.c hello_other.c

This means that the pvm daemon process is running. ADDING AND REMOVING HOSTS

To add host computer manually (assume we name our nodes with fizlan01, fizlan02, fizlan03….., ) type the following:

e.

After we have modify the ‘aimk’ file, we have to save and quit the file by typing ‘:wq!’.

pvm> add fizlan01

f.

We then go to the bin/LINUX directory to compile and run the program. [puaad@masterpuaad pvm3]$cd bin/LINUX

g.

Show the listing of the files in the directory. puaad@masterpuaad LINUX]$ls –al the output would be: hello.c hello_other.c

h.

Now we have to log in into the pvm to add the required hosts. You can choose the number of processors to run this program by adding as many hosts as we wish.

1 succesfully 1 host, 1data format HOST DTID ARCH fizlan01 4000 LINUX

SPEED 1000

or remove the host computer manually pvm> del fizlan01 COMPILING AND RUNNING A TEST HELLO.C AND HELLO_OTHER.C PROGRAM

Hello.c and hello_other.c are programs that are included when you install the PVM. Hello is a program for the master and hello_other.c is a program for the slave. These two programs are the simplest introductory program in PVM, which are analogous to our an EMC program for the master and another EMC program for the slaves. Hello.c and hello_other.c are used to test the functionality of our cluster. To compile the source, we have to write or modify a ‘Makefile.aimk’ and usr ‘aimk’ files by:

[puaad@masterpuaad LINUX]$pvm pvm> add fizlan01 1 successfully 1 host, 1data format HOST DTID ARCH SPEED fizlan01 4000 LINUX 1000

Copying the hello.c and hello_other program from /usr/share/pvm3/examples/ to the / puaad/pvm3/ [puaad@masterpuaad pvm3]$cp/usr/ share/pvm3/ examples/hello.c

SPEED 1000

a.

b.

Copying the hello_other.c program from /usr/share/pvm3/examples [puaad@masterpuaad pvm3]$cp/usr/ share/pvm3/ examples/hello_other.c

pvm>add fizlan02 1 successfully 1 host, 1data format HOST DTID ARCH fizlan01 4000 LINUX

type ‘conf’ to see the number of machines in the pvm pvm>conf HOST master fizlan01 fizlan 02 i.

DTID 4000 4000 4000

ARCH LINUX LINUX LINUX

We then have to quit the pvm pvm>halt pvm still running

SPEED 1000 1000 000

93

(a) FIGURE 3.

j.

(b)

Typical results from the experiment. (a) The curve for the dependence of execution time versus number of processors and (b) the transient charge velocities of GaAs versus time for four different ensembles of particles

When we want to run the program. Type ‘hello’ command inside the directory /pvm/bin/LINUX [puaad@masterpuaad LINUX]$./hello the output is: i’m t80003 from tc0001: hello,world from fizlan02.cc.ukm.my

With the above example of hello.c and hello_other program, we can easily implement the EMC program with the following algorithm on master and slaves (Figure 3b). A standard ‘pi program’ was performed to benchmark the processor’s performance and the result is shown in Figure 3a. CONCLUSION A parallel Ensemble Monte Carlo simulation on a LINUX cluster of eight slaves is proposed and have successfully been implemented, which will prove to be very useful and cheap apparatus for solid state device simulation in novel structures and measurement techniques. Successful implementation of this proposed master-slave network of eight personal computers will pave ways for more masterslave clusters and place Monte Carlo particle (especially ensemble Monte Carlo) simulation well within reach of most research groups, and universities, as most of us do not readily have access to massive multi processors machines and expensive crunching machines. The algorithms presented can further be improved and will certainly generate interest for the popularisation of the MC method. ACKNOWLEDGMENT

The author would like to acknowledge The Ministry of Science, Technology and Innovation and Universiti

Kebangsaan Malaysia for supporting this work under the IRPA grant 09-02-02-0054-EA150. REFERENCES

Bookman, C. 2003. LINUX clustering, building and maintaing LINUX clusters. NewYork. Ferrenberg, A.M., Landau, D.P. & Wong Y.J. 1992. Monte Carlo simulations: Hidden errors from ‘good’ random number generators. Phys. Rev. Lett. 69: 3382. Fischetti, M.V. & Laux, S.E. 1988. Monte Carlo analysis of Electron transport in small semiconductor devices including band structure and space-charge effects. Phys. Rev. B 38(19): 9721-9727. Jacoboni, C. & Reggian, L. 1983. The Monte carlo method for solution of charge transport in semiconductors with application to covalent materials. Rev. Mod. Phys. 55(3): 645. Jacoboni, C. and Lugli, P. 1989. The Monte carlo method for Semiconductor Device Simulation. New York: SpringerVerlag. Karl Hess. 1991. Monte Carlo device simulation: Full band and beyond. Dordrecht, The Netherlands: Kluwer Academic Publishers Group. Kurosawa, T. 1966. Monte Carlo simulation of hot electron problems. J. Phys. Soc. Suppl 21: 527-529. Rubinstein, R.V. 1981. Simulation and the Monte Carlo method. New York: J Wiley & Sons, Vattulainen, I. & Ala-Nissila, T. 1995 Mission impossible: Find a random pseudorandom number generator. Comput. in Phys. 9:500. School of Applied Physics Universiti Kebangsaan Malaysia 43600 Bangi, Selangor D.E. Malaysia Received: February 2006 Accepted: 12 April 2007