A Generic Parallel Pattern-based System for Bioinformatics Weiguo Liu and Bertil Schmidt School of Computer Engineering, Nanyang Technological University, Singapore 639798
[email protected],
[email protected] Abstract. Parallel program design patterns provide users a new way to get parallel programs without much effort. However, it is always a serious limitation for most existing parallel pattern-based systems that there is no generic description for the structure and behavior of a pattern at application level. This limitation has so far greatly hindered the practical use of these systems. In this paper, we present a new parallel pattern-based system for bioinformatics. The underlying programming techniques are based on generic programming, a programming technique suited for the generic representation of abstract concepts. This allows the new system to be built in a generic way at application level. We show how this system efficiently addresses the shortcomings of existing systems and leads to significant runtime savings for some popular applications in bioinformatics on PC clusters.
1 Introduction Parallel design patterns are based on sequential program design patterns used in object-oriented languages [1]. A parallel program design pattern names, abstracts, and identifies the key aspects of a common parallel structure that make it useful for creating a reusable and efficient parallel program. Parallel design patterns are similar to parallel program libraries. However, there is an important difference between them. When a library is used, a programmer must define the structure of the application and make calls to the library code. Conversely, a parallel design pattern defines the structure of a framework and the programmer supplies code for specific applicationdependent routines. In the past decade, many parallel pattern-based systems have been developed to employ design patterns related concepts in the parallel computing domain in the context of object-oriented programming techniques. Some of the systems based on similar ideas include Code [2], Frameworks [15], Enterprise [16], HeNCE [4], Tracs [3], and DPnDP [14]. However, most of these systems lack practical usability for the following reasons [1, 9]: • Although more and more parallel patterns are supported by most of the systems, the parallel patterns are not expressed in a compact generic manner. This is also the root of other shortcomings. • The way to implement a parallel application for a new algorithm is very complex for the user.
• These systems are not flexible enough for the user to reuse the components of existing parallel patterns at application level. • Most of the systems are overstaffed and the code generated by them is inefficient and suffers from great loss of performance. Generic programming is a program design technique that deals with finding abstract representations of algorithms, data structures, and other software concepts [18]. Generic programming can use both the traditional objected-oriented techniques (such as inheritance and virtual functions) and templates. This makes generic programming a practical and flexible technique in program design. In practice, the ideas of generic programming have been successfully applied to the field of high performance computing. The STL (the Standard Template Library, which later was adapted and incorporated into the C++ standard library) and Janus [6] are two examples of generic programming applications. In this paper, we present a new system using generic programming techniques. We demonstrate how the new system can efficiently address most of the shortcomings of existing systems and generate parallel programs with substantial performance gains for some dynamic programming applications in bioinformatics. The rest of the paper is organized as follows: Section 2 describes the characters and classification for dynamic programming algorithms which have wide applications in bioinformatics. In Section 3, we present implementation details for our system. Section 4 gives its application and performance evaluation for some popular algorithms in bioinformatics. Section 5 provides conclusions.
2 Dynamic Programming Problems in Bioinformatics Dynamic programming is an important algorithm design technique in bioinformatics. Typical applications using this technique are highly compute intensive and suffer from long runtimes on sequential architectures. Dynamic programming algorithms can be classified according to the matrix dimension and the dependency relationship of each cell on the matrix [8]: a dynamic programming algorithm is called a tD/eD algorithm if its matrix dimension is t and each matrix cell depends on O(ne) other cells. For example, three dynamic programming algorithms are defined as follows: Algorithm 1 (2D/0D): Given D[i, 0] and D[0, j] for 0 ≤ i, j ≤ n, compute D[i, j] = min{D[i-1, j] + xi, D[i, j-1] + yj, D[i-1, j-1] + zi,j} for 1 ≤ i, j ≤ n, where xi, yj and zi,j are computed in constant time. Algorithm 2 (2D/1D): Given w(i, j) for 1 ≤ i
dealing w ith th e in itializatio n of th e m atrix
e x te
nd
extend e x te
nd
p o licy 1
p o licy 2
p olicy3
< d atatyp e A lg o rith m In i1 S equ en tialC o m p1 P arallelC o m m u 1 >
< d ataty p e A lg o rith m Ini2 S eq u en tialC o m p 2 P arallelC o m m u 2 >
< d ataty p e A lgo rithm In i3 S eq u en tialC o m p 3 P arallelC om m u 1>
instantiate
instantiate
instantiate
p arallel p ro gram 1
p arallel prog ram 2
p arallel p rog ram 3
Fig. 2. Extending the generic pattern to implement application-dependent parallel programs
Because the algorithms in bioinformatics share similar characters, this procedure is not complex. The user only needs to provide the sequential application specific code while the system supplies the necessary parallel code. For example, the SmithWaterman algorithm with linear and affine gap penalty and the syntenic alignment algorithm share analogous characters in all the three aspects. The obvious differences between them are the number of matrices they will compute and the dependency relationship of each cell. Hence, extending the generic pattern to parallelize these three algorithms is quite simple and similar. From Fig. 2 we can also find that each template parameter is defined independent of other parameters. Yet different template parameters can interact with each other via standard interfaces. Consequently, the system has a good flexibility. For instance in Fig. 2, policy3 and policy1 share the same parallel characters. Thus we can entirely reuse the overall design of ParallelCommu1 to develop policy3. The user can
5
therefore reuse the components in existing patterns to develop new applications in a flexible way.
4 Experiments and Performance Evaluation We have used the described system to develop parallel applications for some of the algorithms presented in Table 1. They are running on a distributed memory cluster which comprises 8 AlphaServer ES45 nodes. Each node contains 4 Alpha-EV68 1GHz processors with 1GB RAM for each processor. All the nodes are connected with each other by a Gbit/sec Quadrics switch. Table 2 shows the speedups for different number of processors. It is important to note that these applications are implemented in different methods. The linear space method [13] is used to reduce the RAM needed by the Smith-Waterman algorithm (with linear gap penalty and affine gap penalty) and the Syntenic alignment algorithm for long sequences. Similar space-saving method is used for the three dimensional applications such as the multiple alignment algorithm for three sequences and the spliced alignment algorithm. As to the Nussinov algorithm, we store and compute the whole matrix. Notice the super linear speedups are observed in several applications. This is because of the effects due to better caching are different according to different implementation methods for specific applications. Table 2. Speedups for some well-known parallel applications in bioinformatics
Number of Processors Smith-Waterman algorithm with linear gap penalty (90000×90000) Smith-Waterman algorithm with affine gap penalty (70000×70000) Syntecnic alignment algorithm (60000×60000) Nussinov algorithm (5000×5000) Multiple alignment algorithm for three sequences (3000×3000×3000) Spliced alignment algorithm (4000×4000×1000)
1
2
4
8
16
32
1
1.87
3.7
7.37
14.74
28.25
1
1.85
3.68
7.31
14.13
27.65
1
2
4.25
8.31
16.84
33.68
1
2.04
4.07
8.03
15.96
29.87
1
2
4.04
8.72
17.7
33.37
1
2.12
4.55
9.19
18.2
36
Since the system has been implemented using generic programming techniques, it is interesting to compare its performance with a system using object-oriented programming techniques. In order to investigate this, we have implemented the system in both ways, with generic programming techniques and using inheritance and virtual functions. Table 3 presents the performance comparison for the sequential applications between these two methods.
6
Table 3. Performance comparison between the system using generic programming and virtual functions (system1: using virtual functions; system2: using generic programming techniques)
Sequential Applications Nussinov (5000×5000) Smith-Waterman with linear gap penalty (100000×100000) Smith-Waterman with affine gap penalty (100000×100000) Syntenic alignment algorithm(100000×100000) Multiple alignment for three sequences (3000×3000×3000) Spliced alignment (4000×4000×1000)
Runtimes of system1(sec) 821
Runtimes of system2(sec) 795
Runtimes reduction 3.2%
551
503
8.7%
917
868
5.3%
1543
1319
14.5%
643
624
3%
730
642
12.1%
From Table 3 we can see that the code generated by the system using generic programming techniques is faster. This is because the generic programming relies on static polymorphism, which resolves interfaces at compile time. On the other hand in dynamic polymorphism using inheritance and virtual functions, the determination of which virtual function to use cannot be made at compile time and must instead be made during run time. Thus, more overhead is associated with the invocation of a virtual function.
5 Conclusions In this paper we have described a new parallel pattern-based system using generic programming techniques. The generic representation for a set of algorithms based on their characters is one important feature that distinguishes this system from other pattern-based approaches in parallel computing. To achieve higher extensibility and flexibility, the template parameters of the generic pattern are defined as class templates and they can interact with each other in a flexible way. The system also allows a programmer to reuse the components of a design for a specific application. Thus, the user can develop new parallel applications quickly and easily. Moreover, because the generic patterns are also defined independently from each other, adding new patterns to the system is quite simple. We have presented the effectiveness of this system for some popular two dimensional and three dimensional applications in bioinformatics: Smith-Waterman, Nussinov, syntenic alignment, multiple sequence alignment and spliced sequence alignment. The measurements show that the new system can generate parallel programs with substantial performance gains for these algorithms. Our future work in parallel program design patterns will include adding more patterns to the system and identifying more applications that can benefit from this technique. Now, we are working on the identification of parallel design patterns that are frequently used on popular data
7
structures in bioinformatics such as sequences, trees and matrices. It would also be interesting to develop the cost analysis and performance prediction tools for the system.
References 1. J. Anvik, J. Schaeffer, D. Szafron, and K. Tan: Why Not Use a Pattern-Based Parallel Programming System, Proceedings of EURO-PAR’2003, LNCS 2790, 2003. 2. J.C. Browne, M. Azam, and S. Sobek: CODE: A unified approach to parallel programming, IEEE Software, pp.10-18, 1989. 3. A. Bartoli, P. Corsini, G. Dini, C.A. Prete: Graphical design of distributed applications through reusable components, IEEE Parallel Distrib. Technol, 3, 1995. 4. J. C. Browne, S. I. Hyder, J. Dongarra, K. Moore, P. Newton: Visual programming and debugging for parallel computing, IEEE Parallel Distrib. Technol, 3, 1995. 5. R. Durbin, S. Eddy, A. Krogh, G. Mitchison: Biological Sequence Analysis- Probabilistic Models of Protein and Nucleic Acids, Cambridge University Press, 1998. 6. J. Gerlach: Generic Programming of Parallel Application with JANUS, Parallel Processing Letters, Vol. 12, No. 2, pp. 175-190, 2002. 7. M.S. Gelfand, A.A. Mironov, and P. A. Pevzner: Gene recognition via spliced sequence alignment, Proc. Natl. Acad. Sci, Vol. 93, pp. 9061-9066, 1996. 8. Z. Galil, K. Park: Dynamic Programming with Convexity, Concavity and Sparsity, Theoretical Computer Science, 92, pp. 49-76, 1992. 9. D. Goswami, A. Singh, B.R. Preiss: From Design Patterns to Parallel Architectural Skeletons, Journal of Parallel and Distributed Computing, 62(4), 669-695, 2002. 10.X. Huang, K. M. Chao: A generalized global alignment algorithm, Bioinformatics, 19(2), pp. 228-233, 2003. 11.W. Liu and B. Schmidt: Parallel Design Pattern for Computational Biology and Scientific Computing Applications, Proceedings of IEEE International Conference on Cluster Computing, 2003. 12.D. W. Mount: Bioinformatics-Sequence and Genome Analysis, Cold Spring Harbor Laboratory Press, 2001. 13.W. Miller, E. Myers: Optimal alignments in linear space, Computer Applications in the Biosciences, 4, pp. 11-17, 1988. 14.S. Siu, A. Singh: Design patterns for parallel computing using a network of processors, Sixth IEEE International Symposium on High Performance Distributed Computing, pp. 293-304, 1997. 15.A. Singh, J. Schaeffer, M. Green: A template-based tool for building applications in a multi-computer network environment, Parallel Computing, 461-466, 1989. 16.J. Schaeffer, D. Szafron, G. Lobe, I. Parsons: The enterprise model for developing distributed applications, IEEE Parallel Distrib. Technol, 1, pp. 85-96, 1993. 17.T. F. Smith, M. S. Waterman: Identification of common subsequences, Journal of Molecular Biology, pp. 195-197, 1981. 18.D. Vandevoorde, N. M. Josuttis: C++ Template: The Complete Guide, Addison Wesley, 2002. 19.M. Zuker, P. Stiegler: Optimal computer folding of large RNA sequences using thermodynamics and auxiliary information, Nucleic Acids Research, 9, 1981.
8
