Ccaffeine Framework: Composing and Debugging Applications ...

Ccaffeine Framework: Composing and Debugging Applications Interactively and Running Them Statically Benjamin A. Allan and Rob Armstrong March 1, 2005

Abstract Ccaffeine is a Common Component Architecture (CCA) framework devoted to high-performance computing. In this note we give an overview of the system features of Ccaffeine and CCA that support component-based HPC application development. Object-oriented, single-threaded and lightweight, Ccaffeine is designed to get completely out of the way of the running application after it has been composed from components. Ccaffeine is one of the few frameworks, CCA or otherwise, that can compose and run applications on a parallel machine interactively and then automatically generate a static, possibly self-tuning, executable for production runs. Users can experiment with and debug applications interactively, improving their productivity. When the application is ready, a script is automatically generated, parsed and turned into a static executable for production runs. Within this static executable, dynamic replacement of components can be performed by self-tuning applications.

1 Introduction The Ccaffeine framework supports the CCA specification, allowing single program multiple data (SPMD) [3] and hybrid SPMD applications to be constructed. The Common Component Architecture specification [5] enables highperformance components to be written following the uses-provides pattern [8]. The architecture does not restrict the type of interfaces exchanged between components, or the language used to implement a component, or the type of interprocess communication used to transfer data in a parallel-processing environment. The specification includes a BuilderService interface that component instances can use to create and manipulate other components and connections in the same framework [4]. CCA Pattern for Component Composition The basis of all component interactions in a CCA framework is the exchange of public interfaces (ports) which is mediated by the framework, as described in full detail in [7]. Briefly, each component publishes its provided and used interfaces to the corresponding Services object received from the framework. Figure 1 shows a typical application, created interactively and rendered in the Ccaffeine GUI style. BuilderService and AbstractFramework provide APIs for run-time assembly and modification of applications (instantiate and destroy components, make and break connections between ports). By creating an AbstractFramework instance, any code can control a CCA framework and even make itself appear in the framework as a component. These services form the basis of interactive parallel operation. Not only does the drag and drop GUI use these services, but also the application itself can swap components based on numerical or computational performance [6]. AbstractFramework also enables encapsulation of groups of components so that they can be treated as a single component. This eases management of large component-based applications, including the assembly of multi-scale or multi-physics simulations, where complex applications representing a particular length scale or type of physics can be encapsulated and treated as a single component, exposing only a limited number of ports. Overall Ccaffeine Design Ccaffeine-based applications are built with a layered software system. The layers are structured to support three stages of parallel component application development: interactive assembly, debugging, and optimized production runs.


baallan,rob  @ca.sandia.gov, Sandia National Laboratories, MS9158, 7011 East Ave, Livermore CA 94550

1

GO

MyMesh

TempICPort MyMesh

InitCond

GrACEPort

InitConditions

ChemistryIntegrator

StatsPort

BoundaryConds MyMesh

BC

AMR_Mesh

StatsPort MyMesh

ModelStatistics

AMRDriver

BoundaryConds

DETimeAdvance

ModelODE

Results

ModelIterationMatrixODE

ModelODE

MyMesh

ModelIterationMatrixODE

BCPort

PatchSetter

ESIFactory

ReactionModel

Figure 1: Adaptive Mesh Refinement 2D chemical kinetics application from [2]. Each large box represents a component with provides ports on the left side and uses ports on the right. Lines indicate port connections, not data-flow or control-flow. The methods of each port are defined by the combustion software architect or the author of each reusable numerical component. The flow of control is determined by the AMRDriver component (upper left) once the framework invokes its GO port.

IntegratorConfiguration ModelVectorNorm ModelVectorNorm MPI

VectorExtensionsDIPort

Weighted2Norm

ModelWeights

IntegratorLSODE MPI Advance AMR

ModelWeights

ODEAdvance

VectorExtensionsDIPort

MPIService

ModelWeightsConfiguration

ODEResults

WeightsDefault

ProblemWeightConfig ESIFactory

ESIFactory

MPI

PatchSetter

PatchToVector

Multiplexer (SPMD case)

TTY

Python script

VectorExtensionsDIPort

Generated main (c++)

MPI

ESISupport

ESIFactory_Petra

Figure 2: Layer 1, present only for interactive building and debugging, is a multiplexer for control signals and a Java graphic user interface (GUI) or text interface. Layer 2 of the system is same-address-space drivers, which may be either a custom user code in any babel-supported language including Python, a generated code based on a previous interactive run, an interactive controller, or a simple Ccaffeine script language input file. The core framework (layer 3) provides container objects (frames), that meet CCA’s AbstractFramework definition. Layer 4 wraps the generic uses-provides framework engine to meet a variety of component specifications. The final layer is the application components.

CCAFE-GUI (model-viewcontroller)

Text line-protocol builder interface

1

FILE 2

Babel AbstractFramework & BuilderService Ccaffeine frame core and opaque component model engine implementing the uses-provides pattern.

CCA Services

CCA Components

3

4 5

As shown in Figure 2 any of several user interface configurations can be substituted to set up an application in Ccaffeine. Applications may be built and debugged in two ways: using a drag and drop GUI or typing commands to a terminal. In both cases commands and responses are exchanged with the running framework through a socket, with the GUI translating graphical gestures into commands underneath. Each process instance of the framework expects to carry out this dialog. In a parallel setting, framework instances are multiplexed from the GUI or terminal commandline. Each framework instance is autonomous, requiring (and providing) no parallel communication intrinsically. Parallelism is left to be defined by the final main program in whatever way is suitable to the components being used. For non-interactive runs, a script file derived from interactive use can be parsed into a C++ main program and then compiled to eliminate layers one and two of Figure 2.

2 Interactive and Production Modes of Use Composing and Debugging Applications Interactively The CCA specification provides a variety of opportunities for interaction in component-based programs. Each component can expose a tunable parameter set by providing a CCA-defined ParameterPort. Assuming that the components themselves are production quality, the bulk of interactive debugging in a component-based application consists of determining the correct wiring among ports using the GUI and adjusting the tunable parameters to obtain the desired simulation. For intra-component trouble shooting, a debugger may be applied in the standard way. Ccaffeine defines a simple line-based command language to create and configure component applications. The user’s interactions with the GUI are translated into the command language, logged, and sent to the running framework. These GUI logs can be saved, adjusted off-line if desired, and reloaded or finally converted into scripts for a queued batch system. The GUI implementation follows the model-view-controller pattern, is largely separable from the Ccaffeine framework proper, and is separately extensible. 2

Currently the only interactive parallel mode that is explicitly supported is SPMD. Because there is an instance of the framework running on every participating process and because the frameworks make no effort to synchronize (i.e. are embarrassingly parallel), the provided multiplexer that sits between the interactive user and the cohort of frameworks enforces a SPMD pattern. The GUI and each parallel instance of the framework thinks it is talking to exactly one counterpart. The Ccaffeine multiplexer replicates each action requested by the user to every one of the frameworks identically, preserving the SPMD paradigm across the machine. A number of users have exploited BuilderService to create sequences of SPMD programs running in different partitions in a queued batch setting. The user can request that output be logged to separate per-node files or a parallel debugger may be employed for trouble shooting. The framework and GUI or command-line can be used in a singleton process mode as well. In this case the multiplexer is bypassed in layer 1 of Figure 2. In both the parallel and singleton case, the GUI output can be filtered to create a batch-oriented Ccaffeine script file for queued batch use. Although the interactive mode enforces a SPMD pattern, there is no such restriction in a batch mode when the user defines the final main program. Batch Operation and Executables Derived from Interactive Design An important goal of Ccaffeine is to convert the information generated from interactive use into a static executable for batch runs. The key step in fully automating the generation of a custom static program from components is obtaining the installation data about the component object and header files and ports used in the simulation. While this installation data can be obtained manually from the user, this is repetitive and error prone when the number of components is large (Figure 1). For each installed component there is also installed a CCA XML description of the component. The draft XML specification describes components and the component behaviors which are not readily captured by other parts of the specification, such as which ports it has and what non-component legacy libraries are used by the component. We have extended this description to include the installation data for headers and libraries. Installed with the CCA specification software is a tool, genCCASCL.sh, which helps component developers generate correct deployment description files in XML. For a single component this includes information about any of three library versions: compiled for static linking, compiled for dynamic linking, and compiled with libtool [1]. Helping out adoption of this standard way of preparing a component is the fact that the CCA Tutorial Source and Toolkit Template use this mechanism. Given the location information for the components’ header and object files, the interaction log files are parsed to generate a main() program that emulates the interactive session. The program is automatically linked with the Ccaffeine framework, and upon execution, makes calls on AbstractFramework and BuilderService that replicate the user’s plug and play composition. Originally envisioned as a means to easily move an interactively composed application into a queuing environment, other uses have emerged. For example, application profiling and optimization often require compilation to a statically linked executable to perform well or at all. Example The complete details of using the code-generation can be examined using the examples installed with Ccaffeine in the share/ccafe-$version/examples directory. In Figure 3 we see a script input which enables automated run-time quality-of-service tuning via the Mastermind and TAU components. If the integrators.Simpson and integrators.Trapezoid components are not included in the factory via repository get commands, then at run-time they are not available in a statically linked application. The Mastermind may optimize the integration process by swapping Simpson and Trapezoid component instances as needed.

3 Conclusion Ccaffeine provides a high-performance CCA environment for many interactive, debugging, and production modes of component-based applications. Application structures can be debugged interactively and then fine-tuned during production runs within a statically linked executable. The automatic code-generation system we have described is limited to SPMD applications, but with modest adjustments can be used to generate MPMD applications based on a multiple input scripts. The CCA specification is strongly minimalist in design, with few convenience functions to support over-worked user. Manually writing a correct code that uses BuilderService and AbstractFramework is challenging, so we have created tools to automate writing drivers. Ccaffeine GUI logs can be filtered to generate a statically linked driver based on the CCA AbstractFramework and BuilderService. Interestingly, this generated main() is not Ccaffeine specific. The generated code is portable to other CCA frameworks by changing the choice of implementation class for

3

# Tell the framework (or code generators) where components are found. # The path(s) set will be searched for CCA component XML files. path set /scratch/lib # Link component classes to the factory in the frame. # The XML files will be scanned to find the correct libraries. repository get tau.Measurement repository get cfrfs.Mastermind repository get integrators.Simpson repository get integrators.Trapezoid GOPort IntegratorPort IntegratorPort repository get drivers.CXXDriver # Create the initial component instances integ1 tunerPort instantiate drivers.CXXDriver driver instantiate cfrfs.Mastermind optimizer instantiate tau.Measurement tau driver instantiate integrators.Simpson integ1 # Wire up the initial configuration of ports connect driver IntegratorPort integ1 IntegratorPort connect driver tunerPort optimizer tunerPort tunerPort measurementPort perfMeterPort connect optimizer measurementPort tau perfMeterPort # Hand control to the application logic: optimizer tau go driver GOPort

Figure 3: Script input for self-tuning integration application gov.cca.AbstractFramework from ccaffeine.AbstractFramework to an implementation by another framework, and if the components are fully CCA compliant, the parallel program will execute just as well using another framework. Acknowledgments This work was supported by the United States Department of Energy, in part by the Office of Advanced Scientific Computing Research: Mathematical, Information and Computational Sciences through the Center for Component Technology for Terascale Simulation Software. This is an account of work performed under the auspices of Sandia Corporation and its Contract No. DE-AC04-94AL85000 with the United States Department of Energy and is available as technical report SAND 2005-1135C.

References [1] B. Allan. 2005.

CcaBabel.

http://www-unix.mcs.anl.gov/CCAToolsWiki/moin.cgi/CcaBabel,

[2] B. A. Allan, S. Lefantzi, and J. Ray. ODEPACK++: Refactoring the LSODE Fortran library for use in the CCA high performance component software architecture. In Proceedings of the 9th International Workshop on HighLevel Parallel Programming Models and Supportive Environments (HIPS 2004), Santa Fe, NM, April 2004. IEEE Press. see http://www.cca-forum.org/˜baallan/odepp. [3] Benjamin A. Allan, Robert C. Armstrong, Alicia P. Wolfe, Jaideep Ray, David E. Bernholdt, and James A. Kohl. The cca core specification in a distributed memory spmd framework. Concurrency and Computation: Practice and Experience, 14(5):323–345, 2002. [4] D. E. Bernholdt, R. C. Armstrong, and B. A. Allan. Managing complexity in modern high end scientific computing through component-based software engineering. In Proceedings. of the HPCA Workshop on Productivity and Performance in High-End Computing (P-PHEC 2004), Madrid, Spain. IEEE Computer Society, 2004. [5] CCA specification. http://cca-forum.org/specification/, 2005. [6] Boyana Norris et al. Computational quality of service for scientific components. In Ivica et al. Crnkovic, editor, Proceedings of the International Symposium on Component-Based Software Engineering (CBSE7), volume 3054 of Lecture Notes in Computer Science, pages 264–271, Edinburgh, Scotland, 2004. Springer. [7] David E. Bernholdt et al. A component architecture for high-performance scientific computing, 2004. submitted to Intl. J. High-Perf. Computing Appl. [8] E. Gamma et al. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1994.

4