Simulation Interoperability Workshop, Paper 09S-SIW-033
Multiplatform Support for the OpenMSA/OSAMS Reference Implementation Craig N. Lammers (Contractor) Maria E. Valinski (Contractor) Jeffrey S. Steinman, Ph.D. (Contractor) Joint Program Executive Office, Chemical and Biological Defense Software Support Activity Space and Naval Warfare Systems Center, Pacific 53560 Hull Street San Diego, CA 92152-5001
[email protected] [email protected] [email protected] Key Words: Parallel and Distributed Simulation, Multicore, Manycore, PDMS, OpenMSA, OSAMS, WarpIV Kernel, SPEEDES
ABSTRACT: This paper discusses issues encountered in porting the Open Modeling and Simulation Architecture (OpenMSA) and the Open System Architecture for Modeling and Simulation (OSAMS) open source reference implementation from UNIX (Linux, Mac OS X, Solaris, HP-UX, etc.) to the Microsoft Windows (XP and Vista) operating system. The OpenMSA architecture provides a high-performance parallel-and-distributed event-processing framework for modeling and simulation. OSAMS facilitates high-performance plug-and-play model interoperability within the OpenMSA. Together, these architectures support parallel and distributed processing on multicore computer systems using shared memory and standard network protocols to facilitate interactions between distributed models. Several striking differences in how (a) processes are created, (b) shared memory is created/accessed, and (c) network communications are supported make this port interesting and challenging. In addition to the many lessons learned, this paper presents timing results of several parallel and distributed performance benchmarks on four-processor multicore Windows, Linux, and Mac OS X platforms.
1.0 Introduction
This paper discusses the differences between inter-process communication services on Windows and UNIX-based systems, a solution for supporting multiplatform process creation in OpenMSA/OSAMS, and parallel and distributed performance benchmarks on multicore Windows, Linux, and Mac OS X platforms.
The Open Modeling and Simulation Architecture (OpenMSA) is an open-source architecture that provides a high-performance parallel-and-distributed discrete eventprocessing framework for Modeling and Simulation (M&S). The Open System Architecture for Modeling and Simulation (OSAMS) facilitates high-performance plugand-play model interoperability within the OpenMSA.
2.0 Multiplatform Porting Issues This section discusses the issues encountered during the UNIX to Windows software port. Specifically, the following subsections cover network communication, shared memory communication, process creation, the system command, environment variables, secure string functions, and compilation.
At the core, the OpenMSA utilizes network and shared memory communication to coordinate message passing between multiple processes in parallel and distributed execution environments. Until recently, OpenMSA/OSAMS only supported UNIX-based systems including Linux, Mac OS X, Solaris, and HP-UX. Differences in process creation and inter-process communication introduced roadblocks that prohibited support for Windows systems. Multiplatform support for OpenMSA/OSAMS enables the same models to execute on parallel-and-distributed environments on both Windows and UNIX-based systems without modification.
2.1 Shared Memory Communication A shared memory segment is a block of memory that can be simultaneously accessed by multiple processes. Changes made to the memory by one process are immediately reflected in other processes. By allowing multiple processes access to the block of memory, shared
1
Simulation Interoperability Workshop, Paper 09S-SIW-033
memory communication eliminates the need to duplicate data in multi-core applications. To guarantee coordination between multiple processes, flags are set to specify when it is safe to read or write to a shared memory segment. OpenMSA/OSAMS utilizes shared memory to efficiently communicate between multiple processes.
unique key based upon the process ID and a global counter, and returns the key by reference. When a process creates or accesses a shared memory segment for the first time in Windows, OpenMSA/OSAMS saves the segment handle in a global tree using the segment key as the key for the tree element. This completely hides the Windows handle from the developer, so the integer key is all that is required to uniquely identify a shared memory segment. When cleaning up the shared memory resources for a specific segment, the wrapper function retrieves the handle from the tree given the segment key.
Initially, a process creates a shared memory segment of a specified size with specific access permissions. Given a unique key, other processes can attach themselves to one or more segments in order to read and write to the shared memory. The shared memory segment is mapped into the address space of a process when it attaches to the segment, and un-mapped from the address space when it detaches from the segment. When a shared memory segment is no longer needed, a process cleans up the resources associated with the segment.
2.2 Network Communication OpenMSA/OSAMS provides a variety of services that leverage network communication between machines. At the heart of OpenMSA/OSAMS, a high-speed communications framework integrates shared memory with reliable communication across a network using the Transmission Control Protocol and Internet Protocol (TCP/IP). In addition, interoperability with Distributed Interactive Simulation (DIS) is provided using User Datagram Protocol and Internet Protocol (UDP/IP) connectionless streams. TCP/IP and UDP/IP are the fundamental network communication standards that define the connection, communication, and data transfer procedure between two points.
There are two major differences between using shared memory communication in the UNIX and Windows environments. The first difference corresponds to the creation of a shared memory segment. Windows requires the developer to specify a unique key during creation of the shared memory segment, whereas UNIX can automatically generate unique keys. This key is required for multiple processes to access a particular shared memory segment. The second difference corresponds to cleaning up shared memory resources. In UNIX, shared memory resources for a segment are cleaned given the key of the segment. In Windows, shared memory resources for a segment are cleaned given the reference handle that is returned from creating or accessing a segment. In addition to the segment key, the Windows handle to the segment must be saved to clean up shared memory resources. Table 1 lists the UNIX and Windows functional equivalents to the shared memory utilities. [3]
The Berkeley Software Distribution (BSD), or some variant thereof, provides the core network programming services in most UNIX-based systems. The Windows Sockets (Winsock) Applications Programming Interface (API), derived from BSD, provides the core network programming services in Windows systems. This software port utilized the latest Winsock 2 API. Providing cross-platform support for socket applications is relatively straightforward. With some minor exceptions, the BSD and Winsock APIs almost directly map to one another. There are, however, some differences.
Table 1: Mapping of shared memory utilities in UNIX and Windows. Note the shmget function has a dual purpose in UNIX. A shared memory segment is created if the key is set to IPC_PRIVATE and the IPC_CREAT bit is set in the flag. UTILITY
UNIX
WINDOWS
Create shared memory segment
shmget
CreateFileMapping
Access shared memory segment
shmget
OpenFileMapping
Map segment into address space of a process
shmat
OpenFileMapping MapViewOfFile
Unmap segment from a address space of a process
shmdt
UnmapViewOfFile
Clean up segment resources
shmctl
CloseHandle
For starters, Windows requires initializing the Winsock framework prior to using any of the socket functions. The Winsock framework is initialized using the WSAStartup function and terminated via the WSACleanup function. The Winsock version is specified as a parameter to the startup function. With a few exceptions, socket function names are relatively consistent between the BSD and Winsock APIs. Sockets are closed in Winsock via the closesocket function, versus the close function in BSD. To control the I/O mode of a socket, Winsock provides an ioctlsocket function as an alternative to the ioctl and fcntl BSD functions. These differences are summarized in Table 2. [1]
Wrapper functions were developed in OpenMSA/OSAMS to abstract the cross-platform nuances away from the developer. When creating a shared memory segment in Windows, OpenMSA/OSAMS automatically generates a
2
Simulation Interoperability Workshop, Paper 09S-SIW-033
Table 2: Discrepancies in socket function names. SOCKET UTILITY
BSD
WINSOCK
Close a socket
close
closesocket
Control I/O mode of a socket
ioctl/fcntl
ioctlsocket
conditional compilation are recommended [2]. Conditional compilation is shown in Code Segment 2. #ifdef WINSOCK SOCKET socketFd = socket(AF_INET, SOCK_STREAM, 0); if (socketFd == INVALID_SOCKET) { cout
