Framework for Improving Parallelism by Write- Update ... - CiteSeerX

INFORMATION PAPER International Journal of Recent Trends in Engineering, Vol. 1, No. 2, May 2009

Framework for Improving Parallelism by WriteUpdate Coherence Protocol in Distributed Shared Memory System Prof. Bal Gopal1, Pankaj Kumar2 1

Integral University, Lucknow, India [email protected] 2 Sahara Arts & Management Academy, Lucknow, India [email protected]

Abstract - For increasing parallelism in DSM System, the replication of data is to be done. The replication of data increases parallelism by allowing reads of the same data to be executed in parallel. However there is inconsistent result occurs for parallel writes and required to maintain memory consistency. The coherency protocol is required to maintain memory consistency for ensuring the serialization of write operation and that any subsequent reads or writes access the update data. The updates are propagated by the remote site. Now it is upto the coherency protocol to take these update and incorporate the update into the local copies. We study the memory coherence and memory consistency used in DSM system and analyzed that a framework can be designed that offers the ability to improve parallelism by maintaining the memory consistency with the use of write-update coherence protocol in DSM system.

coherence is just giving a responsibility to take special action when one processor writes to a block of data that exists in other memory which is now old, must be either invalidated or updated with the new value, depending on the protocol [1, 3, 10]. The memory coherence in distributed memory system (DSM) differs form other system. In a DSM, local memories are considered as large caches of the global shared address space. Pages are replicated on demand in the local memory of processors requesting them, and can be loaded either from disk or from another local memory. Each page can be in the form of owner mode or manager mode. The owner of a page has a valid copy of the page; it is the only one to have a copy of the page and it can be accessed through a write operation. The manager of a page keeps a directory entry for that page containing coherence information related to it [7, 9, 11].

Keywords - Distributed Shared Memory (DSM), Memory Coherence, Memory Consistency, coherence protocol

II. I. INTRODUCTION

The most relevant work is that of Weisong Shi, Weiwu Hu and Zuimin Tang [14] on analyzing the interaction of coherence protocol and memory consistency models in DSM System. They concluded that a memory consistency model can be defined as the combination of an event ordering mechanism and coherency protocol Li and Hudak [10] and Bill Nitzberg and Virginia Lo [3] say that two type of memory coherence protocol is used one is write-invalidate coherence protocols and other is write-update coherence protocol. They rejected use of a write-update protocol at the onset with the reason that network latency would make it inefficient. But Bill Nitzberg and Virginia Lo show that in the appropriate hardware environment write-update protocols can be implemented efficiently. Another work is by Jason F. Cantin, Mikko H. Lipasti, Member and James E. Smith [8] gives the complexity of verifying Memory Coherence and Consistency. They concluded that all memory consistency models implemented in practice either provide memory coherence or allow the programmer to serialize memory operations with special instruction and the complexity of verifying adherence to a memory

DSM represents a successful hybrid of two parallel computer classes i.e. shared memory and distributed memory. It provides the shared memory abstraction in system with physically distributed memories and consequently combine the advantages of both approaches [2, 4, 6]. DSM system is the memory system that is physically distributed but logically implements a single shared address space. A consistency defines how the consistency of the data values in the DSM system is maintained. It is a tradeoff between minimizing memory access order constraints and the complexity of the programming model as well as the complexity of the memory model itself. Memory consistency imposes ordering restriction on access depending on a number of attributes [5, 8, 9, 12 13]. A memory is said to be coherent if the value returned by a read operation to the same address is always same as the value written by the most recent write operation. Generally architecture with one memory access path has no coherence problem. All DSM systems provide some form of memory coherence. For DSM, memory coherence occurs if all copies of a memory location in multiple memories remain consistent when the contents of that memory location are modified. Maintaining 201 © 2009 ACADEMY PUBLISHER

RELATED WORK

INFORMATION PAPER International Journal of Recent Trends in Engineering, Vol. 1, No. 2, May 2009 consistency is not a mere consequences of requiring coherence. III.

D.

Coherency Protocol The coherency protocol works as interface between communication service and the memory consistency model. The core of the framework is the coherency protocol because it find what update is actually required and what are the destination of updates. A write-update coherency protocol is used in the frame work to maintain memory consistency for ensuring the serialization of write operation and that any subsequent reads or writes access the update data. The coherency protocol described in the framework is based on three component memory management, synchronization and ownership management as shown in the Figure 3.

FRAMEWORK OVERVIEW

The approach taken for our framework in this paper is to improve parallelism by maintaining the memory consistency using memory coherency protocol for DSM system. The basic relation between memory consistency and memory coherency is shown in the Figure 1 Read/Write Memory Consistency

Memory Coherency Synchronize

Figure 1. Relationship between consistency and Coherency

a.

Memory Management The memory management component is responsible for the management of local copy of data items and it performs two major operation; the processing of access operation and the handling of updates. The access operations comprise of read and write operation to the local copy. The synchronization is provided by connected consistency model. Write operation comprises the memory management component to update a location of local copy with a given value.

The relation only by taking read, write and synchronization operation defines the relationship between consistency and coherency [3, 14]. The memory consistency in the relation defines the level of consistency provided by the system while the coherency is responsible for managing data so that the required level of consistency is actually provided. There are four components are used in the framework which are shown in Figure 2. A.

Consistency For the memory consistency maintenance shared/competing/synchronization category access is taken. All consistency models are derived from a single base class. The base class defines all consistency models with three operations i.e. read, write and synchronization. The read and write operation are comprises into access operation. The all three operation are mapped to corresponding operation provided by an underlying coherency.

Coherency Protocol Ownership Management

Synchronization

Memory Management

Figure 3. Memory Coherence Protocol (MCP)

Memory Consistency

b.

Synchronization The synchronization component can be used distribution component interchangeably. The synchronization component is responsible of exchange of data between copies. It is linked closely to the communication mechanism. The synchronization component provides method to send and request update from remote nodes. The method send update takes as a parameter an update and set the destination. The updates are sent to the address of the given destination.

Read/Write & Synchronization

Coherency Protocol

Concurrency Control

Communication Service

c.

Ownership Management The ownerships management component is for the proper administration of ownership of copies. The main activity of ownership component is to find the location of other copies shred data. For enabling this service it provided the following methods: 1. A method to announce that a local copy is being shared. 2. A method to determine the source of update. 3. A method to determine the destination of update.

Figure 2. The framework overview

B.

Communication Service The communication service is used for processor communication, message passing making input and output. C.

Concurrency Control Concurrency controls the concurrent access. It works as an interface between memory consistency and communication services. 202 © 2009 ACADEMY PUBLISHER

INFORMATION PAPER International Journal of Recent Trends in Engineering, Vol. 1, No. 2, May 2009 The actual definition of ownership management is defined by the activity of synchronization component and communication services. IV.

c)

Step3. Finding the destination (Figure 5.) a) Now the destinations of verified updates are required. b) The synchronization component asks the ownership management about the destination of these updates. c) The ownership management finds the destination of these updates by access event which is provided by the memory consistency model. d) The destinations of updates are sent to the synchronization component.

WORKING AND DEVELOPMENT

The framework provides a general interface that allows the memory consistency model to communicate with coherency protocol. The core method is to update and ensure memory consistency. As we have taken write – update coherence protocol so main concern is on the update operation. An update is a collection of data and descriptor. A descriptor holds the references to the location in the local copies of shared data. References may describe the single location or memory address or set of locations. An update holds descriptor that point to various locations in local copies of shared data. In these locations the corresponding data from the update have to insert in order to update a local copy. The update operations here depend upon the protocol implemented by synchronization component. The working of the framework is described in two parts. First part finds the destination of node to propagate and incorporate the updates and the second part propagates the updates to all nodes.

Update Operation

SYN

Update Request

Update Propagation

Step4. Incorporating updates to local copies a) The updates are sent to its destination by the synchronization component. b) Then source is then asked for update by using the request update of the synchronization component. c) The return update is now incorporated into local copies by the memory management component. B. Propagating updates to all nodes After incorporating updates to local copies now write update coherence protocol propagate the updates to all nodes. The propagation of update is started with block’s master node and proceeds to copy-list chain of nodes. The copy-list chain of a node tells about the master of that particular node and the next copy of that node. The write request is done by the initiator and propagates to all nodes according to the copy list information. The write request is completed when the last node in the copy-list chain acknowledges about the completion of the updates to the initiator of the write request. For example (fig.6.) let A, B, C, D are four nodes. A is the master of all node. Node D is initiator as it wants to write and C is last node as it doesn’t have any next copy. The propagation of updates in the said example completes in following steps: 1. MCP of node D sends write request to node B. 2. MCP of node B sends this update to its master node which is A. 3. As node A is the master node so MCP of node A updates its copy-list X. 4. Now MCP of node A sends this update message to its next copy i.e. node B.

Read & Write Operation Update Operation 1) What to update & Incorporate update

SYN Return update Update Propagation

Figure 4. Disbursement of updates between memory management (MM) component and synchronization (SYN)

b) The memory management component takes these updates and verifies these updates by seeing what the update requests are. It also takes the help of read and write operation provided by the memory consistency model.

203 © 2009 ACADEMY PUBLISHER

What is destination

Figure 5. Disbursement of updates between Synchronization (SYN) and Ownership management (OM)

Step2. Finding the Actual Update (figure 4) a) The synchronization component take these update propagation and incorporate these update and asks to memory management component what to update and what not to be updated.

Update Request

OM 3)

Finding the destination and incorporating updates Propagating updates to all nodes the framework first find the actual updates and their destination. After getting the destination of all updates the framework incorporates all updates to local copies of all node. This activity is described in the following steps.. Step1. The method update distributed update propagation from remote site.

2)

Access Event

4) Return destination

A.

MM

After the verification the verified updates are returned to the synchronization components by the memory management components.

INFORMATION PAPER International Journal of Recent Trends in Engineering, Vol. 1, No. 2, May 2009 5. 6.

7.

MCP of node B updates its copy-list and sends this update message to its next copy i.e. node C. MCP of node C updates its copy-list. As it contains no next copy so it is the last node. Here the propagation of update message completes. MCP of node C sends an acknowledgment to initiator node (D) that updates is completed. Copy- list Master =A

3)

X

Next Copy on B

MCP updates X

4) MCP sends update message to next copy

Node A Copy- list Master =A

Copy- list Master =A

6)

DSM System. The core of our framework is the division of sharing policies into memory consistency and coherency protocol. The paper describes the working of write-update coherence protocol in detail. The framework unifies the memory consistency model, memory coherence, communication service (used for message passing) and concurrency control into one place. To illustrate the usefulness of the framework we defined its working by figures also.

X

No Next Copy

REFERENCES [1] Anant Agarwal, “Richard Simoni, John Hennessy. and Mark Horowitz, “An Evaluation of Directory Schemes for Cache Coherence”, CH2545-2/88/0000/0280S01.00 C 1988 IEEE. [2] Anne-Marie Kermarrec, Gilbert Cabillic, Alain Gefflaut, Christine Morin, Isabelle Puaut, “A Recoverable Distributed Shared Memory Integrating Coherence and Recoverability”, Jan 1995. [3] Bill Nitzhug and Virginia Lo. “Distributed Shared Memory: A Survey of Issues and Algorithms” 001189162/91/0800-0052 IEEE 1991.Changhun Lee "Distributed Shared Memory" Proceedings on the 15th CISL Winter Workshop Kushu, Japan ¢ February 2002. [4] D. Mosberger: “Memory consistency models”, Operating Systems Review, 17(1):18-26, Jan. 1993. [5] Ing. Thomes Seidmann," Distributed Shared Memory in Modern Operating System" Ph.D. Thesis, Slovak University of Technology, January, 2004. [6] J. Silcock, “Distributed Shared Memory: A Survey”, Deakin University, Geelong, 28 May 1997. [7] Jason F. Cantin, Student Member, IEEE, Mikko H. Lipasti, Member, IEEE, and James E. Smith, Member, IEEE "The Complexity of Verifying Memory Coherence and Consistency" IEEE Transactions On Parallel And Distributed Systems, Vol. 16, No. 7, July 2005. [8] John Hennessy, Mark Heinrich, and Anoop Gupta, “Cache-Coherent Distributed Shared Memory: Perspectives on Its Development and Future Challenges, “Proceedings of the IEEE, Vol. 87, No. 3, March 1999. [9] Kai Ll, Paul Hudak “Memory Coherence in Shared Virtual Memory Systems” ACM Transactions on Computer Systems, Vol. 7, No. 4, November 1989, Pages 321-359. [10] Lisa Higham, Jalal Kawash and Nathaly Verwaal, "Define and Comparing Memory Consistency Model" ©1997 ISCA, Proceeding of PDCS'97. [11] Paul Krzyzanowski "Distributed Shared Memory and Memory Consistency Models" Rutgers University – CS 417: Distributed Systems ©1998, 2001. [12] Ramon Lawrence, “A Survey of Cache Coherence Mechanisms in Shared Memory Multiprocessors” May 14, 1998 [13] Weisong Shi, Weiwue Hu and Zhimin Tang “An interaction of coherence protocol and memory consistency models in DSM System” ACM operating system review, Vol 31, No. 5, October 1997. [14] Xiaowei Shen, Larry Rudolph, Arvind, CACHET: An Adaptive Cache Coherence Protocol for Distributed Shared-Memory Systems”, 13th ACM SIGARCH International Conference on Supercomputing, Rhodes, Greece, 1999, January.

MCP updates X

7) MCP acknowledges that updates are complete

Node C X

Next Copy on C

Page Map Table Node B

2) MCP sends update message to master node

X

Page P

1) MCP sends write request to node B

5) MCP updates X and sends update message to next copy

Node D (Initiator)

Node B Figure 6. Implementation of write-updates coherence protocol (MCP: Memory Coherence Protocol)

If the propagation of updates happens successfully it means that all node are now aware about the write operation initiated by any node. So all nodes except the initiator node invalidate its write operation and the consistency of memory is maintained. V. CONCLUSION The parallelism can be improved in DSM system if more than one operation is executed by different processor at the same time. So in this case replication of data is required and more than one write request is to be done. But multiple write-requests create some inconsistent result when the location is same for all write. Now it becomes important to maintain the consistency of memory. The framework described in the paper solves this problem by using write-update coherence protocol and memory consistency model. The framework finds the address of updates and propagates it to all nodes. By this activity all processor comes to know about any writerequest and hence the consistency of memory is maintained. The paper provides a framework which maintains the memory consistency with the help of write-update memory coherency protocol to improve parallelism in

204 © 2009 ACADEMY PUBLISHER