Feb 27, 2016 - arXiv:1610.02288v1 [cs.DC] 27 Feb 2016. VF ... 1, B-2020 Antwerpen. (version revised on April 29, 2015) ... ã Voting Algorithms 29 ã. ã The Voter ...
CWEB
VF
arXiv:1610.02288v1 [cs.DC] 27 Feb 2016
The Voting Farm A Distributed Class for Software Voting Vincenzo De Florio Universiteit Antwerpen Department of Mathematics and Computer Science MOSAIC research group Middelheimlaan 1, B-2020 Antwerpen
(version revised on April 29, 2015)
OUTPUT
1
2
CWEB
OUTPUT
VF
1.
VotingFarmTool. This document describes a class of C functions implementing a distributed software voting mechanism for EPX [11, 12] or similar message passing multi-threaded environments. Such a tool may be used for example, to set up a restoring organ [9] i.e., an NMR (i.e., N -module redundant) system with N voters. In order to describe the tool we start defining its basic building block, the voter. A voter is defined as a software module connected to one user module and to a farm of fellow voters arranged into a cliqu´e.
Node
User Module
Voter
Figure 1. A user module and its local voter. By means of the functions in the class the user module is able: • to create a static “picture” of the voting farm, needed for the set up of the cliqu´e; • to instantiate the local voter; • to send input or control messages to that voter.
§1
VF
VOTINGFARMTOOL
3
Node 1 i1
in
Voter
i1
User Module
Node n
User Module
in Voter
Figure 2. The architecture of the voting farm: each user module connects to one voter and interacts only with it. In particular, the user module sends its local voter only one input value; the voter then broadcasts it across the farm; then it receives N − 1 messages from its fellows so to be able to perform the voting. No interlocutor is needed other than the local voter. The other user modules are supposed to create coherent pictures and instances of voters on other nodes of the machine and to manage consistently the task of their local intermediary. All technicalities concerning the set up of the cliqu´e and the exchange of messages between the voters are completely transparent to the user module. More information about the voting farm may be found in [6, 7, 8]. In the following the basic functionalities of the VotingFarm class will be discussed, namely how to set up a “passive farm”, or a non-alive (in the sense of [4, 5]) topological representation of a yet-to-be-activated voting farm; how to initiate the voting farm; how to control the farm. h Global h Voting h Voting h Voting h Voting h Voting h Voting h Voting h Voting
Variables and # include’s Farm Declaration 4 i Farm Definition 6 i Farm Description 7 i Farm Activation 11 i Farm Control 14 i Farm Read 28 i Farm Destruction 27 i Farm Error Function 51 i
3i
4
VOTINGFARMTOOL
h Voting Algorithms 29 i h The Voter Function 30 i
VF
§1
§2 2.
VF
VOTINGFARMTOOL
5
Prologue: headers, global variables, etc.
#define #define #define #define #define #define #define
VF_MAX_NTS 16 VOTING_FARMS_MAX VF_MAX_INPUT_MSG VF_MAX_MSGS 10 VF_EVENT_TIMEOUT NO 0 YES 1
/∗ size of stacks ≡ max value for N ∗/ 64 /∗ max number of simultaneous active voting farms ∗/ 512 /∗ max size of an input message ∗/ /∗ max size of the message buffer ∗/ 10 /∗ Select time-out is 10 seconds ∗/
#define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define #define
E_VF_OVERFLOW −1 /∗ error conditions ∗/ E_VF_CANT_ALLOC −2 E_VF_UNDEFINED_VF −3 E_VF_WRONG_NODE −4 E_VF_GETGLOBID −5 E_VF_CANT_SPAWN −6 /∗ CreateThread error ∗/ E_VF_CANT_CONNECT −7 /∗ ConnectLink error ∗/ E_VF_RECVLINK −8 /∗ RecvLink error ∗/ E_VF_BROADCAST −9 /∗ Invalid input message - can’t broadcast ∗/ E_VF_DELIVER −10 /∗ Invalid output LinkCB t - can’t deliver ∗/ E_VF_BUSY_SLOT −11 /∗ Duplicated input message ∗/ E_VF_WRONG_VFID −12 E_VF_WRONG_DISTANCE −13 E_VF_INVALID_VF −14 E_VF_NO_LVOTER −15 /∗ exactly one voter is mandatorily needed ∗/ E_VF_TOO_MANY_LVOTERS −16 /∗ exactly one voter is mandatorily needed ∗/ E_VF_WRONG_MSG_NB −17 /∗ wrong number of messages ∗/ E_VF_SENDLINK −18 /∗ SendLink error ∗/ E_VF_INPUT_SIZE −19 /∗ inconsistency in the size of the input ∗/ E_VF_UNDESCRIBED −20 /∗ undescribed vf object ∗/ E_VF_INACTIVE −21 /∗ inactive vf object ∗/ E_VF_UNKNOWN_SENDER −22 /∗ inconsistency—sender unknown ∗/ E_VF_EVENT_TIMEOUT −23 /∗ a Select reached time-out ∗/ E_VF_SELECT −24 /∗ a Select returned an index out of range ∗/ E_VF_WRONG_ALGID −25 /∗ AlgorithmID out of range ∗/ E_VF_NULLPTR −26 /∗ A pointer parameter held Λ ∗/ E_VF_TOO_MANY −27 /∗ Too many opened voting farms ∗/
VF_ERROR_NB 28 /∗ number of errors, plus one ∗/ VF_MAX_FARMS 64 /∗ maximum number of farms available ∗/ /∗ voting algorithms ∗/ VFA_EXACT_CONCENSUS 0 VFA_MAJORITY 1 VFA_MEDIAN 2 VFA_PLURALITY 3 VFA_WEIGHTED_AVG 4 VFA_SIMPLE_MAJORITY 5 VFA_SIMPLE_AVERAGE 6 VF_SUCCESS 1 VF_FAILURE 0 VF_NB_ALGS 7 /∗ nb of algorithms plus one ∗/ /∗ default value for the ǫ threshold of formalized majority voting ∗/ #define VFD_EPSILON 5 · 10−5 #define VFP_INITIALISING 0 #define VFP_CONNECTING 1 #define #define #define #define #define #define #define #define #define #define #define #define
6
VOTINGFARMTOOL
#define #define #define #define #define
VFP_BROADCASTING 2 VFP_VOTING 3 VFP_WAITING 4 VFP_FAILED 5 VFP_QUITTING 6
VF
§2
§3
VF
VOTINGFARMTOOL
7
3. Two global variables have been supplied for the user to quer1y the error status of the application and the name of the function which experienced the error. Their definition constitutes the main part of the following section. h Global Variables and # include’s 3 i ≡ #include #include #include #include #include #include #include #include #include #ifdef SERVERNET #include "server.h" #endif /∗ SERVERNET ∗/ int VF error ; /∗ global variable for storing the error condition ∗/ static double ǫ = VFD_EPSILON; static double ScalingFactor = 1.0; int once = 1; static int VF RequestId (int, int, int); typedef struct { unsigned char ∗item ; int item nr ; } cluster t; /∗ A flag is attached to each object so that the object can be logically “deleted” from the list simply setting its status to NOT_PRESENT. Once the list is created, all its elements are labeled as PRESENT; as the execution goes by, elements are “logically” removed from the list changing their status to NOT_PRESENT. ∗/ typedef unsigned char flag; typedef struct { void ∗object ; flag status ; } value t; /∗ This part has been added in V1.5. It defines a set of (redefineable) symbolic constants representing upper limits for pre-allocated areas used exclusively in the static version of the tool. ∗/ #ifndef VF_STATIC_MAX_INP_MSG #define VF_STATIC_MAX_INP_MSG 64 #endif #ifndef VF_STATIC_MAX_LINK_NB #define VF_STATIC_MAX_LINK_NB 16 #endif #ifndef VF_STATIC_MAX_VOTER_INPUTS #define VF_STATIC_MAX_VOTER_INPUTS 20 #endif #ifdef STATIC #define AllocationClass static LinkCB t ∗ st links [VF_STATIC_MAX_LINK_NB]; Option tst options [VF_STATIC_MAX_LINK_NB]; /∗ used in VFA algorithms ∗/ double st VFA sum ; double st VFA weight [VF_STATIC_MAX_INP_MSG]; double st VFA squaredist [VF_STATIC_MAX_INP_MSG ∗ VF_STATIC_MAX_INP_MSG]; cluster t st clusters [VF_STATIC_MAX_VOTER_INPUTS]; void ∗st voter inputs [VF_STATIC_MAX_VOTER_INPUTS];
8
VOTINGFARMTOOL
char st chars [VF_STATIC_MAX_VOTER_INPUTS]; value t st VFA v [VF_STATIC_MAX_INP_MSG]; char st voter inputs data [VF_STATIC_MAX_VOTER_INPUTS][VF_STATIC_MAX_INP_MSG]; #else #define AllocationClass #endif unsigned char st VFA vote [VF_STATIC_MAX_INP_MSG]; This code is used in section 1.
VF
§3
§4
VF
VOTING FARM DECLARATION
9
4. Voting Farm Declaration. The whole VotingFarm class is built upon type Voting Farm t, which plays the same role as the type FILE in the standard class of C functions for file management: it offers the user a way to refer to some object from an abstract point of view, masking him/her from all unneeded information concerning its implementation. All a user needs to know is that, in order to use a voting farm, he/she has first to declare an object like follows: VotingFarm t ∗ vf ; The newly defined vf variable does not describe any valid voting farm yet; it is simply a pointer with no object attached to it, exactly the same way it goes for a FILE ∗fp variable which has not been fopen ’d yet. For that a special function is supplied: VF open , which is discussed in the next subsection. Each user module which needs to use a voting farm should declare a VotingFarm t ∗ variable. h Voting Farm Declaration 4 i ≡ typedef struct { int vf id ; int vf node stack [VF_MAX_NTS]; int vf ident stack [VF_MAX_NTS]; LinkCB t ∗ pipe [2]; int N ; int user thread ; int this voter ; double (∗distance )(void ∗, void ∗); flag broadcast done ; flag inp msg got ; flag destroy requested ; #ifdef SERVERNET RTC Thread t rtc ; #endif } VotingFarm t; static int VF voter (VotingFarm t ∗); int VF add (VotingFarm t ∗, int, int); void VF perror (void); #ifdef STATIC VotingFarm t Table [VF_MAX_FARMS]; #endif void ∗memdup (void ∗p, size t len ) { void ∗q = malloc (len ); if (q) memcpy (q, p, len ); return q; } This code is used in section 1.
5. In the above, vf id is a unique integer which identifies a voting context, vf node stack is a stack of node ids (a voter thread will be spawned on each of them), vf ident stack is a stack of thread id, N is a stack pointer (only one stack pointer is needed because the two stacks evolve in parallel), and user thread is the thread id of the caller (or user) module. pipe is a couple of pointers to LinkCB t, used to communicate between the user module and the local voter. this voter is the entry of the current voter.
10
VOTING FARM DEFINITION
6. Voting Farm Definition. e.g., as follows:
VF
§6
A voting farm variable vf can be defined by means of function VF open
(1) VotingFarm t ∗vf ; (2) vf = VF open (5, distance ); After this statement has been executed, an object has been allocated, some initializations have occurred, and the address of the newly created object has been returned into vf . The number the user supplies as the argument of VF open is an integer which univokely represents the current voting farm and in fact distinguishes it from all other voting farms which possibly will be used at the same times—we may call it a VotingFarm-id. distance is an arbitrary metric i.e., a function which gets two pointers to opaque objets, computes a “distance”, and returns this value as a positive real number. Each user module which needs to assemble the same voting farm should actually execute a VF open statement with the same number as an argument—it is the user’s responsibility to do like this. Likewise, coherent behaviour requires that the same metric function is referenced as second parameter of VF open . VF open returns Λ in case of error; otherwise, it returns a pointer to a valid object. h Voting Farm Definition 6 i ≡ VotingFarm t ∗VF open (int vf id , double (∗distance )(void ∗, void ∗)) { #ifdef STATIC static int vf max farms ; #endif AllocationClass VotingFarm t ∗vf ; static char ∗VFN = "VF_open"; if (vf id ≤ 0) { LogError (EC_ERROR, VFN, "Illegal VotingFarm Identifier (%d) −−− should be greater than 0.", vf id ); VF error = E_VF_WRONG_VFID; return Λ; } #ifndef STATIC if ((vf = (VotingFarm t ∗) malloc (sizeof (VotingFarm t))) ≡ Λ) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); VF error = E_VF_CANT_ALLOC; return Λ; } #else /∗ if STATIC is defined, then we fetch the next entry from array ‘Table ’ ∗/ if (vf max farms < VF_MAX_FARMS) vf = &Table [vf max farms ++ ]; else { LogError (EC_ERROR, VFN, "Too many farms."); VF error = E_VF_TOO_MANY; return Λ; } #endif if (distance ≡ Λ) { LogError (EC_ERROR, VFN, "Invalid Metric Function (NULL)."); VF error = E_VF_WRONG_DISTANCE; return Λ; } vf ~ N = 0; /∗ zero the stack pointer ∗/ vf ~ vf id = vf id ; /∗ record the vf id ∗/
§6
VF
VOTING FARM DEFINITION
vf ~ distance = distance ; /∗ record the function pointer ∗/ vf ~ user thread = vf ~ this voter = −1; LocalLink (vf ~ pipe ); /∗ Create a means for communicating with the local voter ∗/ return vf ; } This code is used in section 1.
11
12
VOTING FARM DESCRIPTION
VF
§7
7. Voting Farm Description. Once a VotingFarm t pointer has been created and once an object has been correctly defined and attached to that pointer, the user needs to describe the farm: how many voters are needed, where they should be placed, how to refer to each voter, and so on. This is accomplished by means of function VF add . If the voting farm consists of N voters, then the user shall call VF add N times; each call describes a voter by attaching a couple (n, t) to it, where n is the node of the voter and t is its thread identifier. As an example, the following statements: (1) (2) (3) (4) (5)
VotingFarm t ∗vf ; vf = VF open (5, distance ); VF add (vf , 15, tid1 ); VF add (vf , 21, tid2 ); VF add (vf , 4, tid5 );
declare (line (1)), define (line (2)), and describe (lines (3)–(5)) voting farm number 5. A triple of voters has been proposed; voter 0, identified by the couple (n, t) = (15, tid1 ), voter 1, or couple (21, tid2 ), and voter 2, or couple (4, tid5 ). Again it is the user’s responsibility to operate in a coherent, consistent way during these phases: in this case, he or she needs to declare the farm in exactly the same order, with exactly the same cardinality, with the same attributes on all nodes. Exactly one node-id has to be present and equal to the number of the current node. So far no thread has been launched, and no distributed action has taken place—therefore we talk of “passive voting farms” for farms that have been only declared, defined, and described, but not activated yet. VF add returns a negative integer in case of error; otherwise, it returns zero. h Voting Farm Description 7 i ≡ int VF add (VotingFarm t ∗vf , int node , int identifier ) { /∗ set function name ∗/ static int this node ; static char ∗VFN = "VF_add"; if (this node ≡ 0) this node = GET_ROOT( )~ ProcRoot~ MyProcID ; h Has vf been defined? 8 i h Is vf a valid object? 9 i vf ~ vf node stack [vf ~ N ] = node ; vf ~ vf ident stack [vf ~ N ] = identifier ; if (node ≡ this node ) if (vf ~ this voter < 0) vf ~ this voter = vf ~ N ; /∗ store the current value of the stack pointer ∗/ else { LogError (EC_ERROR, VFN, "There must be only one local voter."); return VF error = E_VF_TOO_MANY_LVOTERS; } vf ~ N ++ ; h Check stacks growth 10 i return VF error = 0; } This code is used in section 1.
§8 8.
VF
VOTING FARM DESCRIPTION
Checks if vf holds a valid (non-Λ) address.
h Has vf been defined? 8 i ≡ if (vf ≡ Λ) { LogError (EC_ERROR, VFN, "Undefined VotingFarm_t Object."); LogError (EC_ERROR, VFN, "\t(A VF_open is probably needed.)"); return VF error = E_VF_UNDEFINED_VF; } This code is used in sections 7, 11, 24, and 30.
9. Checks if vf points to valid data. h Is vf a valid object? 9 i ≡ if (vf ~ vf id < 0 ∨ vf ~ vf node stack ≡ Λ ∨ vf ~ vf ident stack ≡ Λ ∨ vf ~ N < 0) { LogError (EC_ERROR, VFN, "Corrupted or Invalid VotingFarm_t Object."); return VF error = E_VF_INVALID_VF; } This code is used in sections 7 and 30.
10.
Check if a stack overflow event has occurred.
h Check stacks growth 10 i ≡ if (vf ~ N ≥ VF_MAX_NTS) { LogError (EC_ERROR, VFN, "Stack Overflow."); LogError (EC_ERROR, VFN, "\t(Increase the value of VF_MAX_NTS; current value is %d.)", VF_MAX_NTS); return VF error = E_VF_OVERFLOW; } This code is used in section 7.
13
14
VOTING FARM ACTIVATION
VF
§11
11. Voting Farm Activation. After having described a voting farm, next step is turning that passive description into a “living” (active) object: this is accomplished by means of function VF run which simply spawns the local voter and connects to it. Any inconsistency like e.g., zero or two local voters are managed at this point and results in specific error messages. The one argument VotingFarm t ∗vf is passed to the newly created thread. VF run returns a negative integer in case of error; otherwise, it returns zero. h Voting Farm Activation 11 i ≡ int VF run (VotingFarm t ∗vf ){ AllocationClass int MyProcID ; AllocationClass GlobId tGlobId ; AllocationClass int Error ; #ifdef SERVERNET extern LinkCB t∗link2server ; #endif /∗ set function name ∗/ static char ∗VFN = "VF_run"; h Has vf been defined? 8 i h Has vf been described? 12 i if (vf ~ this voter < 0) { LogError (EC_ERROR, VFN, "No voter has been defined to be local."); return E_VF_NO_LVOTER; } MyProcId = GET_ROOT( )~ ProcRoot~ MyProcID ; /∗ Get the Global ID structure. ∗/ if (GetGlobId (&GlobId , Λ) ≡ −1) { LogError (EC_ERROR, VFN, "Cannot get the global ID of the thread."); return E_VF_GETGLOBID; } /∗ for the time being, this field is treated as a flag which tells whether VF run has been executed or not on voting farm vf . Its role will be different when FT Create Thread will be used instead of CreateThread . ∗/ vf ~ user thread = 0; /∗ first create a separate thread for the voter function ∗/ #ifdef SERVERNET LogError (EC_MESS, VFN, "Creating thread ‘VF_voter()’ via RTC_CreateLThread()"); vf ~ rtc = RTC CreateLThread (link2server , vf ~ vf ident stack [vf ~ this voter ], DIR_USER_TYPE, Λ, 0, (RTC ptr t )VF voter , vf , sizeof ( LinkCB t ∗ ) ) ; LogError (EC_MESS, VFN, "RTC_CreateLThread() has been executed."); #else if (CreateThread (Λ, 0, (int(∗)( ))VF voter , &Error , vf ) ≡ Λ) { LogError (EC_ERROR, VFN, "Cannot start a voter thread, error code %d.", Error ); return VF error = E_VF_CANT_SPAWN; } #endif /∗ SERVERNET ∗/ return VF error = 0; } This code is used in section 1.
§12 12.
VF
VOTING FARM ACTIVATION
This checks whether the farm has been described by means of at least one call to VF add .
h Has vf been described? 12 i ≡ if (vf ~ N < 0) { LogError (EC_ERROR, VFN, "Voting farm %d needs to be described.", vf ~ vf id ); LogError (EC_ERROR, VFN, "\t(You probably need to execute a VF_add statement.)"); return VF error = E_VF_UNDESCRIBED; } This code is used in sections 11 and 24.
13. This checks whether the farm has been activated by means of a previous call to function VF run . h Has vf been activated? 13 i ≡ if (vf ~ user thread < 0) { LogError (EC_ERROR, VFN, "Voting farm %d needs to be activated.", vf ~ vf id ); LogError (EC_ERROR, VFN, "\t(You probably need to execute a VF_run statement.)"); return VF error = E_VF_INACTIVE; } This code is used in section 24.
15
16
VOTING FARM CONTROL
VF
§14
14. Voting Farm Control. All interactions between the user module and the farm go through the VF control and VF control list functions and objects of VF msg t type, aka messages. h Voting Farm Control 14 i ≡ h Type VF msg t 15 i h Build a VF msg t message 17 i h Function VF control list 24 i h Function VF control 26 i h Function VF send 25 i This code is used in section 1.
15. A message is an object which holds the information needed for a user module to request a service to a voter and for a voter to respond to a previous user’s request. Its definition is simple: h Type VF msg t 15 i ≡ typedef struct { int code ; void ∗msg ; int msglen ; } VF msg t; This code is used in section 14.
16. The code field specifies the nature of the message (see table (see [table1]) for a complete reference); depending on this, some data may be pointed to by the opaque pointer msg . In this case, msglen represents the size of that data. This is an example of its usage: (1) (2) (3) (4)
VF msg t message ; message .code = VF_INP_MSG; message .msg = strdup (input ); message .msglen = 1 + strlen (input );
Note that the voter thread assumes that the user module allocates new memory for each new message .msg that is sent to it i.e., it won’t make a personal copy of the area; on the contrary, it will simply store the pointer and use the pointed storage. Moreover, also deallocation of objects previously defined by the user module will be considered to be managed by this latter. 17. These functions hide the VF msg t structure to the user. h Build a VF msg t message 17 i ≡ h Input message setup 18 i h Scaling factor message setup 19 i h Message to choose the algorithm 20 i This code is used in section 14.
§18 18.
VF
VOTING FARM CONTROL
17
Build up an VF msg t object holding a VF_INP_MSG message.
h Input message setup 18 i ≡ VF msg t ∗VFO Set Input Message (void ∗obj , size t siz ) { static VF msg t m; if (obj ≡ Λ) { VF error = E_VF_NULLPTR; return Λ; } m.code = VF_INP_MSG; m.msg = obj ; m.msglen = siz ; return &m; } This code is used in section 17.
19.
Build up an VF msg t object holding a VF_SCALING message.
h Scaling factor message setup 19 i ≡ VF msg t ∗VFO Set Scaling Factor (double ∗sf ) { static VF msg t m; m.code = VF_SCALING_FACTOR; m.msg = sf ; m.msglen = sizeof (double); return &m; } This code is used in section 17.
20. Build up an VF msg t object holding the chosen algorithm. h Message to choose the algorithm 20 i ≡ VF msg t ∗VFO Set Algorithm (int algorithm ) { static VF msg t m; m.code = VF_SELECT_ALG; m.msglen = algorithm ; return &m; } This code is used in section 17.
21. Function VF control list accepts an array of messages that are transferred across the communication network as one buffer. Function VF control list and VF control return a negative integer in case of error; otherwise, they return zero.
18
VOTING FARM CONTROL
VF
§22
22. List of possible message codes going: from the user module to its local voter (represented as U → L), and vice-versa (represented as U → L): • VF_INP_MSG: a msglen -byte-long input message is stored at the address referenced by the opaque pointer msg (U → L). • VF_OUT_LCB: the information pointed to by msg is the link control block for connecting the local voter to the output module (U → L); see Figure 3. • VF_SELECT_ALG: msg points to a code which identifies a particular majority voting algorithm out of the set of available algorithms (U → L). • VF_DESTROY: a signal meaning that the receiving voter should terminate itself after having freed all no more needed memory (U → L). • VF_NOP: No OPeration, something like an ImAlive signal. For the time being this event is not used. • VF_RESET: on the arrival of this segnal the status is set so to be able to perform a new voting session with the current farm (U → L). • VF_REFUSED: certain operations may be refused; for example, if one tries to execute a VF close ( ) before a broadcasting operation has been completed, then the voter returns this message to its user module (L → U). • VF_QUIT: before quitting, the voter generates a VF_QUIT event (L → U). • VF_DONE: after broadcasting, a VF_DONE event is raised (L → U). • VF_EPSILON: An ǫ threshold value needed by the formalized majority voting algorithm (U → L). • VF_ERROR: A generic error has occurred (L → U). • VF_SCALING_FACTOR: used in the Weighted Averaging Technique (see below) (U → L).
Output Node
OutputLink = ConnectLink(...
SendLink(OutputLink, vote, sizeof(vote))
User OutputLink Module
Voter
Figure 3. The user module connects to an output module (dashed curve) and then sends to its local voter a VF_OUT_LCB message holding the relevant link control. As soon as the voter has performed its voting task, it sends its vote to the output module by means of that link control block. #define #define #define #define #define
VF_INP_MSG 100 VF_OUT_LCB 101 VF_SELECT_ALG 102 VF_DESTROY 103 VF_NOP 104
§22
VF
#define #define #define #define #define #define #define
VOTING FARM CONTROL
19
VF_RESET 105 VF_REFUSED 106 VF_QUIT 107 VF_DONE 108 VF_EPSILON 109 VF_ERROR 110 VF_SCALING_FACTOR 111
23. List of possible message codes going from the local voter to its fellows and vice-versa. They have the same meaning of those defined in the previous section e.g., VF_V_INP_MSG is an input message coming from a fellow voter. #define #define #define #define #define
VF_V_INP_MSG 200 VF_V_DESTROY 203 VF_V_NOP 204 VF_V_RESET 205 VF_V_ERROR 210
20
VOTING FARM CONTROL
24.
This function sends one or more messages to the local voter of voting farm vf :
VF
§24
h Function VF control list 24 i ≡ int VF control list (VotingFarm t ∗vf , VF msg t ∗msg , int n) { AllocationClass int rv ; /∗ set function name ∗/ static char ∗VFN = "VF_control_list"; #ifdef VFDEBUG LogError (EC_MESS, VFN, "within control_list"); #endif /∗ VFDEBUG ∗/ h Has vf been defined? 8 i h Has vf been described? 12 i h Has vf been activated? 13 i #ifdef VFDEBUG LogError (EC_MESS, VFN, "checked: defined, described, and activated"); #endif /∗ VFDEBUG ∗/ if (n < 1 ∨ n > VF_MAX_MSGS) { /∗ doubt: should it be an error or a warning? ∗/ LogError (EC_ERROR, VFN, "Wrong message number (%d) −−− should be between 1 and %d.\n", n, VF_MAX_MSGS); return E_VF_WRONG_MSG_NB; } if (vf ~ pipe [0] 6= Λ) { #ifdef VFDEBUG LogError (EC_MESS, VFN, "Next statement is SendLink(%x, %x, %d)", vf ~ pipe [0], &msg [0], n ∗ sizeof (msg [0])); #endif /∗ VFDEBUG ∗/ rv = SendLink (vf ~ pipe [0], &msg [0], n ∗ sizeof (msg [0])); if (rv 6= n ∗ sizeof (msg [0])) { LogError (EC_ERROR, VFN, "Cannot send the message to local voter."); return E_VF_SENDLINK; } #ifdef VFDEBUG LogError (EC_MESS, VFN, "exiting..."); #endif /∗ VFDEBUG ∗/ return VF error = 0; } LogError (EC_ERROR, VFN, "Corrupted or Invalid VotingFarm_t Object."); return VF error = E_VF_INVALID_VF; } This code is used in section 14.
§25 25.
VF
VOTING FARM CONTROL
21
To be described...
h Function VF send 25 i ≡ #define VF_MAXARGS 31 void VF send (VotingFarm t ∗vf , int argc , . . . ) { AllocationClass va list ap ; AllocationClass VF msg t array [VF_MAXARGS]; AllocationClass VF msg t ∗mp ; AllocationClass int i; if (argc > VF_MAXARGS) argc = VF_MAXARGS; va start (ap , argc ); for (i = 0; i < argc ; i ++ ) { mp = va arg (ap , VF msg t ∗); memcpy (array + i, mp , sizeof (VF msg t)); } va end (ap ); VF control list (vf , array , argc ); } This code is used in section 14.
26. Simply a shortcut. h Function VF control 26 i ≡ int VF control (VotingFarm t ∗vf , VF msg t ∗msg ) { return VF control list (vf , msg , 1); } This code is used in section 14.
27. Voting Farm Destruction. This is another simple shortcut for sending the VF_DESTROY message to a voting farm. h Voting Farm Destruction 27 i ≡ int VF close (VotingFarm t ∗vf ) { AllocationClass VF msg t msg ; AllocationClass int r; msg .code = VF_DESTROY; r = VF control (vf , &msg ); return r; } This code is used in section 1.
22
VOTING FARM CONTROL
28.
to be described.
VF
h Voting Farm Read 28 i ≡ VF msg t ∗VF get (VotingFarm t ∗vf ) { static VF msg t msg ; AllocationClass int recv , n; AllocationClass Option to[5];
/∗ used to be Option to[2] ∗/
static char ∗VFN = "VF_get"; #ifdef VFDEBUG LogError (EC_MESS, VFN, " vf==%x, vf−>pipe==%x, vf−>pipe[0]==%x, vf−>pipe[1]==%x", vf , vf ~ pipe , vf ~ pipe [0], vf ~ pipe [1]); #endif /∗ VFDEBUG ∗/ o[0] = ReceiveOption (vf ~ pipe [0]); o[1] = TimeAfterOption (TimeNow ( ) + VF_EVENT_TIMEOUT ∗ CLOCK_TICK); switch ((n = SelectList (2, o))) { case 1: LogError (EC_ERROR, VFN, "Timeout condition reached."); LogError (EC_ERROR, VFN, "\t(Maybe VF_EVENT_TIMEOUT should be enlarged? Now it’s %d seconds.)", VF_EVENT_TIMEOUT); msg .code = VF_ERROR; msg .msglen = VF error = E_VF_EVENT_TIMEOUT; return &msg ; case 0: if ((recv = RecvLink (vf ~ pipe [0], (void ∗) &msg , sizeof (msg ))) ≤ 0) { LogError (EC_ERROR, VFN, "Can’t RecvLink [A message from the local voter]"); LogError (EC_ERROR, VFN, "\t(return code is %d.)", recv ); msg .code = VF_ERROR; msg .msglen = VF error = E_VF_RECVLINK; return &msg ; } break; default: LogError (EC_ERROR, VFN, "Can’t Select −−− retvalue is %d", n); msg .code = VF_ERROR; msg .msglen = VF error = E_VF_SELECT; return &msg ; } /∗ end switch ∗/ #ifdef VFDEBUG LogError (EC_MESS, VFN, "Received a message (%d bytes) from the voter.", recv ); #endif /∗ VFDEBUG ∗/ return &msg ; } static int voter sendcode (LinkCB t ∗ UserLink , int code ) { AllocationClass int rv ; AllocationClass VF msg t msg ; static char ∗VFN = "voter_sendcode"; msg .code = code ;
§28
§28
VF
VOTING FARM CONTROL
rv = SendLink (UserLink , &msg , sizeof (msg )); if (rv 6= sizeof (msg )) { LogError (EC_ERROR, VFN, "Cannot send the message to the user module."); return E_VF_SENDLINK; } return rv ; } static int voter sendmsg (LinkCB t ∗ UserLink , VF msg t ∗msg ) { AllocationClass int rv ; static char ∗VFN = "voter_sendmsg"; /∗ static function ⇒ VFN is not touched ∗/ rv = SendLink (UserLink , msg , sizeof (∗msg )); if (rv 6= sizeof (∗msg )) { LogError (EC_ERROR, VFN, "Cannot send the message to the user module."); return E_VF_SENDLINK; } return rv ; } This code is used in section 1.
29.
Voting functions are stored into the DoVoting array.
h Voting Algorithms 29 i ≡ typedef struct { unsigned char ∗vote ; int outcome ; } vote t; h Voting Functions 52 i typedef void (∗voter t)(VotingFarm t ∗, void ∗[ ], int, vote t ∗); static voter t DoVoting [VF_NB_ALGS] = {VFA ExactConcensus , VFA MajorityVoting , VFA MedianVoting , VFA PluralityVoting , VFA WeightedAveraging , VFA SimpleMajorityVoting , VFA SimpleAverage , }; This code is used in section 1.
23
24
THE VOTER FUNCTION
30.
The Voter Function.
VF
The function at the basis of the voting thread.
h The Voter Function 30 i ≡ static int VF voter (VotingFarm t ∗vf ) { LinkCB t ∗ UserLink , ∗OutputLink , ∗∗links ; #ifdef SERVERNET LinkCB t ∗ serverlink = ConnectServer ( ); #endif /∗ SERVERNET ∗/ Option t ∗ options ; int this voter , this node , N , i, opt , input nr ; VF msg t msg [VF_MAX_MSGS]; int msgnum ; VF msg t done ; int Algorithm = VFA_MAJORITY; char buffer [VF_MAX_INPUT_MSG + sizeof (int)]; int Error , recv ; int input length ; void ∗∗voter inputs ; unsigned char ∗vote = st VFA vote ; vote t rvote ; unsigned int t0 , tn ; /∗ set function name ∗/ static char ∗VFN = "VF_voter"; t0 = TimeNow ( ); #ifdef SERVERNET Set Phase (serverlink , VFP_INITIALISING); #endif input nr = input length = 0; UserLink = OutputLink = Λ; vf ~ broadcast done = vf ~ inp msg got = vf ~ destroy requested = NO; rvote .vote = vote ; h Has vf been defined? 8 i h Is vf a valid object? 9 i this voter = vf ~ this voter ; this node = vf ~ vf node stack [this voter ]; if (GET_ROOT( )~ ProcRoot~ MyProcID 6= this node ) { LogError (EC_ERROR, VFN, "Corrupted or Invalid VotingFarm_t Object."); return VF error = E_VF_INVALID_VF; } N = vf ~ N ; /∗ Connect to the local Agent ∗/ #ifdef SERVERNET h Ask the Server to set up a connection to an Agent 61 i #endif /∗ SERVERNET ∗/ /∗ Get the ( LinkCB t ∗ ) for communicating with the user module ∗/ UserLink = vf ~ pipe [1]; h Create a cliqu´e 32 i h Poll the user link, the farm links, and the server link } h An example of metric function This code is used in section 1.
31 i
37 i
§30
§31 31.
VF
THE VOTER FUNCTION
25
An example of metric function: a double-returning version of strcmp .
h An example of metric function 31 i ≡ double dstrcmp (void ∗a, void ∗b) { return (double) strcmp (a, b); } This code is used in section 30.
32. A cliqu´e (fully interconnected crossbar) is set up among the voters. This is done in two “flavours:” in the STATIC one we make use of predefined static data, otherwise we perform malloc ( )’s. h Create a cliqu´e 32 i ≡ #ifndef STATIC h Allocate an array of LinkCB t pointers 33 i #else h Initialize an array of LinkCB t pointers 34 i #endif /∗ STATIC ∗/ h Connect to your N -1 fellows 35 i This code is used in section 30.
33.
An array of Link Control Block Pointers is needed in order to realize the cliqu´e.
h Allocate an array of LinkCB t pointers 33 i ≡ links = ( LinkCB t ∗ ∗ ) malloc ( N ∗ sizeof ( LinkCB t ∗ ) ) ; options = ( Option t ∗ ) malloc ((N + 1) ∗ sizeof (Option t)); voter inputs = (void ∗∗) calloc (N , sizeof (void ∗)); if (links ≡ Λ ∨ options ≡ Λ ∨ voter inputs ≡ Λ) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); return VF error = E_VF_CANT_ALLOC; } This code is used in section 32.
34. This section is supplied in case the user is willing to use the static version of this tool. Rather than allocating memory, we link statically-allocated memory to the appropriate pointers and we initialize them— where needed. h Initialize an array of LinkCB t pointers 34 i ≡ links = st links ; options = st options ; voter inputs = st voter inputs ; memset (voter inputs , 0, VF_STATIC_MAX_INP_MSG); This code is used in section 32.
26
THE VOTER FUNCTION
VF
§35
35. “By the implementation of queues the calling order {to ConnectLink } does not matter.” [11]. As a consequence, a simple for should suffice to create the farm. N +1 options are “received”: N -1 for the cliqu´e set-up, 1 for connecting to the user module, 1 for connecting to the server; more precisely: • options [this voter ] regards the user module; • options [N ] regards the local server; • the rest regards the fellow voters. h Connect to your N -1 fellows 35 i ≡ #ifdef SERVERNET Set Phase (serverlink , VFP_CONNECTING); #endif for (i = 0; i < N ; i ++ ) { if (i 6= this voter ) { links [i] = ConnectLink (vf ~ vf node stack [i], VF RequestId (vf ~ vf id , i, this voter ), &Error ); options [i] = ReceiveOption (links [i]); if (links [i] ≡ Λ) { LogError (EC_ERROR, VFN, "Cannot connect to voter %d.", i); return VF error = E_VF_CANT_CONNECT; } } else { options [i] = ReceiveOption (UserLink ); } } #ifdef SERVERNET options [N ] = ReceiveOption (serverlink ); #endif /∗ SERVERNET ∗/ This code is used in section 32.
36.
This function creates a new request id beginning from voting farm id vfn and voter id’s v and w.
static int VF RequestId (int vfn , int v, int w) { AllocationClass int a, b; static int hundreds = VF_MAX_NTS ∗ VF_MAX_NTS; if (v > w) a = w, b = v; else a = v, b = w; return hundreds ∗ (vfn + 1) + a ∗ VF_MAX_NTS + b; }
§37
VF
THE VOTER FUNCTION
27
37. If control reaches this section it means that the cliqu´e has been successfully established between the voters. All future actions depend on the control and input messages of the user; therefore, the voter puts itself into an “endless” loop waiting for new messages to arrive and to be managed. This event-driven loop indeed resembles the one that may be found in any X11 client to manage interactions with the keyboard, the pointer, the display server, and so on. h Poll the user link, the farm links, and the server link 37 i ≡ /∗ t0 = TimeNow(); ∗/ while (1) { #ifdef SERVERNET opt = SelectList (N + 1, options ); #else opt = SelectList (N , options ); #endif /∗ SERVERNET ∗/ h A message from the user module 38 i else h A message from the cliqu´e 43 i #ifdef SERVERNET else h A message from the server module 46 i #endif /∗ SERVERNET ∗/ else { LogError (EC_ERROR, VFN, "Unknown sender\n"); return VF error = E_VF_UNKNOWN_SENDER; } if (input nr ≡ N ∧ vf ~ broadcast done ≡ YES ∧ once ) { FILE ∗f ; char fname [80]; tn = TimeNow ( ); #ifdef TIMESTATS sprintf (fname , "overhead.%d", this voter ); f = fopen (fname , "a+"); fprintf (f , "%u %u %lf\n", t0 , tn , (double)(tn − t0 )/CLOCK_TICK); fclose (f ); #endif once = 0; } } This code is used in section 30.
28
THE VOTER FUNCTION
38.
If the received options is option # this voter , then a message is coming from the user module.
VF
§38
h A message from the user module 38 i ≡ if (opt ≡ this voter ) { /∗ Receives the data and checks whether it is an integral multiple of the message size or not. In that latter case, an error is issued. ∗/ if ((msgnum = RecvLink (UserLink , msg , VF_MAX_MSGS ∗ sizeof (∗msg ))) % sizeof (∗msg )) { LogError (EC_ERROR, VFN, "Can’t RecvLink [A message from the user module]"); LogError (EC_ERROR, VFN, "size of the message is %d, sizeof(*msg) is %d, (1)%(2) is %d", msgnum , sizeof (∗msg ), msgnum % sizeof (∗msg )); return VF error = E_VF_RECVLINK; } msgnum /= sizeof (∗msg ); #ifdef VFDEBUG LogError (EC_MESS, VFN, " Received a msg from the user module ", this node ); LogError (EC_MESS, VFN, "(code==%d). msgnum==%d. Starting User msg management\n", msg [0].code , msgnum ); #endif { int i; h User message management } } This code is used in section 37.
39 i
§39
VF
THE VOTER FUNCTION
29
39. A message from the user has been received. Deal with that message. (Note that messages are buffered into the msg [ ] array.) h User message management 39 i ≡ for (i = 0; i < msgnum ; i ++ ) { void ∗memdup (void ∗, size t); #ifdef VFDEBUG LogError (EC_MESS, VFN, " : User message management: loop %d, code==%d\n", this voter , i, msg [i].code ); #endif switch (msg [i].code ) { case VF_INP_MSG: #ifdef VFDEBUG LogError (EC_MESS, VFN, " VF_INP_MSG received.\n", this node ); #endif voter inputs [this voter ] = memdup (msg [i].msg , msg [i].msglen ); input length = msg [i].msglen ; /∗ t0 = TimeNow(); ∗/ input nr ++ ; #ifdef VFDEBUG printf (" message (as a double) is %lf.\n", this node , ∗((double ∗) msg [i].msg )); #endif h Check for a complete message suite; if so, vote, and possibly deliver the outcome vf ~ inp msg got = YES; if (this voter ≡ input nr − 1) { int j; h Broadcast the Input Message vf ~ broadcast done = YES;
} break; h case VF_DESTROY:
41 i
48 i
40 i
/∗ break; unneeded, because the statement is unreachable ∗/
h case VF_OUT_LCB: 42 ibreak; case VF_SELECT_ALG: Algorithm = msg [i].msglen ; break; case VF_SCALING_FACTOR: ScalingFactor = ∗((double ∗) msg [i].msg ); break; case VF_EPSILON: ǫ = ∗((double ∗) msg [i].msg ); break; case VF_RESET: input nr = input length = 0; OutputLink = Λ, rvote .outcome = ∗vote = ’\0’; vf ~ broadcast done = vf ~ inp msg got = vf ~ destroy requested = NO; #ifndef STATIC { AllocationClass int i; for (i = 0; i < vf ~ N ; i ++ ) { free (voter inputs [i]); voter inputs [i] = Λ; } } #endif break;
30
THE VOTER FUNCTION
VF
§39
case VF_NOP: /∗ for the time being, nothing ∗/ break; default: printf (" : default case in switch: code==%d\n", this voter , msg [i].code ); break; } /∗ end switch ∗/ } /∗ end for ∗/ This code is used in section 38.
40. The actual transmission of this voter’s input message to all other voters in the farm. The buffer is built up so to tie the code of the message and the message itself. h Broadcast the Input Message 40 i ≡ ∗((int ∗) buffer ) = VF_V_INP_MSG; memcpy (buffer + sizeof (int), voter inputs [this voter ], input length ); #ifdef SERVERNET Set Phase (serverlink , VFP_BROADCASTING); #endif #ifdef ZEROPERM for (j = 0; j < N ; j ++ ) { if (j 6= this voter ) { #ifdef VFDEBUG LogError (EC_MESS, VFN, " broadcasting (j==%d)\n", this voter , j); #endif if (input length + sizeof (int) 6= SendLink (links [j], buffer , input length + sizeof (int))) { LogError (EC_ERROR, VFN, "Cannot SendLink to voter %d.", j); return VF error = E_VF_SENDLINK; } } } #else for (j = this voter + 1; j < N ; j ++ ) { if (input length + sizeof (int) 6= SendLink (links [j], buffer , input length + sizeof (int))) { LogError (EC_ERROR, VFN, "Cannot SendLink to voter %d.", j); return VF error = E_VF_SENDLINK; } } for (j = 0; j < this voter ; j ++ ) { if (input length + sizeof (int) 6= SendLink (links [j], buffer , input length + sizeof (int))) { LogError (EC_ERROR, VFN, "Cannot SendLink to voter %d.", j); return VF error = E_VF_SENDLINK; } } #endif This code is used in sections 39 and 45.
§41 41.
VF
THE VOTER FUNCTION
Management of a user message of type VF_DESTROY.
h case VF_DESTROY: 41 i ≡ case VF_DESTROY: /∗ ¡Broadcast a VF_V_DESTROY event¿ ?? ∗/ if (vf ~ broadcast done ≡ NO ∧ vf ~ N 6= 1) { voter sendcode (UserLink , VF_REFUSED); break; } else { #ifdef SERVERNET Set Phase (serverlink , VFP_QUITTING); #endif voter sendcode (UserLink , VF_QUIT); #ifdef SERVERNET { int myident = vf ~ vf ident stack [this voter ]; int error ; #ifdef DoBreakServer if ( ( error = BreakServer (serverlink ) ) 6= 0 ) LogError (EC_ERROR, VFN, "BreakServer failed, error:%d", error ) ; #endif /∗ DoBreakServer ∗/ #ifdef VFDEBUG LogError (EC_MESS, VFN, " myident=%d. Bye.\n", myident , this voter ); #endif } #endif /∗ SERVERNET ∗/ exit (0); return VF error = 0; } This code is used in section 39.
42.
Management of a user message of type VF_OUT_LCB.
h case VF_OUT_LCB: 42 i ≡ case VF_OUT_LCB: OutputLink = ( LinkCB t ∗ ) msg [i].msg ; if (OutputLink 6= Λ) { if (rvote .outcome ≡ VF_SUCCESS) { h Deliver the Outcome 50 i } } else { LogError (EC_ERROR, VFN, "Invalid output link control block − can’t deliver."); } /∗ In the event of a VF_DESTROY message, the local voter should inform its fellow via the broadcasting of a VF_V_DESTROY event. ¡Broadcast a VF_V_DESTROY event¿= ∗/ This code is used in section 39.
31
32
THE VOTER FUNCTION
VF
§43
43. If the received options is not option # N nor option # this voter , then a message is coming from a voter in the farm. Note that this time messages cannot be buffered—only one message at a time will be received. Moreover, messages come from a different memory space—for this reason, the structure of the message can’t be that of a VF msg t object. A different approach must be used: an integer representing the code message should be directly attached to an opaque area. The resulting buffer constitutes the message. h A message from the cliqu´e 43 i ≡ if (opt 6= this voter ∧ opt 6= N ) { if ((recv = RecvLink (links [opt ], buffer , VF_MAX_INPUT_MSG)) < 0) { LogError (EC_ERROR, VFN, "Can’t RecvLink [A message from the clique]"); LogError (EC_ERROR, VFN, "\t(Sender is voter %d, size of message is %d.)", opt , recv ); return VF error = E_VF_RECVLINK; } msg [0].code = ∗((int ∗) buffer ); msg [0].msg = buffer + sizeof (int); msg [0].msglen = recv − sizeof (int); h Cliqu´e message management 44 i } This code is used in section 37.
§44 44.
VF
THE VOTER FUNCTION
33
A new message has come from a voter in the cliqu´e.
h Cliqu´e message management 44 i ≡ switch (msg [0].code ) { case VF_V_INP_MSG: #ifdef VFDEBUG printf ("voter %d received the input message from voter %d: %lf\n", this voter , opt , ∗((double ∗) msg [0].msg )); #endif if (input length ≡ 0) input length = msg [0].msglen ; else { if (input length 6= msg [0].msglen ) { LogError (EC_ERROR, VFN, "Wrong input size"); return VF error = E_VF_INPUT_SIZE; } } #ifndef STATIC if ((voter inputs [opt ] = (void ∗) malloc (input length )) ≡ 0) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); return VF error = E_VF_CANT_ALLOC; } #else voter inputs [opt ] = (void ∗) &st voter inputs data [opt ][0]; #endif memcpy (voter inputs [opt ], msg [0].msg , input length ); input nr ++ ; h Check for a complete message suite; if so, vote, and possibly deliver the outcome 48 i h Check if it’s your turn to broadcast; if so, do it, and take note of that 45 i break; case VF_V_DESTROY: /∗ Is there a use for such a message? ∗/ /∗ for the time being, no action ∗/ #ifdef VFDEBUG LogError (EC_MESS, VFN, "voter %d received a VF_V_DESTROY message from voter %d\n", this voter , opt ); #endif break; case VF_V_ERROR: /∗ The voter notifies an error condition ∗/ /∗ for the time being, no action ∗/ break; case VF_V_RESET: /∗ Is there a use for such a message? ∗/ /∗ for the time being, no action ∗/ break; case VF_V_NOP: /∗ for the time being, no action ∗/ break; } /∗ end switch ∗/ This code is used in section 43.
34
THE VOTER FUNCTION
VF
§45
45. Broadcasting is performed in an ordered fashion so to prevent deadlocks—a voter is allowed to broadcast only when the following two conditions hold at once: • it has not performed a broadcast before, and • the vote-id (a number from 0 to vf ~ N -1) is less than or equal to the current number of input messages that have been received (user message included), minus one. h Check if it’s your turn to broadcast; if so, do it, and take note of that 45 i ≡ /∗ if (this voter ¡= input nr -1) ∗/ if (this voter ≡ input nr − 1) { if (vf ~ inp msg got ≡ YES ∧ vf ~ broadcast done ≡ NO) { int j; h Broadcast the Input Message 40 i vf ~ broadcast done = YES; } } This code is used in section 44.
46.
If the received options is option # N , then a message is coming from the local server module.
h A message from the server module 46 i ≡ if (opt ≡ N ) { int recv ; char msg [FTB_ELEMENT_SIZE]; if ((recv = RecvLink (serverlink , (void ∗) msg , FTB_ELEMENT_SIZE)) ≤ 0) LogError (EC_ERROR, VFN, "couldn’t receive a message from the server\n"); else LogError (EC_DEBUG, VFN, "got server message of %d bytes.\n", recv ); h Server message management 47 i } This code is used in section 37.
47. So far, an empty section. h Server message management 47 i ≡
/∗ should deal with message kept in msg[], size recv. ∗/
This code is used in section 46.
48. Every time a new message has come and consequently input nr has been incremented, we must check if it’s time for voting and if so, after voting, we check if we have an output address h Check for a complete message suite; if so, vote, and possibly deliver the outcome 48 i ≡ #ifdef VFDEBUG LogError (EC_MESS, VFN, " input_nr==%d, N==%d\n", this voter , input nr , N ); #endif if (input nr ≡ N ) { h Perform Voting 49 i if (OutputLink 6= Λ) { h Deliver the Outcome 50 i } /∗ input nr = 0; ∗/ } This code is used in sections 39 and 44.
§49
VF
THE VOTER FUNCTION
35
49. The actual voting algorithm is managed by a function whose address is kept in the DoVoting array at entry no. Algorithm . h Perform Voting 49 i ≡ if (Algorithm ≥ 0 ∧ Algorithm < VF_NB_ALGS) { #ifdef SERVERNET Set Phase (serverlink , VFP_VOTING); #endif DoVoting [Algorithm ](vf , (void ∗∗) voter inputs , input length , &rvote ); } else { LogError (EC_ERROR, VFN, "Wrong Algorithm number: %d, not in [0,%d[", Algorithm , VF_NB_ALGS); return VF error = E_VF_WRONG_ALGID; } #ifdef VFDEBUG if (rvote .outcome ≡ VF_SUCCESS) { LogError (EC_MESS, VFN, " is sending a VF_DONE msg to the user−−−vote == %lf.\n", this voter , ∗(double ∗) rvote .vote ); printf ("vote==%lf\n", ∗(double ∗) rvote .vote ); } else LogError (EC_MESS, VFN, "Vote is undefined."); #endif done .code = VF_DONE; /∗ possibly Λ, which means: “no unique vote is available” ∗/ done .msglen = rvote .outcome ; done .msg = rvote .vote ; recv = voter sendmsg (UserLink , &done ); #ifdef VFDEBUG LogError (EC_MESS, VFN, " sent a VF_DONE msg (%d bytes) to the user.\n", this voter , recv ); #endif This code is used in section 48.
36
THE VOTER FUNCTION
50.
The vote is sent to the output module.
h Deliver the Outcome 50 i ≡ if (rvote .outcome ≡ VF_SUCCESS) { if (input length 6= SendLink (OutputLink , vote , input length )) { LogError (EC_ERROR, VFN, "Cannot deliver the output."); return VF error = E_VF_DELIVER; } #ifdef SERVERNET Set Phase (serverlink , VFP_WAITING); #endif } else { char c = 0; if (SendLink (OutputLink , &c, 1) 6= 1) { LogError (EC_ERROR, VFN, "Cannot deliver the negative result."); return VF error = E_VF_DELIVER; } #ifdef SERVERNET Set Phase (serverlink , VFP_FAILED); #endif } This code is used in sections 42 and 48.
VF
§50
§51
VF
THE VOTER FUNCTION
37
51. VF perror : the perror function of the VotingFarm class. Statically defined as a vector of strings, its entries can be addressed as “−e”, where e is the error condition returned in VF error . The number of messages has been specified so to reduce the risk of inconsistencies. h Voting Farm Error Function 51 i ≡ static char ∗errors [VF_ERROR_NB] = {"no error", /∗ no error ∗/ "An internal stack has reached its upper limit", /∗ E_VF_OVERFLOW ∗/ "The system was not able to execute allocation", /∗ E_VF_CANT_ALLOC ∗/ "This operation requires a defined voting farm", /∗ E_VF_UNDEFINED_VF ∗/ "A wrong node has been specified", /∗ E_VF_WRONG_NODE ∗/ "The system was not able to get the global id", /∗ E_VF_GETGLOBID ∗/ "The system was not able to execute CreateThread", /∗ E_VF_CANT_SPAWN ∗/ "The system was not able to execute ConnectLink", /∗ ConnectLink error ∗/ "The system was not able to execute RecvLink", /∗ E_VF_RECVLINK ∗/ "The system was not able to perform broadcasting", /∗ E_VF_BROADCAST ∗/ "Invalid output (LinkCB_t*) − can’t deliver", /∗ E_VF_DELIVER ∗/ "Duplicated input message", /∗ E_VF_BUSY_SLOT ∗/ "Invalid voting farm id", /∗ E_VF_WRONG_VFID ∗/ "Invalid metric function pointer", /∗ E_VF_WRONG_DISTANCE ∗/ "Inconsistent voting farm object", /∗ E_VF_INVALID_VF ∗/ "No local voters−−−one voter has to be specified", /∗ E_VF_NO_LVOTER ∗/ "More than one local voter has been specified", /∗ E_VF_TOO_MANY_LVOTER ∗/ "A wrong number of messages has been specified", /∗ E_VF_WRONG_MSG_NB ∗/ "The system was not able to execute SendLink", /∗ E_VF_SENDLINK ∗/ "Inconsistency in the size of the input message", /∗ E_VF_INPUT_SIZE ∗/ "This operation requires a described voting farm", /∗ E_VF_UNDESCRIBED ∗/ "This operation requires an active voting farm", /∗ E_VF_INACTIVE ∗/ "Inconsistency − sender unknown", /∗ E_VF_UNKNOWN_SENDER ∗/ "Time−out reached during a Select()", /∗ E_VF_EVENT_TIMEOUT ∗/ "A Select() returned an index out of range", /∗ E_VF_SELECT ∗/ "Algorithm Id out of range", /∗ E_VF_WRONG_ALGID ∗/ "NULL in a call−by−reference pointer", /∗ E_VF_NULLPTR ∗/ "Maximun number of opened voting farms exceeded", /∗ E_VF_TOO_MANY ∗/ }; void VF perror (void) { static char ∗VFN = "VF_perror"; if (VF error ) { fprintf (stderr , "Error condition number %d raised while in function %s: \"%s\"\n", VF error , VFN, errors [−VF error ]); fflush (stderr ); } } This code is used in section 1.
38
THE VOTER FUNCTION
52.
The functions stored in the DoVoting array are defined here.
VF
§52
h Voting Functions 52 i ≡ h Exact Concensus 54 i h Majority Voting 55 i h Median Voting 56 i h Plurality Voting 57 i h Weighted Averaging 58 i h Simple Majority Voting 53 i h Simple Average 60 i This code is used in section 29.
53.
The simplest algorithm, apart from exact concensus—counts the agreement and returns the widest.
h Simple Majority Voting 53 i ≡ static void VFA SimpleMajorityVoting (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i, j; int n = vf ~ N ; int v[VF_MAX_NTS]; int threshold ;
}
threshold = n ≫ 1; /∗ n/2; ∗/ for (i = 0; i < n; i ++ ) v[i] = 0; for (i = 0; i < n; i ++ ) for (j = 0; j < n; j ++ ) if (i 6= j) { if (vf ~ distance (inp [i], inp [j]) < VFD_EPSILON) { v[i] ++ ; } } for (i = 0; i < n; i ++ ) if (v[i] ≥ threshold ) { vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , inp [i], len ); return; } vote~ outcome = VF_FAILURE;
This code is used in section 52.
§54 54.
VF
THE VOTER FUNCTION
Exact concensus means perfect, bitwise equality.
h Exact Concensus 54 i ≡ static void VFA ExactConcensus (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i; int n = vf ~ N ;
}
if (inp [0] ≡ Λ) { vote~ outcome = VF_FAILURE; return; } for (i = 1; i < n; i ++ ) { if (inp [i] ≡ Λ) { vote~ outcome = VF_FAILURE; return; } if (memcmp (inp [0], inp [i], len ) 6= 0) { vote~ outcome = VF_FAILURE; return; } } vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , inp [0], len );
This code is used in section 52.
39
40
GENERALIZED VOTERS
VF
§55
55. Generalized Voters. Several commonly used voting techniques have been generalized in [10] to “arbitrary N -version systems with arbitrary output types using a metric space framework”, including: • formalized majority voter (VFA_MAJORITY; cf. [10, §2.1, pp.445–446]), • generalized median voter (VFA_MEDIAN; cf. [10, §2.2, p.447]), • formalized plurality voter (VFA_PLURALITY; cf. [10, §2.3, pp.447–448]), and the • weighted averaging technique (VFA_WEIGHTED_AVG; cf. [10, §2.4, p.448]). All these techniques are based on the concept of “metric space” which is now recalled: A metric space is a couple (X, d), where X is the output space of the voting threads and d is a real value function defined on X × X which is able in some way to “compare” two objects belonging to X; more precisely, d behaves as a “distance” measure of any two objects in X. More formally, ∀(x, y, z) ∈ X 3 the following properties hold: 1. 2. 3. 4.
d(x, y) ≥ 0 (distances are positive numbers or zeroes); d(x, y) = 0 ⇒ x = y (different points have positive distances); d(x, y) = d(y, x) (distances obey the reflexive property); d(x, z) ≤ d(x, y) + d(y, z) (two consecutive segments are greater than the segment that straightly connects their loose ends, unless the three points lie on the same straight line;)
then d is called a “metric”. In other words, a metric is a function which is able to compare any two input objects and is able to numerically express a degree of “closeness” between them. [10] shows how four different voting algorithms can be executed starting from such a function. It is the user responsibility to supply a valid metric function on the call to VF open : that function shall get two pointers to opaque objets, compute a “distance”, and return that value as a positive real number. These functions take advantage of the Stack class which has been used in order to mimic the list operations in the algorithms in [10]. h Majority Voting 55 i ≡ static void VFA MajorityVoting (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i; int n = vf ~ N ; int v; cluster t ∗c; #ifndef STATIC c = calloc (n, sizeof (cluster t)); #else c = st clusters ; memset (c, 0, n ∗ sizeof (cluster t)); #endif h Create a partition of blocks which are maximal with respect to the metric property /∗ v is set by to the cardinality of the partition ∗/ for (i = 0; i < v; i ++ ) { if (c[i].item nr > n/2) { vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , c[i].item , len ); return; } } vote~ outcome = VF_FAILURE; } This code is used in section 52.
59 i
§56
VF
GENERALIZED VOTERS
41
56. “Generalized Median Voter”, §2.2 of [10, p.447]. See also the “mid-value select” technique in [9, p.60] #define PRESENT 1 #define NOT_PRESENT 0 h Median Voting 56 i ≡ static void VFA MedianVoting (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { void ∗inputs [VF_MAX_NTS]; value t ∗v; int i, j; int ri , rj ; double max , dist ; int n, card ; #ifndef STATIC static char ∗VFN = "VFA/MedianVoting"; #endif #ifdef STATIC v = st VFA v ; #else v = (value t ∗) malloc (len ∗ sizeof (value t)); if (v ≡ Λ) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); VF error = E_VF_CANT_ALLOC; return; } #endif /∗ STATIC ∗/ for (n = vf ~ N , i = 0; i < n; i ++ ) { v[i].object = inp [i]; v[i].status = PRESENT; } ri = rj = 0; do { for (card = i = 0, max = −1.0; i < n; i ++ ) { if (v[i].status ≡ PRESENT) { inputs [card ++ ] = v[i].object ; for (j = i + 1; j < n; j ++ ) { if (v[j].status ≡ PRESENT) if ((dist = (vf ~ distance )(v[i].object , v[j].object )) ≥ max ) { max = dist ; ri = i; rj = j; } } } } if (max 6= −1.0) { v[ri ].status = v[rj ].status = NOT_PRESENT; } } while (card > 2); vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , inputs [0], len ); } This code is used in section 52.
42
GENERALIZED VOTERS
57.
“Formalized Plurality Voter”, §2.3 of [10, p.447].
h Plurality Voting 57 i ≡ static void VFA PluralityVoting (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i, j; int n = vf ~ N ; int v; int max ; cluster t ∗c; #ifndef STATIC c = calloc (n, sizeof (cluster t)); #else c = st clusters ; memset (c, 0, n ∗ sizeof (cluster t)); #endif if (c ≡ Λ) LogError (EC_MESS, "Plurality", "c is NULL"); h Create a partition of blocks which are maximal with respect to the metric property 59 i #ifdef VFDEBUG LogError (EC_MESS, "Plurality", "Partition has been created."); #endif j = −1; for (max = i = 0; i < v; i ++ ) { if (c[i].item nr > max ) j = i, max = c[i].item nr ; } #ifdef VFDEBUG LogError (EC_MESS, "Plurality", "Max computed."); #endif if (max > 1) { vote~ outcome = VF_SUCCESS; /∗ memcpy(vote-¿vote, c[i].item, len); ∗/ memcpy (vote~ vote , c[j].item , len ); } else vote~ outcome = VF_FAILURE; #ifndef STATIC free (c); #endif } This code is used in section 52.
VF
§57
§58
VF
GENERALIZED VOTERS
43
58. “Weighted Averaging Technique”, §2.4 of [10, p.448]. The ScalingFactor variable is used for computing a set of “weights”, defined as follows: given n values, x1 , x2 , . . . , xn , then ∀i ∈ {1, 2, . . . , n} : wi = 1 +
Qn
j=1,j6=i
d2 (xj , xi ) −1
a
Pn where a is to ScalingFactor and d is the metric. Considered S = i=1 wi , the voted value is computed P Pequal n n wi xi wi i=1 i=1 . as x = x which is of course here computed as i S S h Weighted Averaging 58 i ≡ static void VFA WeightedAveraging (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i, j; int n = vf ~ N ; double ∗sum ; double ∗weight , wsum ; double ∗squaredist ; double partial , f ; int r, c; static char ∗VFN = "WeightedAveraging"; #ifdef STATIC sum = &st VFA sum ; weight = st VFA weight ; squaredist = st VFA squaredist ; #else sum = malloc (sizeof (double)); weight = (double ∗) malloc (n ∗ sizeof (double)); squaredist = (double ∗) malloc (n ∗ n ∗ sizeof (double)); #endif /∗ STATIC ∗/ if (sum ≡ Λ ∨ weight ≡ Λ ∨ squaredist ≡ Λ) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); VF error = E_VF_CANT_ALLOC; return; } if (ScalingFactor ≡ 0) { LogError (EC_MESS, VFN, "Illegal scaling factor −−− set to 1"); ScalingFactor = 1.0; } /∗ compute the distances ∗/ for (i = 0; i < n; i ++ ) for (j = 0; j < i; j ++ ) { f = (vf ~ distance )(inp [i], inp [j]); squaredist [i ∗ n + j] = f ∗ f ; } for (wsum = 0.0, i = 0; i < n; i ++ ) { partial = 1.0; for (j = 0; j < n ∧ j 6= i; j ++ ) { if (i < j) r = j, c = i; else r = i, c = j; partial ∗= squaredist [r ∗ n + c]; } partial /= (ScalingFactor ∗ ScalingFactor ); wsum += weight [i] = 1.0/(1.0 + partial );
44
GENERALIZED VOTERS
} for (∗sum = 0.0, i = 0; i < n; i ++ ) { ∗sum += (∗(double ∗) inp [i]) ∗ weight [i]; } if (wsum 6= 0) { ∗sum /= wsum ; vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , sum , len ); } else vote~ outcome = VF_FAILURE; #ifndef STATIC free (sum ); free (weight ); free (squaredist ); #endif } This code is used in section 52.
VF
§58
§59
VF
GENERALIZED VOTERS
45
59. The input values are partitioned into a set of blocks, V1 , V2 , . . . , Vn , such that for each i block Vi is maximal with respect to the property that ∀(x, y) ∈ Vi × Vi : d(x, y) ≤ ǫ, where d is the metric. In order to reproduce as much as possible the algorithmic formalism of [10] we decided • to mimic their Lisp-like statements with stacks, and • to use goto statements. In this way actions (1)–(6) in [10, p. 445–446] can be (more or less) mapped into the statements corresponding to labels one to six that follow. h Create a partition of blocks which are maximal with respect to the metric property { char vt ; char ∗del ; void ∗item ; int i, j, item nr ; vt = GET_ROOT( )~ ProcRoot~ MyProcID ; #ifndef STATIC del = calloc (n, 1); /∗ alloc + set all of them to NO ∗/ #else del = st chars ; memset (del , 0, n); #endif for (v = i = 0; i < n; i ++ ) { if (del [i]) continue; item = inp [i]; del [i] = YES; c[v].item = item ; for (item nr = 1, j = i + 1; j < n; j ++ ) { if (¬del [j] ∧ vf ~ distance (item , inp [j]) < ǫ) { del [j] = YES; item nr ++ ; } } c[v].item nr = item nr ; v ++ ; } #ifndef STATIC free (del ); #endif } This code is used in sections 55 and 57.
59 i
≡
46
GENERALIZED VOTERS
VF
§60
60. Simple Averaging may be useful e.g., to “melt together” n sample values of a same pixel of an image. Of course it requires the objects are numbers. For the time being, the computation is performed in double precision floating point arithmetics. A problem of this technique is that it assumes the samples are not faulty, in the sense that they do not differ too much from each other: an enormously different addendum would cause the average to differ as well from the “correct” values. The weighted averaging technique is a partial solution to this. h Simple Average 60 i ≡ static void VFA SimpleAverage (VotingFarm t ∗vf , void ∗inp [ ], int len , vote t ∗vote ) { int i; int n = vf ~ N ; double ∗sum ; static char ∗VFN = "SimpleAverage"; #ifdef STATIC sum = &st VFA sum ; #else sum = malloc (sizeof (double)); if (sum ≡ Λ) { LogError (EC_ERROR, VFN, "Memory Allocation Error."); VF error = E_VF_CANT_ALLOC; return; } #endif /∗ STATIC ∗/ for (∗sum = 0.0, i = 0; i < n; i ++ ) { ∗sum += (∗(double ∗) inp [i]); } if (n ≡ 0) { LogError (EC_ERROR, VFN, "Inconsistency−−−farm cardinality should be zero."); VF error = E_VF_INVALID_VF; return; } ∗sum /= n; vote~ outcome = VF_SUCCESS; memcpy (vote~ vote , sum , len ); #ifndef STATIC free (sum ); #endif /∗ STATIC ∗/ } This code is used in section 52.
61. This means: • send a message to the server telling it “connect me to the Agent (thread id 1)” • do a ConnectLink with the Agent • send a set up message to the Server so that it propagates that message to the Agent. h Ask the Server to set up a connection to an Agent This code is used in section 30.
61 i
≡
/∗ yet to be implemented ∗/
§62
VF
CLOSINGS
47
62. Closings. This document and source code describes the actual implementation of the Voting Farm Tool as it appears in the EFTOS [1, 2] Basic Functionality Set Library. It has been crafted by means of the CWEB system of structured documentation [3].
48
INDEX
VF
§63
63. Index. Here is a list of the identifiers used, and where they appear. Underlined entries indicate the place of definition. Error messages are also shown. a: 31, 36. algorithm : 20. Algorithm : 30, 39, 49. AllocationClass : 3, 6, 11, 24, 25, 27, 28, 36, 39. ap : 25. argc : 25. array : 25. b: 31, 36. BreakServer : 41. broadcast done : 4, 30, 37, 39, 41, 45. buffer : 30, 40, 43. c: 50, 55, 57, 58. calloc : 33, 55, 57, 59. card : 56. CLOCK_TICK: 28, 37. cluster t: 3, 55, 57. code : 15, 16, 18, 19, 20, 27, 28, 38, 39, 43, 44, 49. ConnectLink : 2, 35, 51, 61. ConnectServer : 30. CreateThread : 2, 11. del : 59. destroy requested : 4, 30, 39. DIR_USER_TYPE: 11. dist : 56. distance : 4, 6, 7, 53, 56, 58, 59. DoBreakServer : 41. done : 30, 49. DoVoting : 29, 49, 52. dstrcmp : 31. E_VF_BROADCAST: 2, 51. E_VF_BUSY_SLOT: 2, 51. E_VF_CANT_ALLOC: 2, 6, 33, 44, 51, 56, 58, 60. E_VF_CANT_CONNECT: 2, 35. E_VF_CANT_SPAWN: 2, 11, 51. E_VF_DELIVER: 2, 50, 51. E_VF_EVENT_TIMEOUT: 2, 28, 51. E_VF_GETGLOBID: 2, 11, 51. E_VF_INACTIVE: 2, 13, 51. E_VF_INPUT_SIZE: 2, 44, 51. E_VF_INVALID_VF: 2, 9, 24, 30, 51, 60. E_VF_NO_LVOTER: 2, 11, 51. E_VF_NULLPTR: 2, 18, 51. E_VF_OVERFLOW: 2, 10, 51. E_VF_RECVLINK: 2, 28, 38, 43, 51. E_VF_SELECT: 2, 28, 51. E_VF_SENDLINK: 2, 24, 28, 40, 51. E_VF_TOO_MANY: 2, 6, 51. E_VF_TOO_MANY_LVOTER: 51. E_VF_TOO_MANY_LVOTERS: 2, 7. E_VF_UNDEFINED_VF: 2, 8, 51.
E_VF_UNDESCRIBED: 2, 12, 51. E_VF_UNKNOWN_SENDER: 2, 37, 51. E_VF_WRONG_ALGID: 2, 49, 51. E_VF_WRONG_DISTANCE: 2, 6, 51. E_VF_WRONG_MSG_NB: 2, 24, 51. E_VF_WRONG_NODE: 2, 51. E_VF_WRONG_VFID: 2, 6, 51. EC_DEBUG: 46. EC_ERROR: 6, 7, 8, 9, 10, 11, 12, 13, 24, 28, 30, 33, 35, 37, 38, 40, 41, 42, 43, 44, 46, 49, 50, 56, 58, 60. EC_MESS: 11, 24, 28, 38, 39, 40, 41, 44, 48, 49, 57, 58. ǫ: 3, 39, 59. Error : 11, 30, 35. errors : 51. exit : 41. f : 37, 58. fclose : 37. fflush : 51. flag: 3, 4. fname : 37. fopen : 4, 37. fp : 4. fprintf : 37, 51. free : 39, 57, 58, 59, 60. FT Create Thread : 11. FTB_ELEMENT_SIZE: 46. GET_ROOT: 7, 11, 30, 59. GetGlobId : 11. GlobId : 11. GlobId t: 11. hundreds : 36. i: 25, 30, 38, 39, 53, 54, 55, 56, 57, 58, 59, 60. identifier : 7. inp : 53, 54, 55, 56, 57, 58, 59, 60. inp msg got : 4, 30, 39, 45. input : 16. input length : 30, 39, 40, 44, 49, 50. input nr : 30, 37, 39, 44, 45, 48. inputs : 56. item : 3, 55, 57, 59. item nr : 3, 55, 57, 59. j: 39, 45, 53, 56, 57, 58, 59. len : 4, 53, 54, 55, 56, 57, 58, 60. LinkCB t: 2, 3, 4, 5, 11, 28, 30, 33, 42. links : 30, 33, 34, 35, 40, 43. link2server : 11. LocalLink : 6.
§63
VF
LogError : 6, 7, 8, 9, 10, 11, 12, 13, 24, 28, 30, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 46, 48, 49, 50, 56, 57, 58, 60. m: 18, 19, 20. malloc : 4, 6, 32, 33, 44, 56, 58, 60. max : 56, 57. memcmp : 54. memcpy : 4, 25, 40, 44, 53, 54, 55, 56, 57, 58, 60. memdup : 4, 39. memset : 34, 55, 57, 59. message : 16. mp : 25. msg : 15, 16, 18, 19, 22, 24, 26, 27, 28, 30, 38, 39, 42, 43, 44, 46, 49. msglen : 15, 16, 18, 19, 20, 22, 28, 39, 43, 44, 49. msgnum : 30, 38, 39. myident : 41. MyProcId : 11. MyProcID : 7, 11, 30, 59. N : 4, 30. n: 24, 28, 53, 54, 55, 56, 57, 58, 60. NO: 2, 30, 39, 41, 45. node : 7. NOT_PRESENT: 3, 56. obj : 18. object : 3, 56. once : 3, 37. one : 59. opt : 30, 37, 38, 43, 44, 46. Option t: 3, 28, 30, 33. options : 30, 33, 34, 35, 37. outcome : 29, 39, 42, 49, 50, 53, 54, 55, 56, 57, 58, 60. OutputLink : 30, 39, 42, 48, 50. p: 4. partial : 58. perror : 51. pipe : 4, 5, 6, 24, 28, 30. PRESENT: 3, 56. printf : 39, 44, 49. ProcRoot : 7, 11, 30, 59. q: 4. r: 27, 58. ReceiveOption : 28, 35. recv : 28, 30, 43, 46, 49. RecvLink : 2, 28, 38, 43, 46. ri : 56. rj : 56. rtc : 4, 11. RTC CreateLThread : 11. RTC ptr t : 11. RTC Thread t : 4.
INDEX
49
rv : 24, 28. rvote : 30, 39, 42, 49, 50. ScalingFactor : 3, 39, 58. Select : 2. SelectList : 28, 37. SendLink : 2, 24, 28, 40, 50. serverlink : 30, 35, 40, 41, 46, 49, 50. SERVERNET: 3, 4, 11, 30, 35, 37, 40, 41, 49, 50. Set Phase : 30, 35, 40, 41, 49, 50. sf : 19. six : 59. siz : 18. sprintf : 37. squaredist : 58. st chars : 3, 59. st clusters : 3, 55, 57. st links : 3, 34. st options : 3, 34. st VFA squaredist : 3, 58. st VFA sum : 3, 58, 60. st VFA v : 3, 56. st VFA vote : 3, 30. st VFA weight : 3, 58. st voter inputs : 3, 34. st voter inputs data : 3, 44. Stack : 55. STATIC: 3, 4, 6, 32, 39, 44, 55, 56, 57, 58, 59, 60. status : 3, 56. stderr : 51. strcmp : 31. strdup : 16. strlen : 16. sum : 58, 60. Table : 4, 6. this node : 7, 30, 38, 39. this voter : 4, 5, 6, 7, 11, 30, 35, 37, 38, 39, 40, 41, 43, 44, 45, 48, 49. threshold : 53. tid1 : 7. tid2 : 7. tid5 : 7. TimeAfterOption : 28. TimeNow : 28, 30, 37. TIMESTATS: 37. tn : 30, 37. t0 : 30, 37. user thread : 4, 5, 6, 11, 13. UserLink : 28, 30, 35, 38, 41, 49. v: 36, 53, 55, 56, 57. va arg : 25. va end : 25. va start : 25.
50
INDEX
value t: 3, 56. vf : 2, 4, 6, 7, 8, 9, 10, 11, 12, 13, 24, 25, 26, 27, 28, 30, 35, 37, 39, 41, 45, 49, 53, 54, 55, 56, 57, 58, 59, 60. VF add : 4, 7, 12. VF close : 22, 27. VF control : 14, 21, 26, 27. VF control list : 14, 21, 24, 25, 26. VF_DESTROY: 22, 27, 41, 42. VF_DONE: 22, 49. VF_EPSILON: 22, 39. VF error : 3, 6, 7, 8, 9, 10, 11, 12, 13, 18, 24, 28, 30, 33, 35, 37, 38, 40, 41, 43, 44, 49, 50, 51, 56, 58, 60. VF_ERROR: 22, 28. VF_ERROR_NB: 2, 51. VF_EVENT_TIMEOUT: 2, 28. VF_FAILURE: 2, 53, 54, 55, 57, 58. VF get : 28. vf id : 4, 5, 6, 9, 12, 13, 35. vf ident stack : 4, 5, 7, 9, 11, 41. VF_INP_MSG: 16, 18, 22, 39. vf max farms : 6. VF_MAX_FARMS: 2, 4, 6. VF_MAX_INPUT_MSG: 2, 30, 43. VF_MAX_MSGS: 2, 24, 30, 38. VF_MAX_NTS: 2, 4, 10, 36, 53, 56. VF_MAXARGS: 25. VF msg t: 14, 15, 16, 17, 18, 19, 20, 24, 25, 26, 27, 28, 30, 43. VF_NB_ALGS: 2, 29, 49. vf node stack : 4, 5, 7, 9, 30, 35. VF_NOP: 22, 39. VF open : 4, 6, 7, 55. VF_OUT_LCB: 22, 42. VF perror : 4, 51. VF_QUIT: 22, 41. VF_REFUSED: 22, 41. VF RequestId : 3, 35, 36. VF_RESET: 22, 39. VF run : 11, 13. VF_SCALING: 19. VF_SCALING_FACTOR: 19, 22, 39. VF_SELECT_ALG: 20, 22, 39. VF send : 25. VF_STATIC_MAX_INP_MSG: 3, 34. VF_STATIC_MAX_LINK_NB: 3. VF_STATIC_MAX_VOTER_INPUTS: 3. VF_SUCCESS: 2, 42, 49, 50, 53, 54, 55, 56, 57, 58, 60. VF_V_DESTROY: 23, 41, 42, 44. VF_V_ERROR: 23, 44.
VF
§63
VF_V_INP_MSG: 23, 40, 44. VF_V_NOP: 23, 44. VF_V_RESET: 23, 44. VF voter : 4, 11, 30. VFA_EXACT_CONCENSUS: 2. VFA ExactConcensus : 29, 54. VFA_MAJORITY: 2, 30, 55. VFA MajorityVoting : 29, 55. VFA_MEDIAN: 2, 55. VFA MedianVoting : 29, 56. VFA_PLURALITY: 2, 55. VFA PluralityVoting : 29, 57. VFA_SIMPLE_AVERAGE: 2. VFA_SIMPLE_MAJORITY: 2. VFA SimpleAverage : 29, 60. VFA SimpleMajorityVoting : 29, 53. VFA_WEIGHTED_AVG: 2, 55. VFA WeightedAveraging : 29, 58. VFD_EPSILON: 2, 3, 53. VFDEBUG: 24, 28, 38, 39, 40, 41, 44, 48, 49, 57. VFN: 6, 7, 8, 9, 10, 11, 12, 13, 24, 28, 30, 33, 35, 37, 38, 39, 40, 41, 42, 43, 44, 46, 48, 49, 50, 51, 56, 58, 60. vfn : 36. VFO Set Algorithm : 20. VFO Set Input Message : 18. VFO Set Scaling Factor : 19. VFP_BROADCASTING: 2, 40. VFP_CONNECTING: 2, 35. VFP_FAILED: 2, 50. VFP_INITIALISING: 2, 30. VFP_QUITTING: 2, 41. VFP_VOTING: 2, 49. VFP_WAITING: 2, 50. vote : 29, 30, 39, 49, 50, 53, 54, 55, 56, 57, 58, 60. vote t: 29, 30, 53, 54, 55, 56, 57, 58, 60. voter inputs : 30, 33, 34, 39, 40, 44, 49. voter sendcode : 28, 41. voter sendmsg : 28, 49. voter t: 29. VOTING_FARMS_MAX: 2. VotingFarm : 4. VotingFarm t: 4, 6, 7, 11, 24, 25, 26, 27, 28, 29, 30, 53, 54, 55, 56, 57, 58, 60. vt : 59. w: 36. weight : 58. wsum : 58. YES: 2, 37, 39, 45, 59. ZEROPERM: 40.
VF
NAMES OF THE SECTIONS
h A message from the cliqu´e 43 i Used in section 37. h A message from the server module 46 i Used in section 37. h A message from the user module 38 i Used in section 37. h Allocate an array of LinkCB t pointers 33 i Used in section 32. h An example of metric function 31 i Used in section 30. h Ask the Server to set up a connection to an Agent 61 i Used in section 30. h Broadcast the Input Message 40 i Used in sections 39 and 45. h Build a VF msg t message 17 i Used in section 14. h Check for a complete message suite; if so, vote, and possibly deliver the outcome
48 i
51
Used in sections 39
and 44.
h Check if it’s your turn to broadcast; if so, do it, and take note of that 45 i Used in section 44. h Check stacks growth 10 i Used in section 7. h Cliqu´e message management 44 i Used in section 43. h Connect to your N -1 fellows 35 i Used in section 32. h Create a cliqu´e 32 i Used in section 30. h Create a partition of blocks which are maximal with respect to the metric property 59 i Used in sections 55 and 57.
h Deliver the Outcome 50 i Used in sections 42 and 48. h Exact Concensus 54 i Used in section 52. h Function VF control list 24 i Used in section 14. h Function VF control 26 i Used in section 14. h Function VF send 25 i Used in section 14. h Global Variables and # include’s 3 i Used in section 1. h Has vf been activated? 13 i Used in section 24. h Has vf been defined? 8 i Used in sections 7, 11, 24, and 30. h Has vf been described? 12 i Used in sections 11 and 24. h Initialize an array of LinkCB t pointers 34 i Used in section 32. h Input message setup 18 i Used in section 17. h Is vf a valid object? 9 i Used in sections 7 and 30. h Majority Voting 55 i Used in section 52. h Median Voting 56 i Used in section 52. h Message to choose the algorithm 20 i Used in section 17. h Perform Voting 49 i Used in section 48. h Plurality Voting 57 i Used in section 52. h Poll the user link, the farm links, and the server link 37 i Used in section 30. h Scaling factor message setup 19 i Used in section 17. h Server message management 47 i Used in section 46. h Simple Average 60 i Used in section 52. h Simple Majority Voting 53 i Used in section 52. h The Voter Function 30 i Used in section 1. h Type VF msg t 15 i Used in section 14. h User message management 39 i Used in section 38. h Voting Algorithms 29 i Used in section 1. h Voting Farm Activation 11 i Used in section 1. h Voting Farm Control 14 i Used in section 1. h Voting Farm Declaration 4 i Used in section 1. h Voting Farm Definition 6 i Used in section 1. h Voting Farm Description 7 i Used in section 1. h Voting Farm Destruction 27 i Used in section 1. h Voting Farm Error Function 51 i Used in section 1. h Voting Farm Read 28 i Used in section 1. h Voting Functions 52 i Used in section 29.
52
NAMES OF THE SECTIONS
h Weighted Averaging 58 i Used in section 52. h case VF_DESTROY: 41 i Used in section 39. h case VF_OUT_LCB: 42 i Used in section 39.
VF
VF
Section Page VotingFarmTool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 Voting Farm Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 9 Voting Farm Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 10 Voting Farm Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 12 Voting Farm Activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 14 Voting Farm Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 16 The Voter Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 24 Generalized Voters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 40 Closings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 47 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 48
References [1] [2]
[3] [4] [5] [6]
[7] [8] [9] [10] [11] [12]
EFTOS K.U.Leuven: The EFTOS Reference Guide and Cookbook . (EFTOS Deliverable 2.4.2, March 1997) Deconinck, G., and De Florio, V., and Lauwereins, R., and Varvarigou, T: EFTOS: A software framework for more dependable embedded HPC applications, accepted for presentation at the European Conf. in Parallel Processing (Euro-Par ’97). Knuth, D.E.: Literate Programming (Center for the Study of the Language and Information, Leland Standard Junior University, 1992) Carriero, N., and Gelernter, D.: How to write parallel programs: a guide to the perplexed. ACM Comp. Surv. 21 (1989): 323–357. Carriero, N., and Gelernter, D.: LINDA in context. Comm. ACM 32 (1989): 444-458. De Florio, V., Deconinck, G., Lauwereins, R.: The EFTOS Voting Farm: a Software Tool for Fault Masking in Message Passing Parallel Environments. In Proc. of the 24th Euromicro Conference (Euromicro ’98), Workshop on Dependable Computing Systems, V¨ aster˚ as, Sweden, August 1998. IEEE. De Florio, V., Deconinck, G., Lauwereins, R.: Software Tool Combining Fault Masking with User-defined Recovery Strategies. IEE Proceedings – Software 145(6), 1998. IEE. De Florio, V.: A Fault-Tolerance Linguistic Structure for Distributed Applications. Doctoral dissertation, Dept. of Electrical Engineering, University of Leuven, October 2000. ISBN 90-5682-266-7. Johnson, B.W.: Design and analysis of fault-tolerant digital systems. (Addison-Wesley, New York, 1989) Lorczak, P.R., and Caglayan, A.K., and Eckhardt, D.E.: A Theoretical Investigation of Generalized Voters. Proc. of the 19th Int.l Symp. on Fault Tolerant Computing, 1989: 444–451. Anonymous. Manual Pages of EPX 1.9.2. (Parsytec GmbH, Aachen, 1996) Anonymous. Embedded Parix Programmer’s Guide. In Parsytec CC Series Hardware Documentation. (Parsytec GmbH, Aachen, 1996)
