Reporting bad news about software projects - IEEE Xplore

64

IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 50, NO. 1, FEBRUARY 2003

Reporting Bad News About Software Projects: Impact of Organizational Climate and Information Asymmetry in an Individualistic and a Collectivistic Culture Bernard C. Y. Tan, H. Jeff Smith, Mark Keil, Member, IEEE, and Ramiro Montealegre

Abstract—The reluctance of people to report bad news can be a major contributor to the phenomenon of runaway software projects. If senior managers receive bad news sooner, they may be able to prevent runaway software projects through corrective action. Two factors that are known to impact predisposition to report bad news are organizational climate (whether reporting bad news is likely to result in reward or punishment) and information asymmetry (whether hiding bad news is likely to be possible over time). Using matching experiments in an individualistic (United States) and a collectivistic culture (Singapore), this study investigates how the individualism-collectivism dimension of national culture may moderate the impact of organizational climate and information asymmetry on human predisposition to report bad news. The results revealed that individualism appeared to amplify the impact of organizational climate on predisposition to report bad news (compared to collectivism) whereas collectivism appeared to amplify the impact of information asymmetry on predisposition to report bad news (compared to individualism). When deciding on whether to report bad news about software projects, people from an individualistic culture seemed to be more sensitive to organizational climate whereas people from a collectivistic culture seemed to pay greater attention to information asymmetry. These results have useful implications for practice and research involving cross-cultural software project teams. Beyond these implications, these results add a cultural dimension to our existing knowledge on software project management. Index Terms—Collectivism, individualism, information asymmetry, mum effect, national culture, organizational climate, report bad news, software project management.

I. INTRODUCTION

F

AILURE is a common occurrence in the realm of software development. A widely reported study of 23 000 software projects showed that only 26% of the projects were delivered Manuscript received September 12, 2001; revised April 22, 2002. Review of this manuscript was arranged by Special Issue Editors M. Martinsons and R. Davison. B. C. Y. Tan is with the Department of Information Systems, National University of Singapore, Singapore 117543, Republic of Singapore (e-mail: [email protected]). H. J. Smith is with the Babcock Graduate School of Management, Wake Forest University, Winston-Salem, NC 27109 USA (e-mail: [email protected]). M. Keil is with the Department of Computer Information Systems, Robinson College of Business, Georgia State University, Atlanta, GA 30302-4015 USA (e-mail: [email protected]). R. Montealegre is with the Leeds School of Business, University of Colorado, Boulder, CO 80309-0419 USA (e-mail: [email protected]). Digital Object Identifier 10.1109/TEM.2002.808292

on-time, on-budget, and with the promised functionality. The remaining 74% were either canceled before the development cycle was completed or were delivered late, over budget, or with less functionality than was originally promised [50]. Traditionally, researchers have attempted to reduce the probability of software project failure through better project management techniques and more user involvement in the development process. This is sound advice for dealing with technical risks. However, software project failure may also arise because of factors pertaining to organizational culture. One major cause of software project failure is the reluctance of people to report bad news about a project and its status. While evidence of failing may be apparent to people involved in a project, this information may not be communicated up the hierarchy [23] or may be substantially distorted in the communication process [19], [21]. As a result, senior managers who have the authority to remedy the situation are unaware of the true status. This reduces organizational ability to prepare for a project failure and, when the failure eventually comes, increases the size of the loss [18]. Statistics regarding the prevalence of such a phenomenon are elusive, but several publicized incidents of software project failure have suggested that nonreporting or distorted reporting of project status contributed to the difficulties e.g., [12], [35], [54]. In one of the few field studies to consider this phenomenon, Keil and Robey [24] discovered that information systems auditors are frequently reluctant to report bad news about project status due to factors pertaining to organizational culture. This is particularly interesting, considering the fact that reporting true project status is a key role of auditors. If auditors are reluctant to report bad news, it is plausible that programmers, systems analysts, and project leaders would be even less willing to report bad news about software projects. Given that human reluctance to report bad news can be detrimental to software projects, senior managers should be interested in learning which factors may promote or inhibit such a phenomenon in an organization. As discussed by Smith and Keil [45], [46], people may be influenced by dozens of factors when deciding whether or not to report bad news about software projects. Two likely factors pertaining to organizational culture are organizational climate and information asymmetry. Organizational climate refers to the communicated expectations regarding the reporting of bad news about software projects. Such expectations are often manifested in the way organizations treat

0018-9391/03$17.00 © 2003 IEEE

TAN et al.: REPORTING BAD NEWS ABOUT SOFTWARE PROJECTS

people who report bad news (e.g., reward, thank you note, no action taken, discouragement message, or punishment). Information asymmetry refers to the extent to which status information about software projects can be hidden. Whether information asymmetry is sustainable or not depends upon the extent to which the projects are being monitored (lax monitoring helps to sustain information asymmetry). By manipulating these two factors in an experiment, Keil et al. [25] showed that both factors affected human intention to report bad news. However, given that this study was conducted solely in the U.S., it is not known whether (and to what degree) the findings would apply in other national cultures [2]. National culture is the collective mindset that distinguishes people of one nation from another [17]. The management literature has clearly established that differences in national culture are reflected in human decisions and practices [9], [10], [52]. Hence, when deciding whether or not to report bad news about software projects, the degree to which people may be influenced by organizational climate and information asymmetry may be contingent upon their national culture. The objective of this study is to add a cultural dimension to our existing knowledge on software project management. Specifically, this study investigates how national culture may moderate the impact of organizational climate and information asymmetry on human predisposition to report bad news about software projects. The remainder of the present paper is organized as follows. Section II covers the background materials while Section III derives the hypotheses for testing. Section IV presents details of the methodology used. Section V analyzes the data collected. Section VI discusses the implications of the findings for practice and research while Section VII concludes by emphasizing the key contributions. II. BACKGROUND Previous research models associated with reporting of software project status [25], [45]–[47] have been grounded in the whistle-blowing literature. While there has been some debate on the definition of whistle-blowing [20], a widely embraced viewpoint is that whistle-blowers disclose information about organizational dysfunction to persons or organizations who may be able to address the problems [8, p. 824]. In the context of this study, organizational dysfunction is associated with resources being devoted to software projects that are not delivering the outcomes that were intended when the resources were allocated. Whistle-blowing theory assumes that whistle-blowers “lack the power and authority” to handle the situation and must, therefore, “appeal to someone of greater power or authority” [31, p. 31]. The premise of whistle-blowing theory is that people choose to disclose (or not disclose) information due to personal, project team, and organizational factors. In a general context, Miceli and Near [31] reviewed the body of theoretical and empirical work on whistle-blowing and identified dozens of factors that could have causal relationships with human decisions to report bad news. Smith and Keil [45], [46] extended the Miceli and Near [31] framework by including additional causal variables, some outside the whistle-blowing literature, that may be salient in the specific context of software projects.

65

In spite of this emerging theoretical development, empirical evidence on whistle-blowing in software projects appears to be very limited. In one study, Smith et al. [47] conducted an experiment in which perceived behavioral immorality (i.e., deception by a superior) and perceived impact from project loss were manipulated. Both these factors significantly impacted the intention of subjects to report bad news. In another study, Keil et al. [25] manipulated two aspects of organizational culture (organizational climate and information asymmetry). Four treatment scenarios and questions for manipulation checks were developed. In a 2 2 factorial controlled laboratory experiment that involved 122 subjects, organizational climate and information asymmetry were found to impact the intention of subjects to report bad news. In the present study, we choose to extend the model by Keil et al. [25] (by testing it in a cross-cultural context) for several reasons. First, among the many factors identified by Smith and Keil [45], [46] for their potential impact on human intention to report bad news, organizational climate and information asymmetry appear to be two of the most important ones. Second, unlike personal factors that cannot be controlled (e.g., moral development and risk propensity), organizational climate and information asymmetry are aspects of organizational culture that can be influenced to some extent by managerial actions. Therefore, findings related to such factors may guide managerial actions. Third, the relationship between organizational culture (organizational climate and information asymmetry) and national culture is an important research issue that has rarely been explored [6]. Only one other study has examined differences associated with whistle-blowing behavior across cultures [4]. In that study, which did not examine how various factors might impact reporting decisions, all subjects were given the same third-person scenario and asked if an accountant had a responsibility to report irregularities to various entities. The present study is the first cross-cultural study that attempts to identify factors affecting reporting intention in any context (not just in the context of software projects). A. Organizational Climate Organizational climate determines the extent to which people have an incentive to shirk because their interests diverge from those of the organization. In many instances, the organizational climate is such that people perceive serious reprisal risks for reporting bad news. Such risks can take the form of job loss or other adverse impact on career prospects. This gives people an incentive to keep bad news to themselves even when it is not in the best interest of the organization to do so. Theoretical arguments within the whistle-blowing literature suggest that such an organizational climate has a significant impact on perceived responsibility to report bad news. Through formal and informal communication as well as organizational stories, cues that guide behavior are transmitted to members of an organization. When there is a history of reprisals against whistle-blowers, people are likely to consider this as a signal regarding reporting responsibility [31, p. 153]. In an examination of the perceived personal obligations of first-level managers to blow the whistle, Keenan [22] found that “fear of retaliation”

66


within the organizational climate played a great part in reducing perceived obligations. However, when organizations have a climate in which people are expected to report bad news (or are even rewarded for doing so), people would assess their responsibility based on these cues and, thereby, be more obliged to report bad news. One other body of literature offers some insight into these causal relationships. In the 1970s, a small number of studies were conducted on the “mum effect.” The “mum effect” label captured the notion that people are generally reluctant to transmit unpleasant messages [34, p. 39]. Several studies examined this phenomenon, primarily in dyadic contexts and outside of organizational boundaries, so the unit of analysis differs from that in the whistle-blowing literature. Nevertheless, the “mum effect” literature also suggested that a fear of retribution reduces felt personal responsibility to report bad news to another party (see reviews in [5] and [53]). As demonstrated by Keil et al. [25], in a study conducted within the U.S., an organizational climate that was conducive for reporting bad news could reduce the reluctance of people to do so. B. Information Asymmetry Information asymmetry can also influence human intention to report bad news. In some organizations, software projects may be so tightly monitored that it would be difficult for people to hide bad news from others for long. This is a situation where information asymmetry cannot be sustained. In other organizations, the monitoring of software projects may be more lax and there may be opportunities for people to hide bad news from others over a long period of time. This is a situation where information asymmetry can be sustained. When information asymmetry cannot be sustained, people are likely to believe that bad news ought to be reported because nothing can be gained from hiding such information. They may also fear that, by not reporting the bad news, others may quickly arrive at the conclusion that they are deliberately trying to cover up problems. However, when information asymmetry can be sustained, people are likely to believe that delaying the reporting of bad news can help to buy more time so that the software project can be turned around. Under such circumstances, they may conclude that there is less of a need to report bad news. In a study conducted within the U.S., Keil et al. [25] found that sustainable information asymmetry could increase the reluctance of people to report bad news. C. National Culture Based on responses from about 120 000 subjects in 50 countries, Hofstede [16] deduced a model of national culture comprising four dimensions. Each country is given a relative score on each dimension based on a mathematical formula. Although some scholars [11] have criticized the questions and mathematical formula used by Hofstede [16], the validity and stability of the cultural dimensions have been confirmed by many other researchers [39], [42], [49]. These cultural dimensions have also been shown to possess explanatory power in information systems studies, e.g., [13], [26], [36], [43], [51], [52], [57]. So far, the most important and stable dimension in cross-cultural psychology has proven to be individualism–collectivism

[41], [48], [52], [55]. Individualism pertains to nations in which ties between people are loose and people focus on their own needs [17]. In such a culture, people tend to put personal interests above team interests [9]. People take their actions independently of what others think and base their self-understanding on these personal actions [10]. Collectivism pertains to nations in which people are integrated into cohesive groups, which take care of them in exchange for unquestioning loyalty [17]. In such a culture, people tend to put team interests above personal interests [9], [10]. They base their self-understanding on how others around them react [10], largely because they consider themselves members of cohesive groups [1]. Along this dimension of national culture, Singapore is found to be much more collectivistic than the U.S. [17]. III. HYPOTHESES In an individualistic culture, people tend to behave largely in accord with their personal interest, with team interest being subordinated [9], [10]. In such a culture, when the organizational climate is conducive for reporting bad news (because those who report are known to receive rewards for their behavior), many people are likely to embrace reporting behavior. By reporting bad news promptly, they can further their personal interest by trying to be “positive” people and thereby reaping personal rewards. However, in such a culture, when the organizational climate is not conducive to reporting bad news (because those who report are known to receive punishment for their behavior), many people are unlikely to embrace reporting behavior. By hiding bad news, they can avoid being seen as “negative” people and, thereby, avert personal punishment. In a collectivistic culture, people tend to let their personal interest be subordinated to team interest [9], [10]. In such a culture, even when the organizational climate is conducive for reporting bad news, many people may not embrace reporting behavior. Rather than going for personal rewards, they may report the bad news only if they believe the reporting decision is good for their project team (e.g., brings about team rewards). Likewise, in such a culture, even when the organizational climate is not conducive to reporting bad news, many people may not shun reporting behavior. Rather than trying to avoid personal punishment, they may embrace reporting behavior if they believe the reporting decision is good for their project team (e.g., allows the team to address its problems). Together, these arguments suggest that people in an individualistic culture may be more willing to report bad news than people in a collectivistic culture if organizational climate is conducive. However, people in an individualistic culture may be less willing to report bad news than people in a collectivistic culture (the opposite result) if organizational climate is not conducive. Hypothesis 1: The impact of organizational climate (conducive versus not conducive) on predisposition to report bad news about software projects would be stronger in an individualistic culture than in a collectivistic culture. In a collectivistic culture, people tend to put project team interest before personal interest [9], [10] when deciding whether or not to report bad news. In such a culture, when information asymmetry can be sustained (perhaps due to lax monitoring of


67

Fig. 1. Research model.

software projects), many people are likely to shun reporting behavior. By hiding bad news, they can promote their project team interest by giving it time to facilitate project turnaround. However, in such a culture, when information asymmetry cannot be sustained (perhaps due to tight monitoring of software projects), many people are likely to embrace reporting behavior. By reporting bad news promptly, they can further their project team interest by diffusing the possibility that others would later accuse their project team of attempting to hide bad news. In an individualistic culture, people tend to put personal interest before project team interest [9], [10] when deciding whether or not to report bad news. In such a culture, even when information asymmetry can be sustained, many people may not shun reporting behavior. Rather than trying to further their project team interest, they may embrace reporting behavior if this helps to promote their personal interest (e.g., frees themselves from the possibility of having to share the blame should the bad news become known in the future). Likewise, in such a culture, even when information asymmetry cannot be sustained, many people may not embrace reporting behavior. Rather than trying to advance their project team interests, they may shun reporting behavior if their personal interest is at stake (e.g., risks themselves to the possibility of having to shoulder more than their fair share of the blame). Together, these arguments suggest that people in an individualistic culture may be more willing to report bad news than people in a collectivistic culture if information asymmetry is sustainable. However, people in an individualistic culture may be less willing to report bad news than people in a collectivistic culture (the opposite result) if information asymmetry is not sustainable. Hypothesis 2: The impact of information asymmetry (sustainable versus not sustainable) on predisposition to report bad news about software projects would be stronger in a collectivistic culture than in an individualistic culture. Fig. 1 depicts the research model of this study. Hypotheses 1 and 2 suggest that individualism–collectivism would moderate the impact of organizational climate and information asymmetry, respectively, on human predisposition to report bad news about software projects.

IV. METHODOLOGY A key objective of this study was to test and advance a cross-cultural theory that could explain reluctance to report bad

news for different populations of people. Given that internal validity would be a critical issue when testing theories, we used laboratory experiments to achieve precision of measurement and control over extraneous variables, so that causal relationships between constructs in the research model could be established [7]. Matching laboratory experiments were conducted in two distinct national cultures. Results were pooled to obtain a 2 2 2 factorial design with three independent variables: organizational climate (conducive versus not conducive), information asymmetry (sustainable versus not sustainable), and national culture (individualism versus collectivism). The dependent variable was predisposition to report bad news. A. Scenario In the scenario given to the subjects (see Appendix), each subject was asked to play the role of a software project leader in System Solution Corporation (SSC), a large consulting firm. He/she was leading Project Y (a software project to build a new order processing system for a client organization). In the middle of Project Y, he/she discovered a serious problem with the software system under development. If not addressed, this problem would have a significant negative impact on the ability of the client organization to process orders correctly. This scenario was developed by Keil et al. [25] and was subjected to an iterative series of pilot tests and refinements in that study. Two authors of the present paper had extensive software project management experience and could assess the realism of this scenario. B. Operationalization of Organizational Climate Organizational climate was varied at two levels: conducive and not conducive to reporting bad news. The conducive condition was operationalized using a context that emphasized the importance of information disclosure. The not conducive condition was created using a context that discouraged information disclosure (see Table I). As shown in Table I, the manipulation for organizational climate was checked using four questions (OC1–OC4) taken from Keil et al. [25]. Each question was anchored on a seven-point scale ranging from “strongly disagree” (1) to “strongly agree” (7). These four questions had a Cronbach’s alpha of 0.80 for this study. The scores for these questions were averaged for the manipulation check (see Section V-A). A higher score indicated an organizational climate that was more conducive for reporting bad news.

68


TABLE I OPERATIONALIZATION OF ORGANIZATIONAL CLIMATE

C. Operationalization of Information Asymmetry Information asymmetry was varied at two levels: sustainable and not sustainable. The sustainable condition was operationalized using a context that allowed subjects to hide bad news for some time. The not sustainable condition was created using a context that would not allow subjects to hide bad news (see Table II). As shown in Table II, the manipulation for information asymmetry was checked using two questions (IA1 and IA2) taken from Keil et al. [25]. Each question was anchored on a seven-point scale ranging from “strongly disagree” (1) to “strongly agree” (7). These two questions had a Cronbach’s alpha of 0.52 for this study.1 The scores for both questions were averaged for the manipulation check (see Section V-A). A higher score indicated a context where information asymmetry was sustainable. D. Operationalization of Individualism–Collectivism Hofstede [17, p. 53] provided individualism–collectivism scores for more than 50 countries. The scores range from 6 to 91 with the average being 43. High scores are associated with individualistic cultures while low scores are associated with collectivistic cultures. The U.S. has a score of 91 (highly individualistic) while Singapore has a score of 20 (highly collectivistic). Although Hofstede’s [16] data were collected in 1Since the Cronbach’s alpha was low, the result of the manipulation check for information asymmetry was confirmed using separate analyses for IA1 and IA2 (see Section V-A).

the 1970s, recent studies (e.g., [41], [48], [52]) have provided support for Hofstede’s [16] data by showing that Singapore is much more collectivistic than the U.S. These findings lend validity to the claim that national culture is fairly stable in a temporal sense. In this study, individualism and collectivism were operationalized by conducting matching experiments in the U.S. and Singapore, respectively (see Table III). Differences between these two countries on individualism–collectivism have been found to be very salient in prior studies, e.g., [52], [57]. Other than cultural differences, the U.S. and Singapore shared several common characteristics. First, citizens of both countries are educated in English. Second, both countries are economically developed. Third, the software industry plays a key role in the economy of both countries. As shown in Table III, the manipulation for individualism–collectivism was checked (see Section V-A) using a question (IC1) taken from Hofstede [16]. The question was anchored on a five-point scale ranging from “of no importance” (1) to “of utmost importance” (5). A higher score indicated collectivism while a lower score indicated individualism. E. Operationalization of Predisposition to Report Bad News Predisposition to report bad news (the dependent variable) was measured using three questions (RR1 to RR3) taken from Keil et al. [25] (see Table IV). Each question was anchored on a seven-point scale ranging from “very unlikely” (1) to “very


69

TABLE II OPERATIONALIZATION OF INFORMATION ASYMMETRY

TABLE III OPERATIONALIZATION OF NATIONAL CULTURE

likely” (7). These three questions had a Cronbach’s alpha of 0.75 for this study. The scores for these questions were averaged to obtain the dependent variable. A lower score indicated greater reluctance to report bad news about the software project. F. The Procedure Subjects in each treatment condition were told that this was an experiment on business decision-making and that their answers would remain anonymous. They were reminded that their participation was voluntary and those who did not wish to participate could leave. All subjects chose to participate in the ex-

periment. Subjects in each country were randomly assigned to one of the four treatment conditions (obtained by varying organizational climate and information asymmetry). During the experiment, subjects received a copy of the scenario corresponding to their respective treatment condition. They were asked to read the scenario and complete a questionnaire that measured their likelihood of reporting the bad news about their project (RR1 to RR3). Subjects also responded to a series of questions for manipulation checks (OC1 to OC4, IA1, IA2 and IC1). They then provided their demographic information for control checks.

70


TABLE IV OPERATIONALIZATION OF RELUCTANCE TO REPORT BAD NEWS

G. The Subjects A total of 354 subjects (162 citizens of the U.S. and 192 citizens of Singapore) participated in this study.2 The subjects were working professionals who were attending graduate (masters-level) classes part-time in the evenings. They were enrolled in information systems courses at a large university in their respective countries. Subjects in the U.S. had an average age of 27.7 years, an average work experience of 5.5 years (2.5 years on software projects), with 56% males and 44% females. Subjects in Singapore had an average age of 24.0 years, an average work experience of 3.4 years (2.1 years on software projects), with 60% males and 40% females. Many empirical studies have used student subjects in decision-making tasks, e.g., [44] and, specifically, decisions associated with project management, e.g., [15], [26], [47]. While some researchers have argued that the use of student subjects could limit the generalizability of the results to organizational decision-makers, other scholars have provided support for using students as surrogates for managers, e.g., [7], [37]. In this study, the issue of generalizability was addressed by using (graduate student) subjects who were full-time working professionals with experience on software projects. These subjects should be capable of grasping the business and political aspects of the scenario. Their demographic characteristics were close to those of a project leader (the role prescribed in the scenario). Thus, this subject pool appeared to be appropriate for this study. V. ANALYSES A. Manipulation and Control Checks Results of manipulation and control checks that were stronger than the 5% level were considered significant. For the manipulation of organizational climate, subjects assigned to the treat4.31, std. ment of conducive organizational climate (mean 1.01) scored significantly higher on the manipulation dev. check (OC1–OC4) than subjects assigned to the treatment of 2.28, std. dev. not conducive organizational climate (mean 1.37) ( ). For the manipulation of infor2The responses of 58 other subjects in the U.S. experiment and eight other subjects in the Singapore experiment were dropped from this study because these subjects were not citizens of the U.S. and Singapore, respectively. The 354 responses used were all obtained from citizens of the U.S. and Singapore.

mation asymmetry, subjects assigned to the treatment of sustain4.13, std. dev. 1.48) able information asymmetry (mean scored significantly higher on the manipulation check (IA1 and IA2) than subjects assigned to the treatment of not sustainable information asymmetry (mean 2.24, std. dev. 1.30) ( ). 3 For the manipulation of individualism–collectivism, subjects from the collectivistic culture (Singapore) (mean 4.43, std. dev. 0.63) scored significantly higher on the manipulation check (IC1) than subjects from the individ4.19, std. dev. 0.67) ( ualistic culture (U.S.) (mean ). The manipulation of the three independent variables appeared to be successful. Control checks were carried out on subject demographics for each country. Mann–Whitney tests showed that the gender ratio of subjects in each country did not differ significantly across the different treatments for organizational climate and information asymmetry. F-tests showed that the age and work experience of subjects in each country did not differ significantly across the different treatments for organizational climate and information asymmetry. B. Hypothesis Tests Table V presents the descriptive statistics for the dependent variable. Table VI shows the results of an ANOVA test on the dependent variable. Given that the dependent variable could not simultaneously fulfill the homogeneity and normality requirements of the ANOVA test, all significant results found were confirmed with nonparametric tests. Results of statistical tests that were stronger than the 5% level were deemed significant. The ANOVA test revealed main effects due to organizational ) and information asymmetry climate ( ). In addition, there were two-way inter( actions between organizational climate and individualism–col) and between information lectivism ( 3Since the questions for this manipulation check had low Cronbach’s alpha (see Section IV-C), this result was confirmed using separate analyses for IA1 and IA2. Subjects under the treatment of sustainable information asymmetry (mean = 4.76, std. dev. = 1.89) scored significantly higher on IA1 than subjects under the treatment of not sustainable information asymmetry (mean = 2.23, std. dev. = 1.50) (t = 13:97; p < 0:01). Subjects under the treatment of sustainable information asymmetry (mean = 3.51, std. dev. = 2.01) also scored significantly higher on IA2 than subjects under the treatment of not sustainable information asymmetry (mean = 2.26, std. dev. = 1.73) (t = 6:28; p < 0:01).


71

TABLE V MEAN (STD. DEV., SAMPLE SIZE) OF DEPENDENT VARIABLE

TABLE VI RESULTS OF ANOVA TEST ON DEPENDENT VARIABLE

** p