Visualizing the Invisible

Visualizing the Invisible ______________________________________________________________________________________________________

Application of Knowledge Domain Visualization to the Longstanding Problem of Disciplinary and Professional Conceptualization in Emergency and Disaster Management

Joseph George Martin III

©Copyright 2012-2014 by Joseph George Martin III All rights reserved.

Cover Image: Part of a 725-node Author Co-Citation (ACA) Network created by the author from 2930 articles in Emergency and Disaster Management, 1994-2013.

VISUALIZING THE INVISIBLE: Application of Knowledge Domain Visualization to the Longstanding Problem of Disciplinary and Professional Conceptualization in Emergency and Disaster Management

ii

AMERICAN PUBLIC UNIVERSITY SYSTEM Charles Town, West Virginia

VISUALIZING THE INVISIBLE: APPLICATION OF KNOWLEDGE DOMAIN VISUALIZATION TO THE LONGSTANDING PROBLEM OF DISCIPLINARY AND PROFESSIONAL CONCEPTUALIZATION IN EMERGENCY/DISASTER MANAGEMENT

A thesis submitted in partial fulfillment of the requirements for the degree of MASTER OF ARTS in EMERGENCY AND DISASTER MANAGEMENT by Joseph George Martin III

Department Approval Date: December 20, 2012

The author hereby grants the American Public University System the right to display these contents for educational purposes. The author assumes total responsibility for meeting the requirements set by United States Copyright Law for the inclusion of any materials that are not the author’s creation or in the public domain.

iii

DEDICATION This thesis is dedicated with love to my parents, family, and dear friends: Mom, Mary, Jeff, Sheryl, Ashley, Alex, Stacy, Jen, Randy, Peggy, Cindy, and many others both present and long past. It has been a long road and you have all helped make completion of the journey to this point possible. Most of all, however, this work is dedicated with much love, and some sadness, to the memories of my father, Joseph George Martin, Jr., who lived to see this thesis’ beginning but not its finish; and my stepfather, William A. “Bill” Carlson, who passed away less than a month after the final version was accepted. This is for you both and I hope it makes you proud...

iv

ACKNOWLEDGMENTS

I would like to gratefully thank my thesis advisor, Dr. James Smith, for his time, patience, enthusiasm, and understanding throughout the past four months. There were a couple of times I was unsure if me and this thesis would make it across the finish line, but Dr. Smith did not waiver in his confidence. I would also like to thank the outstanding faculty of American Military University who have made the past two years of study a pleasure, especially Dr. Randall Cuthbert, Dr. Katie Crosslin, and Dr. Tim Bagwell. Thank you to Claire Rubin as well for her thoughtful questions, comments, and support. Thank you to Dr. Chaomei Chen, of Drexel University, not only for taking the time to offer helpful comments and suggestions, but for also making his incredible KDViz software freely available to the world. Gratitude is also owed to the University of Texas at Dallas (and its fine staff of librarians) for allowing access to the Web of Science, without which this thesis would have not been possible. Finally, I would be remiss if I did not say thank you to my employer (you know who you are...) who has shown infinite flexibility and understanding in not only putting up with the demands and neuroses of a graduate student completing a thesis, but continuing to pay me while doing so.

v

ABSTRACT OF THE THESIS

VISUALIZING THE INVISIBLE: APPLICATION OF KNOWLEDGE DOMAIN VISUALIZATION TO THE LONGSTANDING PROBLEM OF DISCIPLINARY AND PROFESSIONAL CONCEPTUALIZATION IN EMERGENCY/DISASTER MANAGEMENT By Joseph George Martin III

American Public University System, December 20, 2012 Charles Town, West Virginia

Professor James Smith, Thesis Professor The status of emergency and disaster management (EDM) as an academic and professional discipline remains one of the field’s lingering, unresolved questions. A majority of the literature appears to support the claim that emergency management either is, or is in the process of becoming, a recognized academic and professional discipline. The claim’s key belief is that the field possesses a unique body of knowledge, an essential conceptual requirement for disciplinary status. This thesis examines the concept of professional/academic disciplines, as it relates to bodies of knowledge, and more specifically, the EDM body of knowledge. The technique of knowledge domain visualization (KDViz) using co-citation analysis is discussed. Analysis and visualization of the disaster literature is conducted using CiteSpace II, a KDViz software program, and a dataset of 2385 EDM articles, 1994-2011, obtained through the Web of Science bibliographic database. Results are presented and discussed within the context of both practical and theoretical concerns affecting not only EDM, but the entire field of disaster studies.

vi

TABLE OF CONTENTS PAGE ACKNOWLEDGEMENTS.........................................................................................v ABSTRACT.................................................................................................................vi LIST OF TABLES ......................................................................................................viii LIST OF FIGURES.....................................................................................................ix I.

INTRODUCTION..............................................................................................1 Statement of the problem..........................................................................2 Nature, purpose and significance of study.................................................3 Research questions and hypotheses............................................................4 Assumptions and limitations.....................................................................5 Organization of remainder of paper..........................................................6

II. LITERATURE REVIEW.....................................................................................8 Conceptualizing professional and academic disciplines.............................8 Disciplinary status and the body of knowledge in EDM.........................10 Bibliometrics, co-citation, KDViz, and CiteSpace II...............................14 III. METHODOLOGY...........................................................................................21 Design and rationale..............................................................................21 Dataset construction..............................................................................22 Pre-analysis CiteSpace II Settings...........................................................28 Quantitative and qualitative analysis procedures....................................34 . IV. RESULTS...........................................................................................................37 General results........................................................................................37 Author Co-Citation Analysis (ACA).......................................................39 Document Co-Citation Analysis (DCA).................................................47 Journal Co-Citation Analysis (JCA)........................................................54 V. DISCUSSION AND CONCLUSIONS.............................................................61 General findings of the analyses..............................................................61 Research hypotheses and questions.........................................................61 Creating a disciplinary framework..........................................................65 Recommendations for further study.......................................................65 REFERENCES..........................................................................................................68 APPENDIX A: SAMPLE OF ORIGINAL WEB OF SCIENCE DATA..................73 APPENDIX B: SAMPLE OF FINAL DATASET (ACA VERSION).......................76

vii

LIST OF TABLES PAGE 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16.

FEMA Body of Knowledge Project Top Recommended Readings, 2006-2009..................13 Selected Co-citation/Knowledge Domain Visualization Studies.........................................16 Journals Used in the Dataset..............................................................................................26 Summary of Dataset Characteristics...................................................................................26 Ten Most Cited (per WoS) Source Articles in Dataset.......................................................29 CiteSpace II Analysis Settings............................................................................................36 CiteSpace II Network and Cluster Analysis Results............................................................38 ACA: Fifteen Most Cited Individual First Authors, 1994-2011 (All Citations)..................40 ACA: Fifteen Most Cited Individual First Authors, 1994-2011 (Unique Instances)...........40 ACA: Fifteen Most Cited Agency Authors, 1994-2011 (All Citations)...............................41 ACA: Fifteen Most Cited Agency Authors, 1994-2011 (Unique Instances)........................41 ACA: Top Fifteen Authors (Combined) Ranked by Sigma, 1994-2011..............................42 DCA: Fifteen Most Cited References, 1994-2011..............................................................47 DCA: Top Ten Cited References Ranked by Sigma, 1994-2011........................................69 JCA: Twenty-five Most Cited Journals/Book Series, 1994-2011 (Unique Instances)..........55 JCA: Top Ten Journals/Book Series Ranked by Sigma, 1994-2011....................................56

viii

LIST OF FIGURES PAGE 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.

Screenshot of CiteSpace II Control and Visualization Panels.............................................18 Basic visualization concepts used in CiteSpace II...............................................................19 Distribution of articles in dataset by year...........................................................................24 Distribution of dataset articles’ WoS times cited................................................................27 Threshold settings in CiteSpace II: number of nodes.........................................................32 Threshold settings in CiteSpace II: number of links...........................................................33 Author Co-citation Analysis (ACA) merged network, 1994-2011......................................43 Close-up of core concentration of authors in ACA network...............................................44 ACA network visualization with topic/subject/concept labels.............................................45 Evolution of the ACA network, 1994-2011.......................................................................46 Document Co-citation Analysis (DCA) merged network, 1994-2011................................49 Close-up of lower right quadrant of super-cluster in DCA network....................................50 Super-cluster at center of DCA visualization, with labels....................................................51 Close-up of densest part of DCA network, with labels........................................................52 Evolution of the DCA network, 1994-2010.......................................................................53 Journal Co-citation Analysis (JCA) merged network, 1994-2011.......................................57 Close-up of lower right section of JCA visualization...........................................................58 Close-up of center right section of JCA visualization..........................................................59 Evolution of JCA network, 1994-2010..............................................................................60 Possible disciplinary structure for Disaster Studies and Sciences.........................................66 Preliminary framework for the discipline of Disaster Studies and Sciences.........................67

ix

CHAPTER I: INTRODUCTION That the field of disasters as an academic and occupational pursuit, including what has become known as emergency management (EM), or emergency and disaster management (EDM)1, has “come of age” in the last quarter-century, is a statement unlikely to generate significant disagreement. The progress would be hard to deny. Born of the Cold War’s vision of civil defense, the fledgling concept of emergency management survived a neglected childhood as an often poorly staffed, poorly funded, and poorly understood function of local, state, and federal government. Today, as disasters seem to grow in number, size, scope, and impacts at a worrying rate with each passing year, the field enters young adulthood. Where it was once more of an afterthought, it is now almost impossible to imagine a future without EDM. In the United States, EDM now exists in some form at every jurisdictional level of government, and significant strides have been made in moving EDM from being primarily an interest of the public sector to being an interest of both the public and private sectors. EDM has also grown in acceptance as a valid academic subject area, with a firm foothold now established in higher education at the undergraduate and graduate level. Finally, in the occupational arena, two national associations (the National Emergency Managers Association [NEMA] and the International Association of Emergency Managers [IAEM]) exist to promote/advance EDM as a career, as well as provide support to those currently employed in EDM or a related area. The rapid progress the field has enjoyed recently in development as an occupation and academic area of study is not, however, without downside. Such rapid progress and development has potential to diminish the perceived need to address necessary but difficult issues within the field. Quarantelli (1998/2005a), then Perry and Quarantelli (2005), have twice demonstrated with the same simple (and still unresolved) question that the very focal point of the field lacks a generally accepted definition. It is entirely conceivable that some, seeing the positive growth and expansion of EDM, might find the need to reach some acceptable working solution to the question of “What is a disaster?” an unnecessary definitional quest. Or perhaps the issue, it might be optimistically felt, will work itself out as the field continues evolving with time. But Quarantelli and Perry’s definitional question is quietly deceptive. It is not simply a question of definition. It is actually a fundamental question of structure: how one defines “disaster” determines what is and is not part of the field. The definitional issue of disasters, however, is not the only unresolved fundamental structural question within EDM. Over the course of the past fifteen or so years, unresolved questions have arisen in the literature about the disciplinary and professional status of EDM. On the one hand are authors like Darlington (n.d.) and key figures like Mileti (as related by Phillips (2003)), who believe that there is both an academic discipline and either an outright profession, or at the very least, a profession-in-the-making. For the purposes of the first two chapters, “emergency and disaster management” or “EDM” refers to both an academic area of study and an occupational area., and is broadly construed for the purposes of the first two chapters as a specialty within a larger field called “disaster studies” concerned with the application of knowledge and practical skill to the management of emergencies and disasters in the public and private sectors. EDM should be read as equivalent to “emergency management” and “EM”. Although the more common terms could have been used, EDM has been selected as a more neutral term which avoids any preconceptions and biases that might attach to the use of EM. 1

1

The belief also manifests through frequent appearances of the words “profession” and “professional” in the literature. On the other hand, Phillips, and Cwiak (2009b) have indicated the issue is still in doubt, and proclamations to the contrary are premature. Who is right? Is the question of disciplinary and professional status important? Is the question even capable of definitive answer? This thesis will examine the issue of disciplinary and professional status in EDM. The technique of knowledge domain visualization (KDViz), using the bibliometric technique of co-citation analysis, will be applied in an attempt to provide a visible representation of EDM’s intellectual structure, which may provide essential clues to answering questions about the field’s disciplinary and professional status. Statement of the Problem Just as it is with Quarantelli and Perry’s definitional question, the professional and disciplinary status of EDM also reduces to a fundamental question of structure, though the route to that realization is less direct than might be supposed. At first the task appears relatively simple: 1) analyze conceptions of academic disciplines and professions to determine the essential characteristics of disciplines and professions; 2) compare the characteristics to those of EDM. If the characteristics are found, one concludes that EDM is a discipline and/or profession. If the characteristics are not found, one concludes that EDM is not a discipline. As will be shown in Chapter II’s literature review, an unrecognized problem emerges during the first step. The problem is unrecognized because if it were correctly identified, authors would understand that moving to the second step is an impossibility, and any conclusions unsupportable, until the problem is addressed. The problem centers on the idea of a disciplinary/professional body of knowledge. Authors from a variety of diverse fields, including Wilson (2001), Evetts (2003), Schmidt (2008), and Johnson (2012), have identified the concept of a specialized body of knowledge as an essential characteristic of academic and professional disciplines. Exactly what constitutes an acceptable body of knowledge, and how one might empirically test for its presence in a field is more problematic. The problem is especially acute for claims of disciplinary status by multidisciplinary fields. Depending upon how one defines the concept, such as adding the requirement of uniqueness to make a body of knowledge acceptable, it can be legitimately argued that multidisciplinary fields have no bodies of knowledge themselves. They only possess knowledge that can already be claimed by other disciplines. This is not to say that the argument is correct or incorrect, but instead to say that such issues must be addressed by any attempt to define the concept of a body of knowledge, or by any field, including EDM, that attempts to justify its own claims to disciplinary status. It is intellectually unacceptable, if one agrees that a body of knowledge is an essential requirement for disciplinary and professional status, to fail to provide a conceptual definition of a body of knowledge, and to demonstrate how a particular field’s claim of disciplinary status satisfies the body of knowledge requirement. Unfortunately, as will be shown in this paper, this is, with some exceptions, what has happened in EDM. The importance of the problem goes beyond the obvious reasons why EDM and other fields desire the recognition and acceptance as an academic and/or professional discipline, such as the increased prestige, higher income, market control, and possible access to power that can come from such status (Larson, 1979). As pointed out at the beginning of the section, the question of disciplinary status, by way of the body of knowledge problem, is a fundamental question of the structure of EDM. If EDM has an identifiable body of knowledge, then the field possesses a knowable intellectual structure that is

2

at the same time both abstract and concrete. The idea of a body of knowledge is itself a conceptual representation of the structured knowledge within an academic/professional discipline, but that knowledge also exists in concrete form through the field’s published literature. Just as the question of “What is a disaster?” seeks to ascertain the boundaries of EDM, so the question of EDM’s body of knowledge speaks directly to not only possible knowledge boundaries, but also how the field’s knowledge is internally organized and structured. If evidence of these boundaries, structures, and/or organization cannot be found, EDM likely has little legitimate claim to disciplinary status. Nature, Purpose, and Significance of Study This thesis study is an exploratory bibliometric analysis of the literature of EDM, 1994-2011, as contained in a dataset created using the Thomson Reuters Web of Science (WoS) bibliographic database, and analyzed using the knowledge domain visualization (KDViz) software package, CiteSpace II. KDViz and co-citation analysis will be the bibliometric methods utilized. 2 Much of the study uses the quantitative (statistical and mathematical) methods of bibliometric and citation analysis developed within the field of Library and Information Sciences (LIS) since the late 1950’s (Börner, Chen, & Boyack, 2003; Hood and Concepción, 2001). KDViz (also called science mapping, scientific domain mapping, and a variety of similar terms) is a specific application within the broader ILS subject area known as knowledge visualization. KDViz seeks to visually depict, or map, scientific knowledge domains (and in fact, the entirety of science) using bibliometric data, like that contained within bibliographic databases and citation indexes such as Web of Science, Scopus, and Google Scholar. Rapid advances in computer science and technology near the start of the 1990’s (especially the growth of raw processing and graphics processing power in desktop computing) has enabled KDViz software programs to be developed capable of performing the complex calculations necessary for analyzing and creating visual depictions of networks based upon huge amounts of bibliographic data. It is important to point out, however, that although KDViz is based in quantitative methods, qualitative analysis is also an integral part of KDViz, and is essential in the later stages of analysis for identifying possible patterns, as well as for evaluating the general accuracy/validity of the domain visualizations. The third part of Chapter II will thoroughly examine the history, methods, and applications of bibliometric analysis, citation analysis, and KDViz. Though many, if not most, academics and professionals within EDM will likely find KDViz and many of the topics covered in this thesis unfamiliar territory, a key starting point is the understanding that a significant research literature has developed during the past 15 years (which will be discussed in the third part of Chapter II) demonstrating that KDViz is capable of revealing the subtle intellectual structure of scientific disciplines. If this is the case, then it seems reasonable to suggest that KDViz might be of use in looking for an EDM body of knowledge and a possible disciplinary structure to the field. Although various types of bibliometric, co-citation, and and/or visualization analysis has previously been applied within the field of terrorism research (Chen, 2006a; Reid, 1997; Reid & Chen, Technically, KDViz and co-citation analysis are not entirely equivalent. It is possible to conduct KDViz using other bibliometric techniques, but co-citation has become the primary method. For the purposes of maintaining consistency throughout this thesis, unless otherwise specified, KDViz will refer to domain visualization based upon co-citation analysis. 2

3

2007); and direct citation analysis has been applied within EDM (Janssen, Schoon, Ke, & Börner, 2006; Janssen, 2007; Comfort, Waugh, & Cigler, 2012); an attempt to reveal the possible underlying disciplinary structure of EDM (if it exists) through co-citation and KDViz has not yet been attempted. The purpose of this thesis is to reveal the structure of EDM using KDViz through CiteSpace II, an open-source KDViz software package created by Dr. Chaomei Chen of Drexel University, a leading KDViz researcher for many years. As this is the first application of KDViz within EDM, this study has potential for significance on several levels. First and foremost, if the research hypotheses are confirmed, it opens the door to further examination and application of KDViz by others as a tool for examining EDM from new perspectives. This will hopefully facilitate progress towards resolving the longstanding, niggling, questions and confusions that surround EDM regarding its identity and structure, as both body of knowledge and possible academic/professional discipline. It is believed that resolving these issues would be a major step forward for the field. Secondly, the methods used in this thesis, if shown to have value, also have future use in tracking and depicting the development of the field and its literature, on a yearly basis as a sort of yearbook of the field. Third, by applying and showing the general applicability of KDViz to a complex, emerging, multidisciplinary, field, this thesis contributes to the LIS and KDViz bodies of knowledge by demonstrating new applications for the approach, as a majority of previous research on KDViz has focused on its use in fairly well-defined scientific disciplines. Finally, if the hypotheses of this thesis are shown correct, then it is also a powerful reminder of the importance of disciplinary crossfertilization in the creation of new knowledge, as well as demonstrating the growing problem in science of what is called latent domain knowledge, where information that may be highly valuable to one particular discipline is overlooked because it originates in different disciplines or fields, or is published in less recognized sources (Chen, Kuljis & Paul, 2001). Research Questions and Hypotheses At the core of this study, there are three primary research questions, and five research hypotheses. These are the primary research questions asked in this study: 1. Is EDM a suitable area for application of co-citation analysis and KDViz? If it is, what does it reveal about the possible structure and organization of EDM? 2. Does the analysis reveal evidence of a significant academic and/or professional EDM body of knowledge; and how does KDViz compare to other attempts to determine the key works, journals, and trends in the EDM body of knowledge, such as the FEMA Higher Education Program Body of Knowledge Project? 3. How do the findings of the analysis relate to questions of disciplinary and professional status in EDM? These are the primary research hypothesis: 1. Co-citation analysis and KDViz are approaches ideally suited, so long as methodological limitations are understood, to provide unique views of the possible intellectual structures and relationships within EDM that have not previously been detected. 2. The sources (authors, articles, and journals) that are the foundation of

4

EDM/EM are vastly larger in size and scope; are more diverse in their multidisciplinary origins; and are more poorly integrated into the field than possible realized. 3. Analysis and visualization will reveal an identifiable disciplinary structure to a larger knowledge domain than what is commonly thought of as EDM/EM. The larger knowledge domain will refer to that which has been called many names, but in this thesis will be called the discipline of Disaster Studies and Sciences. EDM/EM do not represent the entirety of the field, but instead are subdomains or specialties within the larger knowledge domain of Disaster Studies and Sciences. 4. Uncertainty in the field regarding a professional body of knowledge in EDM will be reflected in the visualizations. Visualizations are hypothesized to show structure for EDM/EM as an academic discipline within the larger field of Disaster Studies and Sciences, but less so for evidence of a structure of the practical application of that knowledge, which should be expected in the knowledge structure of a profession. Professions should show a distinction between the basic science of the field that underlies the profession, and the knowledge that is part of the practice of the profession. In medicine, for example, there is a discrete knowledge structure for the branches of basic medical sciences and for the branches of clinical medical practice. Such structure is anticipated to be poorly realized, if at all, in professional EDM. 5. When compared to other attempts to identify a professional EDM/EM body of knowledge, such as FEMA’s yearly Body of Knowledge Survey, KDViz will show a gap between the actual knowledge domain that exists and the knowledge domain presented by the survey. Assumptions and Limitations As will be seen shortly in Chapter II, citation analysis and KDViz are founded on empirical not theoretical grounds, though greater effort is being made now to develop an underlying theory (Chen, 2006b; Purchase, Andrienko, Jankun-Kelly, & Ward, 2008). Although the origins may be empirical, there are still theoretical assumptions underlying citation analysis and KDViz: 1) the citation is a fundamental unit of communication, acknowledgement, and recognition within, and across, the literature of scientific/academic/professional disciplines; 2) because of the function of citations, direct and indirect measures and methods can be developed to evaluate the influence/significance of authors, articles, journals; subjects, etc. based on citation patterns; 3) the connections, linkages, and relationships between cited authors, articles, journals, subjects, concepts, keywords, disciplines, etc., form networks capable of being analyzed and visually represented the same as social networks; and 4) the intellectual structure and organization of scientific/academic/professional disciplines is reflected in the structure of their citation (particularly co-citation) networks, allowing them to reveal the unseen structure of disciplinary knowledge. The use of bibliometric and citation analysis techniques also carries with it limitations and ethical implications, particularly when such analysis is presented as representing a true and accurate picture of the weight of individuals’ intellectual contributions to a field. Such considerations, controversies, and criticisms of bibliometric methods are not new in the history of citation analysis, examples of which can be found in papers by Schoonbaert and Roelants (1996), and Van Raan (2005). As seen in these papers,

5

the harshest criticisms are usually reserved for the use of bibliometrics and citation analysis as a form of academic quality evaluation and/or academic ranking. Although this is not the purpose or intention of the present study, the same issues that have led authors to question the use of bibliometrics as an evaluation metric are also issues that affect the general use of citation analysis. The issues can be broadly divided into those of theoretical assumptions and those of methodology. Osareh (1996a, 1996b) presents a comprehensive literature review of citation and co-citation analysis, detailing eleven key limitations of the method, including both theoretical assumptions and methodological issues. These theoretical problems are also presented by Harzing (2010). The issues center around the general bibliometric assumption that citations have a similar meaning and role across all scientific and academic disciplines: in fact, it is likely that not all citations are created equal. Differences and variations in citation patterns and rates have been found to exist across fields of study, and with time within a single field. There are also disciplinary differences in preferred methods of scientific communication (e.g. journals; conference papers and proceedings; books; and monographs), which can mean that the source of a citation can affect its value, depending upon the field of study. Methodological issues pointed out by Osareh include: the problem of whether to count or remove selfcitations from totals (including the self-citations of secondary and tertiary authors where there is multiple authorship); incompleteness of sources indexed within the major bibliographic databases; biases in the databases for citations in English-language publications; variations in how the same name or title is presented within databases, including misspellings (“E.L. Quarantelli”, for example, could appear as “EL Quarantelli”, E.L Quarranntelli”, “Enrico L. Quarantelli”, E Quarantelli, E. Quarantelli, etc.-- each of which would be recognized as a distinct author by citation and co-citation analysis software); difficulties distinguishing different authors who share identical surnames and initials; and difficulties in verifying that the citation counts listed within databases are accurate. Within the same vein, Harzing (2010) reports various studies that have found significant variations in citation totals produced by different databases, with the variation being affected by factors such as the database being used (which determines the sources indexed), and the particular field under study (different databases represent different fields of study to greater or lesser degrees). Despite these problems and limitations, Osareh concludes that, aside from the questionable use of citation totals to assess the quality of an author’s work, “...bibliometric methods, particularly, citation analysis techniques are useful tools in evaluating science and technology activities...” (1996b, p. 223). Bibliometric and citation analyses should be carried out conscientiously, and results presented carefully, not only to ensure that the analyses are technically correct, but that the interpretation of results are presented so that others are directly steered away from erroneous conclusions that are not justified by data, or that could be taken out of context. Some of the problems and limitations discussed in this section did surface during the study, and will be discussed in detail in Chapter III. Organization of Remainder of Paper In Chapter II, a three-part literature review is conducted. First, the conceptualization of disciplines and professions is examined. The next section turns the focus onto the literature regarding the body of knowledge, and the disciplinary status of EDM. The final part of Chapter II gives an overview of the history and methods of bibliometric analysis, citation- and co-citation analysis, KDViz, and CiteSpace II. Chapter III provides details of the methodology used in this study, including its structure and

6

rationale; details of dataset collection and construction; and the software settings used in the analysis and the reasons for those choices. The results of the analysis, including Author, Document, and Journal Co-citation network visualizations, are presented in Chapter IV. In Chapter V, these results are discussed in relation to the original research questions and hypotheses. Implications of the research, both present and future, and how the steps taken in this study might be progressed in future research, are also presented.

7

CHAPTER II: LITERATURE REVIEW The present study involves three distinct but inter-related topics. The first concerns how academic/professional disciplines have been historically conceptualized, particularly as related to the concept of a body of knowledge. The second requires the first topic as a foundation, for it examines attempts to establish the disciplinary status of EDM, academically and professionally, especially in terms of demonstrating a body of knowledge. Finally, the third topic provides an overview of bibliometric methods, including co-citation analysis, KDViz, and the CiteSpace II software that resides at the analytic core of this thesis. It therefore seems entirely appropriate that the chapter be divided into three parts, with each part devoted to one of the three topics. Conceptualizing Professional and Academic Disciplines Wilson (2001) provides an excellent historical, linguistic, and sociological survey of professional emergence. As detailed in her paper, although some authors have asserted a far earlier historical beginning to professions, the term itself dates back to the 15th-16th Century, and the three original university-taught “learned professions”: medicine, law, and theology. Attempts to define a set of generalized criteria to describe professions, though, were still several hundred years away, and did not begin until the early 1900s (Schmidt, 2008). The core sociologic traits of a profession, originally put forth by the sociologist Friedman, and summarized in Wilson, are “autonomy and monopoly” (Wilson, 2001, p. 26). Professions are autonomous in that they control the profession’s entry requirements, education, training, and discipline, and that the professional controls their own work. As a monopoly, professions prevent unapproved individuals from practice, either directly through government licensure or indirectly by development of market preference for a professional credential. Half a century later, sociologist Ernest Greenwood (1957) proposed a slightly expanded set of essential characteristics: “Succinctly put, all professions seem to possess: (1) systematic theory, (2) authority, (3) community sanction, (4) ethical codes, and (5) a culture” (Greenwood, 1957, p. 45). He also proposed that there is no bright line distinction between professions and non-professions based on the presence or absence of the characteristics. The characteristics are present in many occupations but it is the degree to which the characteristics are present that is important in deciding where along the line of professionalism the occupation resides: “...we must think of the occupations in a society as distributing themselves along a continuum” (p. 46). In the 1960’s, Wilensky (cited in Schmidt, 2008) defined the professionalization of occupations as a progressive process starting first with the establishment of training schools/programs; university programs follow; local associations are created then national associations; state licensure is established; and reaches completion with the establishment of a code of ethics. These traditional views of professions, however, have been questioned in recent years, with some finding the criteria out of step with modern developments. Evetts (2003) proposes a broader understanding that defines professions as: “dealing with work associated with the uncertainties of modern lives in risk societies. Professionals are extensively engaged in dealing with risk, with risk assessment, and through the use of expert knowledge, enabling customers and clients to deal with uncertainty” (p.397). For Evetts, rigid boundaries are no longer useful, and even identification of an occupation as a profession can vary between societies/cultures. For Evetts, it is more useful to examine

8

the rise of occupational professionalism instead of professions, per se. Blending the traditional criteria with a more modern interpretation of professions, the work of Bayes (cited in Schmidt, 2008) identifies three essential characteristics: extensive training; significant intellectual investment; and provision of an important service to society. Common to both traditional and modern definitions of a profession, whether stated explicitly or implicitly, directly or indirectly, is the idea that a profession requires possession of some specialized sphere of knowledge/theory. This is its body of knowledge (BOK), or common body of knowledge (CBOK). According to Swick (2000), and echoing Evetts, it is the “application of expert knowledge” (p. 613) which has become the predominant characteristic of modern professions. Of course, the question then becomes what is it that distinguishes expert knowledge from the “non-expert” knowledge required of other occupations and skilled trades? Why is the knowledge applied by a carpenter not considered “expert” but that applied by a structural engineer is? The answer lies in the fact that distinct bodies of knowledge are not limited to professions, but are also one of the defining characteristics of scientific and academic disciplines (Johnson, 2012). As presented by Johnson, a discipline can be said to exist where either a distinct body of knowledge, and/or a unique methodology can be shown to exist in a field. Thus, although it may at first appear that speaking of academic and professional disciplines is to speak of concepts with separate identities, they both must meet, more or less, the requirement of possessing specialized knowledge and/or methods. The difference between an academic and a professional body of knowledge lies in the knowledge content contained within that body. Academic bodies of knowledge contain the unique knowledge of the field itself, whereas professional bodies of knowledge pertain to the practical application of one or more academic bodies of knowledge. It is the relationship between disciplines, professions and professional practice, and the included bodies of knowledge that gives rise to expert knowledge. The relationship between professional and scientific/academic bodies of knowledge is logical and necessary. It is not accidental. Professions, in the view of this thesis, are occupations that require application from one or more scientific and/or academic bodies of knowledge towards issues of human concern. This often first requires basic mastery of skills and knowledge from multiple foundation disciplines (e.g. chemistry, biology, physics, and mathematics) before one is allowed access to the core academic discipline of the profession (e.g. medical science), its subspecialties (e.g. internal medicine), and the body of knowledge required for professional practice (e.g. internist). The professional body of knowledge contains information, skills, methods, techniques, results, and standards of application that must be mastered (and perpetually updated) by those who wish to practice (Sefton, Shea, & Hines, 2011). This leads to an important conclusion: before there can be a professional body of knowledge, and thus a professional discipline, there must first exist one or more academic disciplines that the profession is based upon. One should not confuse a profession’s unique body of knowledge with knowledge, skills, and abilities (KSAs) needed for professional practice. An example of KSAs can be found in those offered by the International Association of Emergency Managers (n.d.). There are many generic KSAs (e.g. communication; interpersonal; team-building) that many professions require but they are not unique to any profession. A professional body of knowledge is also generally conceived to be distinct from the core discipline of the profession it is based upon (Klingenberg, 2009). This is to say that professional

9

bodies of knowledge are generated from within professions, not from outside. Although knowing the legal aspects of medicine is essential to the practicing physician, the knowledge itself could be legitimately said to belong to the body of knowledge of law, not to medicine. Without a unique body of knowledge separate from those of other disciplines and professions, an occupation has no expert domain, and any claim it may make to professional status might ultimately lack legitimacy. What constitutes uniqueness, however, may be less the absolute requirement it first appears to be. It is well within reason to imagine many important pieces of knowledge that may be rightly claimed by more than one discipline’s body of knowledge, just as the preceding law/medicine example illustrates. As pointed out in the Introduction, absolute uniqueness is a difficult, if not impossible, standard for multidisciplinary fields to meet. Many of these fields may begin life as subfields within an existing discipline. Sociology and sociologists, for example, have played an essential role in the early history and development of disaster studies (Drabek, 2007). Would the development of a disaster studies discipline mean that the contributing work of these sociologists cannot continue to exist within both disciplinary bodies of knowledge? This seems patently absurd. Yet removing the requirement of uniqueness entirely would also appear to make the very idea of disciplinary bodies of knowledge meaningless. What is missing is to understand that uniqueness can exist along a continuum just as Greenwood suggested a continuum exists for whether an occupation is or is not a profession. When uniqueness is viewed in this way and applied to the present problem, it allows for parts of bodies of knowledge to be held in common between disciplines, so long as there is a much larger share of knowledge not held in common. In the case of multidisciplinary fields, they become more unique as the number of disciplines contributing to the body of knowledge increases, and as the production of distinct knowledge under the banner of the new field grows. For example, if the bulk of current disaster knowledge is found to still fall under the purview of sociology, or it is found to be held in common between only a few disciplines, then it would be right to conclude that the field of disaster studies is not yet a discipline. It remains a subfield/specialty within one or more other fields of study. If, however, we find that the knowledge in disaster studies is split across numerous existing disciplines, and we find there is also a growing, substantial part of its body of knowledge recognized as being unique to the field of disaster studies, then we could rightly conclude the field either is, or is on the verge of becoming, a discipline. Disciplinary Status and the Body of Knowledge in EDM The significance of connecting EDM to a unique body of knowledge is understood by many authors within the field. Jensen (2010), who is among those authors that believe that EDM is both an academic and professional discipline, states: “the body of knowledge that one has to know to be competent in emergency management is vast, specialized, and separate from any one other discipline” (Jensen, 2010, p. 15). Phillips (2003) while never directly placing the discussion within the context of the body of knowledge question, doubts that the debate in EDM regarding its disciplinary status has been settled, even though key figures like Mileti believe that the debate has been settled in favor of disciplinary status. Phillips, however, does seem to be optimistic that EDM is far closer to such status than it is distant. Lindell, Prater, and Perry (2006) see EDM as an emerging profession and repeatedly stress that a key goal for continued progress must be to “build and grow an identifiable body of knowledge for practitioners” (Lindell et al., 2006, p. 363). As mentioned in the Introduction, it is much easier to find opinions in the literature regarding EDM’s body of knowledge and disciplinary status than it is to find

10

empirical support for the opinions. If there is an EDM body of knowledge, it should be discoverable, identifiable, quantifiable, and qualitatively describable. Many fields/occupations, including massage therapy, geographic information science (GIS), and civil engineering, have formal Body of Knowledge (BOK) documents (Massage Therapy Body of Knowledge (MTBOK) Task Force, 2010; University Consortium for Geographic Information Science, 2006; American Society of Civil Engineers (ASCE), 2008). To date, there have been two attempts, one indirect and one direct, to detail the body of knowledge in EDM: the International Association of Emergency Managers (IAEM) Associate Emergency Manager (AEM)/Certified Emergency Manager (CEM) examination; and the Federal Emergency Management Agency’s (FEMA) Emergency Management Higher Education Program’s Body of Knowledge Project. IAEM AEM/CEM Examination The AEM/CEM examination is one of four elements in IAEM’s AEM/CEM credentialing program (IAEM, 2011; Lindell, Prater & Perry, 2006). The 100-item, multiple-choice exam includes countryspecific versions for the United States, Canada, Australia, and New Zealand, with sample questions provided in IAEM’s exam brochure (IAEM, 2011). Although a credentialing examination is not a body of knowledge, a well-designed, well-constructed professional credentialing or licensing exam is a reliable and valid sample of a professional field’s body of knowledge. Many professions have no formal BOK documents. It then becomes the credentialing/licensing examinations that establish, through their content and structure, the body of knowledge requirements practitioners must master. In the case of the AEM/CEM exam, there is little available information regarding test design and construction, question/item generation, and other important metrics of the exam. Lindell et al., who discuss the CEM program at great length, also fail to elaborate on the question of exam validity, only saying than it is “comprehensive” (Lindell, Prater & Perry, 2006, p. 359). The IAEM brochure suggests that applicants “’brush up’ on basic emergency management literature” (IAEM, 2011, p. 4), and that the “exam questions will focus on emergency management principles and practices reflected in the publications listed on the back page” (IAEM, 2011, p. 4). It is exactly the publications listed on the back page of the brochure that reveal the most about the relationship between the AEM/CEM exam and the EDM body of knowledge it purportedly draws from. The recommended publications that “may be used to make up all exams” (IAEM, 2011, p. 16) are FEMA Independent Study courses. Looking at country-specific recommendations, only Australia’s country-specific references includes books and journal articles. For the United States, Canada, and New Zealand, virtually every other reference cited is a law, policy document, or product of government agency. Thus, according to the AEM/CEM exam, a substantial part of the EDM professional body of knowledge can be found in FEMA’s online selfstudy training catalog. If studying FEMA training courses, public law, and public policy contains all there is to know in professional EDM, one might be led to question why there is any need for EDM undergraduate and graduate programs. With FEMA being identified as the primary source of the EDM body of knowledge rather than IAEM itself first identifying and creating the BOK for professional EDM, the certification examination as currently constructed does not appear to be a tool capable of validly illuminating the EDM body of knowledge. There is, however, another more direct effort, also supported by FEMA, to define the body of knowledge within the field and profession.

11

FEMA Higher Education Program Body of Knowledge Project In 2005, the FEMA Higher Education Program started the Body of Knowledge Project, a highly promising yearly survey of the academic and practitioner communities in EDM to identify the essential readings of the field/profession (Spiewak, 2005). The purpose of the program, as explained by Spiewak, is to assist in the development of an actual EDM BOK, which could also provide a source for updating IAEM CEM examination questions. The initial one-question survey asked respondents to name three books, articles, policy documents, etc. that they would recommend to others. Over 1000 surveys were sent; 326 usable responses were received; and the result was the top 50 reading recommendations. In addition to expected recommendations like the National Incident Management System, the Stafford Act, Emergency Management: Principles and Practices for Local Governments, and Disasters by Design, the 2005 survey included classic EDM works like Who Moved My Cheese, Rudi Guiliani’s Leadership, and Volcano: The Movie (Spiewak, 2005). Since 2006, the survey sample for each year has alternated between the EDM academic community and practitioners, the number of works for respondents to recommend has increased to 10, and the recommendations are ranked according to the number of respondent lists a work appears on. The 2009 edition (Cwiak, 2009a) also includes a summary of top selections from the 2006-2009 surveys, which is presented in Table 1. The 2011 edition (Cwiak, 2011) is the most recent, and includes 93 entries based on responses from 56 out of 133 (42%) surveys sent to EDM higher education programs. At the top of the 2011 list is Haddow, Bullock, & Coppola’s Introduction to Emergency Management, which was included on the lists of 12 respondents. The idea that there is value in an EDM recommended reading list cannot be faulted. Blanchard (2007; 2008) not only created his own top 50 reading list as a companion piece to the Body of Knowledge Project, but also created perhaps the most detailed bibliography of the field in existence, totaling 750 pages in length. FEMA also has its online ALL-HAZARTS database of EDM scholarly articles (http://www.lrc.fema.gov/allhazarts.html). The idea that a survey-based recommended reading list can be used to extract a body of knowledge, however, is rather naïve. If anything, the surveys in practice have shown a greater potential for revealing the field’s lack of awareness of its own body of knowledge. Cwiak (2009a) comments that the increasing repetition of certain works from year to year indicates growing consensus among respondents. This is likely true, but such consensus is also evidence of intellectual/technical stagnation within EDM. Examining the lists produced by the project, it is hard not to notice that although entire journals have been included, not a single specific journal article, white paper, or conference paper appears. Since journal articles and conference proceedings are often the preferred source for communicating empirical research and theoretical development within scientific and professional fields, this may be a sign EDM is failing to develop an infrastructure that supports integrating research and practice so that there is the eventual emergence of evidence-based EDM practice. The ultimate problem is deeper still. Not even creation of an infinitely comprehensive bibliography of emergency management can overcome the fact that a bibliography, although closer to the goal, is still not the same as a body of knowledge. EDM’s disciplinary body of knowledge (should it actually be shown to exist) lives within the bibliography, waiting to be extracted, analyzed, defined, detailed, organized, structured, and disseminated. Methods have emerged that could assist in this task, though it

12

Table 1 FEMA Body of Knowledge Project Top Recommended Readings, 2006-2009. 2009-Practitioner

2008-Academic

2007-Practitioner

2006-Academic

1. Principles of Emergency Management (Blanchard, et al.) 2. National Incident Management System (NIMS) 3. National Fire Prevention Association 1600 (NFPA 1600) 4. National Response Framework (NRF) 5. Disasters By Design: A Reassessment of Natural Disasters in the U.S. (Mileti) 6. Emergency Management: The American Experience (Rubin) 7. Emergency Management: Principles and Practice for Local Government (Drabek& Hoetmer) 8. Emergency Planning (Perry & Lindell) 9. Stafford Act

1. Emergency Planning (Perry & Lindell) 2. Introduction to Emergency Management (Haddow & Bullock) 3. Disasters by Design: A Reassessment of Natural Disasters in the U.S. (Mileti) 4. Emergency Management: Concepts and Strategies for Effective Programs (Canton) 5. Emergency Management: The American Experience (Rubin) 6. Introduction to Emergency Management (Lindell, Prater & Perry) 7. 9/11 Commission Report 8. Emergency Management Principles and Practices for Local Government (Waugh & Tierney) 9. At Risk: Natural Hazards, People’s Vulnerability & Disasters (Wisner, et al.) 10. Disaster Response and Recovery (McEntire) 11. Facing the Unexpected: Disaster Preparedness and Response in the U.S. (Tierney, Lindell, Perry) 12. Living with Hazards, Dealing with Disasters (Waugh) 13. NRF

1. Living with Hazards,Dealing with Disaster (Waugh) 2. Emergency Management: Principles and Practice for Local Government (Drabek & Hoetmer) 3. Disasters by Design: A Reassessment of Natural Disasters in the U.S. (Mileti) 4. FEMA IS 100/200, ICS 300, 400, 402 5. 9/11 Commission Report 6. NIMS 7. National Response Plan (NRP) 8. Stafford Act

1. Disasters by Design: A Reassessment of Natural Disasters in the U.S. (Mileti) 2. Introduction to Emergency Management (Haddow & Bullock) 3. Facing the Unexpected: Disaster Preparedness and Response in the U.S. (Tierney, Lindell, Perry) 4. Living with Hazards, Dealing with Disaster (Waugh) 5. 9/11 Commission Report 6. Disasters & Democracy (Platt) 7. NIMS 8. NRP

13

requires looking for methodological help from a very different discipline, that of Library and Information Sciences (LIS). This will be explored in the final section of the chapter. Bibliometrics, Co-citation, KDViz, and CiteSpace The application of mathematical and statistical methods to the analysis of bibliographic materials is an LIS specialty usually referred to as bibliometrics. Hood and Concepción (2001) have traced the earliest history of bibliometrics to the late 1800’s and early 1900’s. The term itself, although some believe it existed as early in French publications as early as the 1930’s, is generally held to have been coined and defined by Pritchard in 1969. The term is often used synonymously with the term “scientometrics”, which according to Hood and Concepción, was also coined in 1969, and applies bibliometric methods in analyzing the literature of science and technology. The two specialties are almost indistinguishable from one another most of the time, and where the distinction lies depends on whose definitions are being used. Börner, Chen, and Boyack (2003), see all scientometric research as bibliometric in nature, but not all bibliometric research is concerned with science and technology. Hood and Concepción, however, define scientometrics as the quantitative analysis of all outputs of science and technology, which includes bibliographic and non-bibliographic materials. For them, not all scientometric analysis is bibliometric, and not all bibliometric analysis is scientometric. This thesis accepts Hood and Concepción’s distinction, though as it applies to the present study, it can be correctly considered both bibliometric and/or scientometric, regardless of definition. For the sake of consistency, only “bibliometrics” and its variations will be used throughout the rest of the paper. Bibliometric methods were substantially advanced, as explained by Börner et al., by the birth of modern citation indexing and citation analysis, which can be traced to Eugene Garfield’s (1955) seminal article on citation indexing and its possible applications in science. Garfield would eventually establish the Institute for Scientific Information (ISI) and create the Web of Science (WoS) bibliographic database. Garfield and scientist Henry Small are generally considered the founding fathers of modern bibliometrics. Creation of map-based visualizations based upon the analysis of direct citations (when later papers or authors cite earlier ones) began as early as 1964, and soon other researchers were exploring the possibility of mapping the networks of science (Börner et al., 2003). Further groundbreaking in the field took place a few years later when Small (1973) advanced the idea of co-citation analysis. Co-citation analysis is similar to bibliographic coupling (examining the shared references among papers or authors), which had already joined direct citation analysis in the bibliometric toolbox. What makes co-citation different is the fact that it is an indirect citation relationship. Co-citation is defined as “the frequency with which two items of earlier literature are cited together by the later literature” (Small, 1973, p. 265). White and McCain also provide an excellent definition: “Co-citation occurs when any two works appear in the references of a third work. The authors of the two co-cited works are co-cited authors. If the cocited works appeared in two different journals, the latter are co-cited journals” (as cited in Börner et al., 2003, p. 11). If, in a sample of journal articles, Article 1 cites Articles C and D; then C and D are said to be cocited. If Article 2 also cites C and D, then C and D have been co-cited twice. The greater the frequency of co-citation occurrences, the stronger the likely connection or similarity between the co-cited items. Co-cited pairs, such as Articles C and D, are not required to have cited one another. Small suggested in his paper that co-citation, because it appears to measure the strength of an intellectual connection

14

between items, could be used to reveal and map the structure of scientific specialties, including how the structure changes over time. What can be revealed by co-citation, in Small’s view, are very much the invisible colleges proposed by Crane (1972), which are created by the invisible links between scientists and academics, and are key mechanisms of knowledge transfer and dissemination. Since the publication of Small’s paper in 1973, co-citation has become the primary form of bibliographic analysis for examining how scientific fields develop and are structured. Table 2 lists some of the many co-citation studies where the method has been used to visualize various scientific knowledge domains, including the entire domain of scientific knowledge over a single year. The most commonly used forms of co-citation includes analysis of cited authors (Author Co-citation Analysis, or ACA); cited articles (Document Co-citation Analysis, or DCA); and cited journals (Journal Co-citation Analysis, or JCA). ACA is the most frequently used form of co-citation analysis. More recently, co-citation analysis using WoS journal classifications has also emerged as a promising tool for mapping very large scientific domains (Vargas-Quesada, de Moya-Anegón, Chinchilla-Rodriguez, & González-Molina, 2006; Vargas-Quesada & de Moya-Anegón, 2007 ). Almost from the beginnings of citation analysis, visualization (in the form of mapping) of resulting citation networks appear as an integral part of analysis. All of the studies in Table 2 include both analysis and visualization. The capabilities of visualization and graphic technology in the early years of citation analysis, unfortunately, lagged some years behind advances seen in citation analysis methods. In fact, it would take until the later parts of the 1990’s for the pace of research and publication on information visualization, and more specifically knowledge domain visualization (KDViz) to quicken substantially (Börner et al., 2003). Since then there has been an explosion in visualization methods, procedures, algorithms, refinements, interpretations, evaluations, and visualization software packages (many of which are available freely online for research purposes). This has allowed for the simultaneous growth in KDViz studies of scientific domains, especially in the last ten years, which is also reflected in Table 2. In addition to the technical aspects of producing visualizations, research attention has also focused on other issues such as user interfaces, user interaction with visualized information, and aesthetic concerns (Chen, 2004). Within the KDViz research literature of the past fifteen years, one of the frequently recurring names is that of Dr. Chaomei Chen, of Drexel University. He has participated in, and researched most of the major developments of the field, including: methods of information visualization (Chen, 2004); methods of analysis and mapping of various scientific knowledge domains (Börner et al., 2003; Chen, 2003); identification and visualization of citation bursts (high rates of citation for a particular author, article, or journal during the first few years after publication, which can be an indication that the item is of particular importance to a field) using Kleinberg’s (2002) algorithm (Chen, 2006a, 2006b); visualizing latent domain knowledge (Chen, Kuljis, & Paul, 2001; Chen, 2003); quantitative measures of the quality of a visualized network (Chen, 2005); and most recently, use of KDViz in the analysis of multidisciplinary fields (Chen, Hu, Liu, & Tseng, 2012). Chen, like many in LIS, is influenced by Thomas Kuhn’s landmark work on the nature of scientific discovery, The Structure of Scientific Revolutions (Kuhn, 1962/1996), and his work takes special interest in the use of KDViz to identify key turning points in scientific knowledge domains (Chen, 2003, 2006a). Chen believes these visualized turning points may indicate the paradigmatic shifts in scientific fields referred to by Kuhn.

15

Table 2 Selected Co-citation/Knowledge Domain Visualization Studies STUDY

DOMAIN(S) ANALYZED

White and McCain (1998)

Information Science, 1972-1995

Chen & Paul (2001)

Hypertext; Computer Graphics; Virtual Environments

Boyack (2004)

20 years of Proceedings of the National Academy of Sciences (PNAS)

Chen (2004)

Superstring Field in Physics

Mane & Börner (2004)

PNAS, 1982-2001

Boyack, Klavans, & Börner (2005)

Entire structure of science

Synnestvedt, Chen, & Holmes (2005)

Medical Informatics

Chen (2006a)

Mass Extinction; Terrorism

Vargas-Quesada, de MoyaAnegon, ChinchillaRodriguez, & GonzalezMolina (2006)

Co-citation analysis and mapping of Web of Science categories for all indexed articles produced in the United States in 2002; all records of scientific literature produced in China in 1990 and 2002

Reid & Chen (2007)

Terrorism research

Vargas-Quesada & de Moya-Anegon (2007)

Category co-citation of world scientific literature, 2002; comparison EU and US domains, 2002; evolution of Spanish scientific literature 1990-2002

Dwivedi, Lal, Mustafee, & Williams (2009)

Analysis of articles published in Information Systems Frontiers, 19992008

Zhao & Wang (2011)

Ubiquitous Computing

Chen, Hu, Liu, & Tseng (2012)

Regenerative Medicine

Rorissa & Yuan (2012)

Information Retrieval

16

All of these various interests and developments find full expression in CiteSpace and the most recent version, CiteSpace II, the open access, Java-based, Windows and Linux-compatible KDViz software program created by Chen, and widely used in KDViz research (Chen, 2003, 2004, 2006a, 2006b, 2011). The program has also performed extremely well in comparisons to other KDViz applications (Cobo, Lopez-Herrera, Herrera-Viedma, & Herrera, 2011). CiteSpace can either be downloaded or started directly from the CiteSpace download page (http://cluster.ischool.drexel.edu/~cchen/citespace/ download.html). Figure 1 shows a screenshot of the program’s two main control panels. CiteSpace is a progressive KDViz program capable of dividing a time period under study into smaller time slices that can be displayed in a variety of formats, including merged networks representing the entire study timeframe; or a series of individual visualizations of each time slice (Chen, 2004). This allows the temporal evolution of co-citation networks to be visualized. Other visualization formats are also offered. CiteSpace is primarily designed for analyzing bibliographic data obtained from the Thomson Reuters’ Web of Science (WoS), though several other data formats are supported to different degrees. CiteSpace offers multiple types of citation analysis, including Author, Document, and Journal co-citation analysis. The user is given numerous options for configurations and settings to tailor/finetune analyses and visualizations to need. CiteSpace also uses a distinctive approach to network visualization, one that utilizes a “tree ring” node design, as well as size and color, to represent citation frequencies, link strengths, years, and significant metrics values. This approach allows a single cocitation network visualization to incorporate large amounts of quantitative and temporal information. The most important of these visual concepts are shown in Figure 2. Although a full explanation of all of the features in CiteSpace is beyond the scope of this paper, interested readers are directed to the CiteSpace website (http://cluster.cis.drexel.edu/~cchen/citespace/), which includes links to user guides, tutorials, Wiki, and videos. One of the primary reasons why Chen created CiteSpace was to research the identification of emerging research areas and trends in scientific literature (Chen, 2003, 2006a). This is in keeping with his interest in scientific turning points and Kuhn’s philosophy of science. As discussed in Synnestvedt, Chen, and Holmes (2005), and Chen (2006a), work by Price showed there is a structural relationship between the source articles used as data in co-citation analysis (the citing articles) and the references that are contained within those articles(the cited references) . The set of citing articles during a given time period with high citation counts represent the research fronts of the discipline, those emerging, transient, areas of current research, some of which may continue to grow into significant new disciplinary directions, and others which will not bear fruit and will be abandoned. The cited references of these articles establish the intellectual base of the discipline, which is the prior literature that supports the research fronts. This intellectual base is what is directly visualized in KDViz, and this finding is the basis for the present research. Although discovering disciplinary research fronts is an important part of KDViz, the answers to the research questions in the present study are to be found in the intellectual base, not the research fronts. Work by Persson, discussed by Chen (2006a, 2006b), found that intellectual bases are quite stable over time; and that lowering the co-citation thresholds used in an analysis results in the expansion of the visualized intellectual base. The observed characteristics of disciplinary intellectual bases in KDViz are exactly the characteristics one would expect to observe in disciplinary bodies of knowledge. Thus, the intellectual bases found in KDViz are equivalents, or near-

17

Figure 1. Screenshot of CiteSpace II Control and Visualization Panels. The program can be intimidating at first, and takes substantial time to learn. Available documentation explains the basic program functions adequately, but the use of many advanced programs functions must be gathered from the research literature.

18

Figure 2. Basic visualization concepts used in CiteSpace II. CiteSpace’s use of size and color allows a large amount of important information to be conveyed within a single visualization.

19

equivalents, of disciplinary bodies of knowledge. CiteSpace should therefore be a means of visualizing the body of knowledge within a discipline, if it exists. Finally, a word of caution needs to be given to any readers tempted to immediately download CiteSpace and begin their own KDViz research projects. CiteSpace is a powerful, interesting, and extremely useful KDViz software program. But it should be remembered that CiteSpace is research, not commercial software. This author has found that for someone coming to the software from outside the world of LIS and KDViz, it can also be intimidating and at times, highly frustrating. The User’s Guide (Chen, 2011) shows that a basic analysis can be conducted in a set of six steps. Once a basic analysis and visualization is completed, however, understanding and interpreting the results is never directly discussed in depth in the documentation. More importantly, one can complete a basic analysis without understanding if the program settings are appropriate for the intended research purpose. Much of this material is absent or incompletely present in CiteSpace’s documentation, and must be determined as best as possible from the research literature, as well as through a substantial amount of trial and error. Ultimately, this requires a major investment of time and mental energy to learn not only operation of the program, but also the underlying aspects of bibliometrics, citation analysis, and network analysis CiteSpace is based upon. These comments are not meant to in any way to lessen the value and usefulness of the program, for these issues appear to be present in many, if not all, KDViz applications available. Potential researchers coming from outside LIS and KDViz, however, should have realistic expectations of the learning curve required.

20

CHAPTER III: METHODOLOGY The idea for the present study first originated during the spring of 2012, when the author accidentally discovered KDViz and CiteSpace II while performing online research into the connection between bodies of knowledge and ontologies. Dataset construction took place during September and October, 2012, during which time there were numerous trial runs of CiteSpace II as part of the process of learning the software and determining the thresholds to be used in the final analyses. Multiple analysis runs on the dataset to finalize CiteSpace II settings, and to further refine the quality of the dataset took place during late October, 2012. The final dataset was completed and the final analyses and visualization runs presented in this paper were conducted during early and mid-November, 2012. All analyses were conducted on the author’s desktop computer running on an AMD Phenom II X4 830 2.8 GHz, processor with 8 GB RAM, and Windows 7 64-bit operating system. Design and Rationale One of the difficulties in developing a KDViz study is that there is no singular analysis and visualization method or rationale to be found that can be turned to for guidance. The research literature can be used to provide possible models of study design, and the literature on KDViz presented in Chapter II was used as an essential resource. There are some frequently used methods, such as only using in a co-citation analysis those dataset articles that have received at least 5-7 citations. Traditionally this has been used to find out who those that are cited most in a field are themselves referencing. Thus, it helps focus knowledge domain visualizations on those most likely to be influential within their disciplines. Although this may be quite useful in a well-established scientific discipline, would it still hold true for a small, multidisciplinary, emerging candidate discipline that may not have the same research and publication pressures present in other disciplines? As it is believed that the answer is in fact “no”, and because there may be other ways in which the nature of EDM might need to be taken into consideration, research proceeded under four guiding principles/assumptions: 1. EDM is much smaller than even many of the specialties previously examined by KDViz, and its multidisciplinary nature may further decrease the likely number of citations articles receive, particularly in any EDM specialties that may exist. 2. The professional side of EDM does not have the tradition of research and publication common to academic/scientific disciplines, and many professional disciplines like medicine and clinical psychology. Again this is a factor that would contribute to relatively low citation rates in some parts of the field. This would also suggest that citation totals may or may not necessarily represent how important or influential something is to the field. 3. Selection of articles for the dataset should come from a broad spectrum of possible core disaster-related journals, rather than potentially influencing results with preconceptions of what EDM should be by only searching for a particular term or using a single journal as the source of data. 4. Larger, more complex, and “messier” co-citation network visualizations will be preferable in this study to smaller, more elegant appearing networks.

21

The purpose of the study is to first visualize and understand as much of the EDM body of knowledge and its structure as possible. A leaner network visualization, although this may be useful in future studies, may likely not provide the full picture required to answer questions of disciplinary status and structure. CiteSpace II threshold settings therefore should generally favor as much as possible, enlarging rather than contracting the intellectual base visualized (recall the work of Persson discussed in Chen, 2006a, 2006b). This would need to be balanced with the practical limits of how large a network visualization CiteSpace II and the computer hardware used can handle before system performance/stability is substantially diminished; as well as the amount of time required for the program to process the analysis and produce a visualization. Because KDViz has not been attempted before in EDM, all three forms of co-citation analysis (ACA, DCA, and JCA) would be included. Initially, an analysis and visualization of the Web of Science (WoS) subject categories of the citing articles was also to be conducted, but it was discovered, as confirmed by Vargas-Quesada and de Moya-Anegón (2007), that multidisciplinary journals are classified by WoS under multiple categories, regardless of the subject matter of the article (e.g. an article on earthquakes in Natural Hazards would be classified under all three of the journal’s WoS subject categories—Geosciences, Multidisciplinary; Meteorology and Atmospheric Sciences; and Water Resources). Any co-citations to the journal co-cite to all three subject categories. Many of the journals included in the dataset are affected by this problem, and it fatally affects the validity of results obtained unless each article is individually reassigned to a single WoS subject category based on article subject matter. Because of the additional effort and substantial time this would require, the analysis was eliminated from the research plan. It was also eventually decided that the three primary analyses were sufficient enough in themselves, and a plan to visualize the country affiliations of first authors in the dataset was also dropped. Having established the basic principles/assumptions of the research approach, and the types of analysis to be included, attention turned to acquisition and construction of the dataset. Dataset Construction Data for the study comes from the WoS bibliographic database. Access to WoS is generally only available to institutional subscribers, such as university libraries and research centers. Access to WoS through the researcher’s home institution was not available, but visitor access was successfully obtained through the University of Texas at Dallas McDermott Library. Prior to accessing WoS, an online list of WoS indexed journals was reviewed and eleven EDM-related journal titles were identified for research inclusion. Additionally, based in part on the findings of Comfort, Waugh, and Cigler (2012), Public Administration Review and Administration and Society were selected for conducting a search for EDM articles. For the eleven journal titles, all articles and proceedings papers indexed in WoS between 1986 and 2011 were exported in the complete format (which includes cited references) as a series of text files. Multiple text files were required, as WoS limits the number of records that may be exported at once. The two public administration journals were searched in WoS for any articles or proceedings papers between 1986 and 2011 containing “emergency management”, or “disaster” in the title, subject, or

22

keyword fields, and the records were also exported to file. Appendix A provides a sample of what the WoS data looks like in its original form. The individual text files containing the WoS data were loaded into a duplicate removal tool that is part of CiteSpace II. The tool also reorganizes the articles into new WoS text files organized by date. Only articles and proceedings papers were included for analysis. Examination of these files revealed that many of the proposed study years had 10 or fewer articles, and that there was a distinct drop in number of articles for years prior to 1994. This drop is seen in the distribution of dataset articles shown in Figure 3. For this reason, the research time period was established at 1994-2011. Test runs began being conducted to learn and test both CiteSpace II and the data. It was during this process that a significant problem with the dataset emerged, one that would take up enormous amounts of time and effort to resolve. If one looks closely at Appendix A, and specifically the cited references section of the original WoS record (abbreviated as “CR”) it may be noticed that there is great inconsistency in how the references are presented, including capitalization, punctuation, and even in how a name or title is spelled or abbreviated. This problem was first discussed in the Introduction section of this paper. These discrepancies will result in each variation of an author’s name, journal title, or document, being treated by CiteSpace II as separate entities, which will affect not only citation totals but also the construction of networks. There is an alias function within CiteSpace II that is supposed to, in theory, allow such variations to be assigned to a primary alias. Unfortunately, this function is barely documented in the CiteSpace literature, and this author was unable get the function working properly. The eventual solution decided upon was to edit the dataset, but this cannot be done using a regular word processing application, for WoS text files are in Unicode and follow a specific format. If the correct formatting is altered, which can happen if normal word processing programs are used, it may result in the data being unreadable to CiteSpace II. A moderate amount of online searching discovered the open source Unicode text editor, BabelPad (West, 2012). BabelPad was used first to reassemble the individual files of the dataset into a single file, containing approximately 185,000 lines of text. This single data file would allow faster editing. As a first step, all fonts used in the data were converted to uppercase. Then, a general system was established for prioritizing the correction of the dataset. Results of trial CiteSpace II runs were used to prioritize by highest citation frequencies which authors, journals, and documents would be given attention first. A format was created for standardizing any variations discovered. The standard format for author names used the last name, one space, then any initials without spaces or punctuation. Journal titles were standardized by their WoS abbreviation. For book and government documents, a standardized title was simply decided upon. Not all items were changed to standardized versions: names and titles were left “as is” if no variations were discovered. Correction was a slow and painstaking process, and required first searching for a particular name variation, changing all instances of that variation to the standard version, then looking for the next variation. On numerous occasions internet searching was required to identify items when doubt emerged (e.g. D. Alexander might refer to D.E. Alexander or D.A. Alexander, and this type of problem often required using a Google search to locate

23

Figure 3. Distribution of articles in dataset by year. Although it may be tempting to say, the rapid growth in number of articles cannot be wholly attributed to growth of the field. Such conclusions are confounded by the fact that several journals in the dataset did not begin being indexed by Web of Science until 2004 or later. This graph also shows the substantial difference between the number of articles found in 1993 and in the years following. Because of the relative paucity of pre-1994 articles, the study period was set at 1994-2011.

24

the work cited and full author name). The data would then again be run in CiteSpace II to look for more variations in the results. This process continued over three to four weeks before it was felt the dataset was sufficiently corrected to move on preparing for the actual research analysis runs. However, anytime afterwards, if a variation was discovered, even if it only a single instance, the dataset would be corrected, and if an analysis had been completed, it would be rerun with the most recently corrected dataset. This did happen during JCA, when it was discovered that certain journal abbreviations in the data that were thought to refer to different journals were discovered, in fact, to be the same journal. No claim is made that every single variation in the data has been found and corrected. It is believed, however that these variations have been sufficiently corrected to make the research results valid. One additional data issue emerged that also requires mention, as it also raises a very interesting question. During Cite Space II trial DCA runs, it was noticed that the classic work, At Risk, appeared as two distinct entries with different authors. When this was investigated, a previously unrecognized fact emerged: the first and second editions of At Risk have different first authors. Blaikie is first author of the 1994 edition, and Wisner is first author of the 2004 edition. The question became whether to leave the data as is, or to consider both editions of At Risk as representing a single work. This issue would also affect the JCA, because both journals and books were to be included in the JCA. Either decision would produce slightly different analysis results. After consideration, it was decided for the purposes of this research to consider both editions as a single intellectual entity. As a result the dataset was copied and all citations within it to At Risk were changed to rename “Blaikie-Wisner” as the author, and “1994”as the date, as test runs indicated the 1994 edition was the more frequently cited of the two editions. This dataset was labeled as “DCA/JCA Data” and the original first dataset renamed “ACA Data”. These are the two sets of data that would be used for complete and final co-citation analysis and visualization runs that are presented in this research. The ACA Data file was loaded into CiteSpace II’s database utility to provide a summary of the dataset. Table 3 shows the composition of the final dataset in terms of journals and number of articles. All citing articles from 1994 to 2000 come from either Natural Hazards or Disasters. The other eleven journals began being indexed in WoS at varying times from 2001 onward. Table 4 details the characteristics of the final dataset. It is considered somewhat amazing that the 2385 citing articles used in the analyses contain over 77,000 cited references. The enormous number of cited references also shows the enormity of the task encountered in attempting to correct the naming variations. For comparison, in Zhao and Wang’s (2011) study of the emerging, multi-disciplinary field of ubiquitous and pervasive computing, 5,914 citing articles were identified between 1994 and 2009. Chen, IbekweSanJuan, and Hou’s (2010) study of information science from 1996-2008 found: 10,853 records in 12 source journals that were cited 45,453 times; 8,408 unique authors; and 129,060 unique cited references that were cited a total of 206,180 times. Figure 4 shows the distribution of total citations received by the citing articles, according to WoS. As expected, much of the field does not generate a large number of citations, with over half of the citing

25

Table 3 Journals Used in the Dataset JOURNAL

YEARS

Biosecurity and Bioterrorism: Biodefense Strategy, Practice and Science Disaster Advances Disaster Medicine and Public Health Preparedness Disaster Prevention and Management Disasters Environmental Hazards: Human and Policy Dimensions Global Environmental Change: Human and Policy Dimensions Journal of Homeland Security and Emergency Management Natural Hazards Natural Hazards and Earth System Sciences Natural Hazards Review Public Administration Review Administration and Society

# ARTICLES

2004-2011

29

2008-2011 2007-2011 2009-2011 1994-2011 2009-2010

62 163 86 439 14

2003-2011

32

2006-2011

155

1994-2011 2004-2011 2009-2011 2001-2011 2006-2011

1230 74 56 36 9

TOTAL SOURCE ARTICLES IN DATASET :

2385

Table 4 Summary of Dataset Characteristics

Number of Source Articles

2385

Number of Authors

7402

Number of Institutions

4595

Number of Keywords

23,793

Number of Cited References Total Citations Received

74,774 14,352

26

Figure 4. Distribution of dataset articles’ WoS times cited (TC). The 2385 articles in the dataset have received a total of 14,352 citations (as of October 2012), which is a citation rate of 6.0 citations per article. This citation rate is slightly above the average citation rate in the social sciences (4.0-5.0) but lower than that within many of the natural and behavioral sciences (The Times Higher Education, 2011). Approximately 53% of articles have received 2 or fewer citations. The non-citation rate (percentage of articles receiving no citations) is 24.8%, which is consistent with rates found within the social sciences, but slightly higher than rates found in the natural and behavioral sciences.

27

articles from the field of EDM receiving 2 or fewer citations, though the number of citations per article is slightly above the rate found in the social sciences, but lower than that found in natural sciences (The Times Higher Education, 2011). Finally, Table 5 shows the ten citing articles with the highest number of total citations, according to WoS. Interestingly, the ten articles represent only three journals: Natural Hazards, Disasters, and Global Environmental Change-Human and Policy Dimensions. Pre-Analysis CiteSpace II Settings Prior to the final complete analyses runs of the dataset in CiteSpace II, the settings to be used in the final analyses were determined through a combination of documentation reading, research literature review, and conducting test runs to examine the effects of particular settings by trial and error. There are several important settings in CiteSpace II that must be determined before running a co-citation analysis. Basic Program Set-Up 3 CiteSpace II was started by first opening the download page in Google Chrome, available at http://cluster.ischool.drexel.edu/~cchen/citespace/download.html. The “WebStart (JVM XMX)” option was used rather than downloading the program. The program is available with several memory configurations. The 1.0 GB memory configuration was chosen for use in this research. Once CiteSpace II had started, a new project, named “EMDisDataRun”, was created. Another folder was created in Windows and named “FULLANALYSISFINAL”. This folder was designated in CiteSpace II as the “Project Home”. The Project Home is where the program will automatically export analysis files. A data folder directory must also be designated as the location for CiteSpace II to look for the data to analyze, so a new folder was created for this use and to store the final two (ACA and DCA/JCA) versions of the dataset. The majority of the project settings were left at their default values, with some exceptions: Enable Export Matrices (allows the full co-citation matrix to be exported as a comma separated value file) was set to “on”; Include GP (Group Author) was set to “on”; and Include ED (Editors) was set to “on”. Within the FULLANALYSISFINAL folder in Windows, several additional folders were created as locations to store the results and outputs of each analysis, as any files CiteSpace II automatically exports all go to the folder named as the Project Home. It is highly recommended that the same or similar procedure be used, for having all of the outputs from multiple analyses in a single folder can result in confusion, disorganization, and potentially copied-over analysis files. After each analysis was completed, any important files from that analysis were moved into the correct analysis folder (ACA, DCA, or JCA).

A new build of CiteSpace II was released on November 23, 2012. This version adds some user-defined settings and options that are not present in the May 24, 2012 build that was used for the analyses in this thesis. Interested readers can open or download the newer build of CiteSpace II and click “What’s New” under the Help menu to see a list of the changes. Most of the changes are in the Set-Up Options, and the major pre-analysis settings (Time Slicing, Thresholds, Pruning, etc.) discussed in the next few pages remain unchanged, 3

28

Table 5 Ten Most Cited (per WoS) Source Articles in Dataset TIMES CITED

359 278 156

132 114

102

100

83

78 74

ARTICLE DETAILS WOS:000239752200006 SMIT, B. ( 2006) ADAPTATION, ADAPTIVE CAPACITY AND VULNERABILITY GLOBAL ENVIRONMENTAL CHANGE-HUMAN AND POLICY DIMENSIONS (16), pp. 282-292 WOS:000239752200005 ADGER, W. (2006) VULNERABILITY GLOBAL ENVIRONMENTAL CHANGE-HUMAN AND POLICY DIMENSIONS (16), pp. 268-281 WOS:000229514100008 BROOKS, N. (2005) THE DETERMINANTS OF VULNERABILITY AND ADAPTIVE CAPACITY AT THE NATIONAL LEVEL AND THE IMPLICATIONS FOR ADAPTATION GLOBAL ENVIRONMENTAL CHANGE-HUMAN AND POLICY DIMENSIONS (15), pp. 151-163 WOS:000082705900004 RICKENMANN, D. (1999) EMPIRICAL RELATIONSHIPS FOR DEBRIS FLOWS NATURAL HAZARDS (19), pp. 47-77 WOS:000187068600013 CHUNG, C.J.F. (2003) VALIDATION OF SPATIAL PREDICTION MODELS FOR LANDSLIDE HAZARD MAPPING NATURAL HAZARDS (30), pp. 451-472 WOS:000089616200002 GALADINI, F. (2000) ACTIVE TECTONICS IN THE CENTRAL APENNINES (ITALY) INPUT DATA FOR SEISMIC HAZARD ASSESSMENT NATURAL HAZARDS (22), pp. 225-270 WOS:000080736000005 FOTHERGILL, A. (1999) RACE, ETHNICITY AND DISASTERS IN THE UNITED STATES: A REVIEW OF THE LITERATURE DISASTERS (23), pp. 156-173 WOS:000086011000002 CARRARA, A. (1999) USE OF GIS TECHNOLOGY IN THE PREDICTION AND MONITORING OF LANDSLIDE HAZARD NATURAL HAZARDS (20), pp. 117-135 WOS:000225185700008 TINTI, S. (2004) THE NEW CATALOGUE OF ITALIAN TSUNAMIS NATURAL HAZARDS (33), pp. 439-465 WOS:A1995RP08200005 FOWLER, A.M. (1995) POTENTIAL IMPACTS OF GLOBAL WARMING ON THE FREQUENCY AND MAGNITUDE OF HEAVY PRECIPITATION NATURAL HAZARDS (11), pp. 283-303

Note. WOS= Web of Science identification number.

29

Time Slicing The research timeframe of 1994-2011 had already been established during data construction. CiteSpace II, however, as a progressive visualization program, requires the user to determine how to divide up the timeframe under study into slices of N years per slice. The most complete and detailed visualizations will be networks composed of one year slices. The size and complexity of the network decreases as the timeframe is compacted and analyzed in 2-year, 3-year, 4-year, etc. slices. For studying very long periods of time, time slicing can be essential, or the complexity and size of the network may exceed the visualization limits of software and hardware, or simply take too much time to produce. In keeping with this study’s principle of favoring a larger rather than smaller network visualization, time slicing was set at 1 year per slice. Threshold Controls Among the most important settings involve those that directly control how the co-citation network is constructed. CiteSpace II provides the user several ways of exercising this control. One way is by specifying a minimum and maximum number of times cited (TC) a citing article must have for it to be included in the analysis. This will result in a smaller network that represents the network produced from citing articles within the specified citation range. The TC threshold can also be used together with other threshold methods. While TC affects the citing articles, the other thresholds control which cited references will be included in the network. At the simplest level, the top N cited references or items can be chosen from each time slice; or the top N% from each slice can be chosen (with the ability to set an absolute upper limit to the number of items per time slice). At the most complex level, Threshold Interpolation can be used. This allows user control over the three threshold values that determine the co-citation network: the citation threshold (C), which is the number of citations a cited item must have before it is included; the co-citation threshold (CC), which is the number of co-citation occurrences an item must have before inclusion; and the co-citation coefficient value threshold (CCV), which is the minimum value of the normalized (using Salton’s cosine) co-citation frequency, also referred to as the link strength, required for inclusion of the link in the network (Chen, 2006b). These three values can be set for the first, middle, and last time slice. However, threshold values are only required to be set for the first and last time slice, as CiteSpace II will then automatically interpolate the threshold values used in all other time slices. Threshold Interpolation is the method preferred by Chen (2006b). The ability to individually adjust the different thresholds allows the researcher to adjust the values according to the needs of the study, or characteristics of the data. Figure 5 and Figure 6 show the effect different threshold values have on the number of nodes and links in several ACA co-citation networks generated using the final dataset. As a point of reference regarding the effects the number of nodes and links have on system performance, the author’s experience was that once the number of nodes exceeded 1000, and number of links approached 10,000, system and graphic processing became increasingly sluggish. At very high levels of nodes and links, although it may only take several seconds for the program to process the basic co-citation calculations, it can then

30

take extremely long periods of time (> 10 minutes) for the visualization to appear, and the processing and graphical strain of such visualizations makes working with them difficult. Threshold Interpolation was eventually chosen, after trying each of the other methods, for use with this study. It was chosen because of the finer adjustment of network size it allows. Thresholds (C-CCCCV) for all analyses were set at 2-2-0.30 for the start of study period; 3-3-0.30 for the middle; and 44-0.30 for the end of the study period. These values were chosen empirically by trying different values and observing the effects on overall network size, number of nodes and links in each time slice (which CiteSpace II calculates and displays when running an analysis), length of time required for a visualization to be produced, and system performance. This is, in fact, the basic method recommended by Chen (2006b). Because of the number of articles in the dataset grew over the years, threshold values were gradually increased over time. This also has an advantage of setting a preference in the more recent study years for references that have shown faster citation growth than others, which can be an indicator of the reference’s importance. For all analyses, TC limits were set (min= 0 and max= 359) so that all citing articles would be included. Network Pruning Algorithms and Network Visualization Display CiteSpace II allows the user to choose using one of two well-known network pruning algorithms, if desired: Pathfinder or Minimum Spanning Tree (Chen, 2004, Chapter 4; Chen 2006b; Samoylenko, Chao, Liu, & Chen, 2006) Which visualizations the algorithm is applied to can also be specified: the individual time slices, the merged network that shows the entire study timeframe, or both, can be pruned. These mathematically-based algorithms are designed to simplify networks by removing likely extraneous or non-salient nodes and links. A more detailed description of these algorithms is beyond the scope of the current study. No pruning algorithms were selected for any of the analyses. It was felt that these network algorithms, in addition to adding another layer of complexity to the research, would contract, rather than expand, the visualized intellectual base. These algorithms, however, are important in KDViz, and would likely play a role in future EDM applications of KDViz. Users must also select, prior to running the analysis, from two ways of displaying the visualization. A set of network visualizations for each time slice can be produced, or a synthesized, merged network can be produced that encompasses in a single visualization the co-citation network during the entire time period under study. As the individual time slice display would result in a total of 57 individual network visualizations when all three analyses were completed, the merged network visualization was selected for use in all three co-citation analyses.

31

Figure 5. Threshold settings in CiteSpace II: number of network nodes. The number of nodes in a visualization can be controlled through adjustment of the cited reference citation (C) and co-citation (CC) thresholds; and the citing articles’ times cited (TC) threshold. This graph shows the effects of these settings on the number of nodes in Author Co-Citation Analysis (ACA) visualizations.

32

Figure 6. Threshold settings in CiteSpace II: number of links. These results show the effects of different settings on the number of links in ACA visualizations. The of number of links in a visualization can be controlled by changing the cited references citation (C), co-citation (CC), and co-citation coefficient value (CCV) thresholds; and the dataset source articles times cited (TC) threshold. The CCV threshold setting has no effect on the number of nodes in a visualization.

33

Other Settings and Summary Additional pre-analysis settings include the link strength and link scope settings. These are settings that govern more technical aspects regarding how the number of co-citation occurrences are determined and normalized (which becomes the indicator of the strength of the link). The link scope setting was left at its default value of “within slice” for all analyses. Link strength (which becomes the measure of the similarity between co-cited items, and can be determined by several methods) was set to be determined by Salton’s cosine, which is the default. Salton’s cosine is one of the most commonly used methods to normalize co-citation frequencies, and produces a value from 0 to 1, with 0 indicating no co-citation link, and 1 being the highest measure of a co-citation link’s strength/similarity (Chen, 2006a, 2006b). Table 6 summarizes the important pre-analysis settings. The table also provides details as to the number of citing articles used in the analyses, as well as details as to how many of the cited references were able or unable to be used in the co-citation analysis (valid/invalid). High numbers of invalid references can indicate an error in the dataset, or other issue in need of attention. As can be seen, over 99% of cited references were valid and used in the analyses. Quantitative and Qualitative Analysis Procedures Each of the three final analysis runs were completed one at a time in CiteSpace II, in the following order: ACA; DCA; JCA. When performing an analysis, the program first completes the basic calculations, and these processes and their results for each time slice can be observed through a “status space”, and a “progress report” window on the main control panel. This basic processing normally takes only a few seconds, and the status windows allow the processing to be checked for signs of data problems (e.g. number of records reported processed does not match the number of records actually in the dataset) CiteSpace II then begins processing the visualization, and this process can take several seconds up to several minutes, or longer depending on the size of the network. Each of the co-citation network visualizations in this thesis took approximately 5-6 minutes before they were ready for use. Although it is conceivable to start a new co-citation analysis during this delay, which would allow the analyses to be run in quick succession, having visualizations from different analyses running simultaneously was judged unwise due to the extra demands it would place on computer resources. For each analysis, the same basic procedures were followed: 1. The correct version of the dataset (ACA or DCA/JCA) was copied into the data directory folder; the co-citation analysis to be performed was selected in the control panel; all settings for the analysis were verified. 2. Analysis was started and results of the basic processing were checked to ensure validity. 3. Once the network visualization appeared, it was allowed to go through a rapid series of iterations until it reached a stable form. The user must manually stop the visualization process once the network form stabilizes. In this study the visualization process was stopped for each analysis at approximately 1200 iterations.

34

4. Citation burst detection was performed. This produce the appearance of red “burst indicators” in some nodes, indicating a sudden rise in the number of citations at some point in the study period. 5. CiteSpace II’s cluster analysis routine was selected; once this was completed all network details and metrics were recorded; the visualization and related network summary tables were saved to file. 6. The evolution of the network year by year was inspected using the “Link Walkthrough” feature in the visualization control panel; images were saved (both CiteSpace II and screenshot software (Snagit) were used for recording images) at different points of time in the walkthrough to show the development of the network across the entire study period. 7. Labels were added and modified using a variety of different available display options; multiple network images with different label and display options were saved. 8. The visualization, with and without labels, was examined for possible clusters of similar content. These areas were noted and saved for further work after all co-citation analyses were completed. 9. Once it was felt sufficient images had been obtained, the visualization was closed and the next analysis started. It was very important to ensure that all images had been obtained and saved before the visualization was closed because it was discovered during trial runs that even when a saved visualization is used, the appearance of the network will not remain precisely the same each time. The exact position of nodes will change but the relationship of nodes and links, and the network metrics will, remain unchanged. This makes sense considering that a network visualization is exactly that: a visualization of a network solution based on algorithms that can have multiple correct solutions so long as it does not violate the results of the analysis calculations. The difference in appearance is less when using the saved visualization than if the entire analysis is run again. From a practical standpoint, this means that if a visualization has to re-opened, it will need to be re-labeled and an entire new set of images saved. 10. Once all co-citation analyses were completed, further qualitative analysis of the visualizations continued in the same order as the original analyses: ACA; DCA; JCA. For each visualization, saved network images were opened in Photoshop for editing, and areas of possible similarity noted earlier were examined again, using internet searches where necessary to identify the possible relationship between particular authors, articles, and journals. These areas were then labeled and the images saved as a new file.

35

Table 6 CiteSpace II Analysis Settings ANALYSIS ACA

DCA

JCA

Total Source Records=2385 Citing Article Inclusion Criteria: Minimum TC = 0 Analysis Years/Time Slicing

1994-2011 / 1 year per slice

Similarity Measure INTERPOLATION THRESHOLDS

Salton’s Cosine Start

C= 2 CC= 2 CCV= 0.30

Middle

C= 3 CC= 3 CCV= 0.30

End

C= 4 CC= 4 CCV= 0.30

Pruning Algorithm

None

Visualization Displayed

Source Records in Range

Maximum TC= 359

Merged Network

2369

2368

2369

Valid References

74,707 (99.91%)

74,642 (99.91%)

73,197 (99.91%)

Invalid References

65 (0.09%)

65 (0.09%)

65 (0.09%)

Note. TC= total number of times article cited per Web of Science; C= minimum citation threshold; CC= co-citation threshold; CCV= co-citation value/coefficient threshold (maximum = 1.0).

36

CHAPTER IV: RESULTS In presenting the results of the analyses, the reader is reminded that the purpose of the present research is evaluate the applicability and success of KDViz in revealing the structure of EDM. The purpose is not an attempt to establish a quantitative EDM ranking system of importance or quality. The methods and data utilized in this study are not sufficient to justify any such statements. Certain works, authors, articles/books, and journals, however, are well-known for their importance, and one would expect any list to represent this fact. If this study finds that this is the general case, then it supports the ability of KDViz to present a broadly accurate picture of the field. Asking whether a specific “Author A” or “Article B” should be ranked higher than some other author or article distracts from the more relevant question of whether “Author A” or Article B” are generally regarded to be important to the field. General Results Table 7 displays the overall results for each of the analyses, including number of nodes and links; density; number of nodes with a citation burst; and cluster analysis results. The density of a network is defined as the number of direct links in a network divided by the number of total possible direct network connections (Kadushin, 2012). The highest possible density, therefore, is 1.00. Density is affected by network size, and as the size increases density will normally decrease. Lower densities are associated with less-centralized, dispersed networks. The values shown in Table 7 can be compared for reference to the network density of 0.083 found in Chen, Hu, Liu, and Teng’s (2012, Figure 3) study of regenerative medicine. Density results in this study are consistent with a de-centralized, quite dispersed network. As has been mentioned previously, citation bursts are rapid, statistically significant, increases in citation frequency, and are detected using Kleinberg’s (2002) algorithm (Chen, 2006a). Nodes with the red-colored citation burst rings may have special significance. CiteSpace II provides a quantitative burst strength value, as well as information on the duration, and the year(s) in which the burst occur, which can assist in discovering the significance of the burst. Bursts will not be directly analyzed in this study, but figure indirectly, as busts are used in the calculation of sigma, which will be discussed momentarily. The reader should also be aware of their appearance and recognize the red burst rings in the visualizations. CiteSpace II’s cluster analysis routine calculates the modularity Q, the silhouette, and number of clusters found. The cluster analysis routine used, and the metrics calculated, are explained in Chen, Ibekwe-SanJuan, and Hou (2010). The cluster analysis method attempts to divide the network into non-overlapping clusters as opposed to overlapping clusters. Modularity Q, which can range from 0 to 1, measures whether the network can be reduced to modules with distinct boundaries. The silhouette, which can range from -1 to 1, is a measure of how much uncertainty there is in determining the nature of the clusters. A silhouette value of 1 represents a network where every cluster is completely separate from every other cluster. According to Chen and colleagues, silhouette values greater than 0.7 are desirable (p. 8). As can be seen from Table 7, CiteSpace’s cluster analysis routine was unable to separate the ACA and JCA networks into non-overlapping clusters, but was able to do so with the DCA. The DCA silhouette also falls within the range of desirability. The results would appear to indicate that

37

Table 7 CiteSpace II Network and Cluster Analysis Results NETWORK

CLUSTER STATISTICS # BURSTS

# CLUSTERS

Mean Silhouette

Network Analysis

Nodes / Links

ACA

1078/ 7728

0.0128

111

1

0.0018

∞

DCA

601 / 1953

0.0107

50

95

0.6626

0.7846

JCA

934 / 7211

0.0165

131

1

0.0022

∞

Density

Modularity (Q)

in the case of the ACA and JCA, there is, as might be expected due to the multi-disciplinary nature of EDM, significant overlap in possible clusters, and another method of cluster analysis may be better suited. In the individual co-citation analysis results and visualizations that follow, there are several other metrics the reader should be aware of, which are reviewed below. Betweenness centrality. This measures the degree to which a node is in the center of the shortest path that connects two nodes, with higher centrality meaning the node has more of these shortest paths passing through them (Chen et al., 2010; Kadushin, 2012). This makes these nodes like “gateways” connecting parts of a network to other parts. The highest possible centrality value is 1. In CiteSpace II visualizations, nodes with centrality scores greater than 0.10 are displayed surrounded by a pink ring, with higher centrality denoted by the ring thickness. Chen and colleagues (2010) note that items with high centrality are more likely to represent revolutionary works in science. Sigma. This measure of scientific novelty was introduced in 2009 by Chen, Chen, Horowitz, Hou, Liu, and Pellegrino (cited in Chen et al., 2010), and a revised version is incorporated into CiteSpace II. The measure takes into account both centrality and citation burst strength to identify significant contributors and contributions to scientific fields, so that sigma is highest for items with both high centrality and high burst strength, rather than just one of the values. To calculate sigma, (centrality +1) is raised to the power of the burst strength. Total Citations (Unique Instances). CiteSpace II co-citation analyses display the citation frequencies based on the unique instances of the author and journal. This means that if an author or journal is referenced in the same article four times, it is only counted as one instance. This does not affect the totals displayed in the DCA, as a specific document can only be referenced once. This method

38

of calculation reduces the impact of any self-citation, or one article citing large number of works from a single author, though some authors, like Harzing (2010), doubt whether self-citation is of any real concern in citation totals. The method also places emphasis on the scope of influence, rather than the sheer total of citations received. Total Citations. In the case of ACA, CiteSpace II’s database function includes a query that allows for all author citations in the dataset to be tallied and exported to a comma separated value file. Results from this query will also be presented for comparison. Author Co-Citation Analysis (ACA) Authors were subdivided into two groups: individual and corporate/agency authors. Table 8 displays the number of total citations received by the top 15 individual authors, and Table 9 displays the total unique citation instances for the same group. The listing of corporate/agency authors are displayed in Table 10 (total citations) and Table 11 (unique instances). For unique citation instances, CiteSpace II also calculates an average year of citation, which can be used to help place the author and his/her contributions within historical context. For individual cited authors, the majority of authors should come as little surprise, as they are wellknown in the EDM world. Some authors, however, like Adger (vulnerability and adaptation), Ambraseys (earthquake engineering), and Anderson (complex emergencies and aid issues) may not be so recognized. Also unsurprisingly, among cited corporate/agency authors, FEMA and the Centers for Disease Control and Prevention (CDCP) top the list. United Nations-related branches and initiatives, including the World Health Organization (WHO) the Intergovernmental Panel on Climate Change (IPCC), are also, perhaps somewhat surprisingly well-represented. When looking between the results of total citations versus unique instances, it would appear that the lists are fairly similar, with some slight differences. When the two groups of authors are combined, and organized by sigma, which is presented in Table 12, the results are substantially different. Only one governmental author (Department of Homeland Security (DHS)) appears now on the list. Many of the well-recognized EDM authors from the citation total lists, though not all, are no longer at the top of the list, while known authors not on those lists appear. D.E. Alexander, Drabek, and Waugh are here. This list includes more authors from different disaster-related areas: including Ambraseys (earthquake engineering); McGuire, Gutenberg, Grunthal, and Aki (seismology); Murty (seismic sea waves/tsunamis); Varnes (landslides); Macrae and Anderson (complex emergencies and aid issues); and Auf der Heide (disaster medicine/disaster planning). It is important to note, however, that the sigma values here are rather small across the board, and the numerical differences between authors is not very large. For comparison, in Chen, Hu, Liu, and Tseng (2012), the five highest sigma values found in their analysis of regenerative medicine literature had sigma values of 15.97 at the lowest, and 377,340.00 at the highest. The entire ACA visualization is shown in Figure 7. Within the visualization there are largely two halves. The half composed of blue and green links that developed in the early part of the study period contains authors associated largely with the scientific study of natural hazards. The other half composed of yellow and orange links that developed in the later parts of the study period, contains a large cluster

39

Table 8 ACA: Fifteen Most Cited Individual First Authors, 1994-2011(All Citations) CITES 151 150 145 134 128 124 119 115 112 110 109 105 104 97 94

AUTHOR S.L. Cutter D.S. Mileti E.L. Quarantelli M.K. Lindell N.N. Ambraseys W.L. Waugh D.E. Alexander W.N. Adger S.A. Changnon K. Hewitt B. Wisner T.E. Drabek L.K. Comfort P. Blaikie R.W. Perry

Table 9 ACA: Fifteen Most Cited Individual First Authors, 1994-2011 (Unique Instances) CITES 122 95 93 93 90 88 77 70 68 64 59 59 58 56 54

AUTHOR, YEAR* D.S. Mileti, 1999 E.L. Quarantelli, 1996 P. Blaikie, 1985 D.E. Alexander, 1993 B. Wisner, 2000 S.L. Cutter, 2003 N.N. Ambraseys, 1982 M.K. Lindell, 1992 T.E. Drabek, 1986 K.J. Tierney, 1995 W.L. Waugh, 1996 L.K. Comfort, 1999 W.N. Adger, 2003 R.W. Perry, 1979 M.B. Anderson, 1996

Note. In this and other tables where it appears, YEAR indicates the average year of author’s works cited.

40

Table 10 ACA: Fifteen Most Cited Agency Authors, 1994-2011 (All Citations) CITES 230 196 140 123 123 119 116 113 98 96 91 70 67 63 56

AUTHOR Federal Emergency Management Agency Centers for Disease Control and Prevention World Health Organization Dept. Homeland Security United Nations Development Programme World Bank United Nations International Strategy for Disaster Reduction Intergovernmental Panel on Climate Change Government Accountability Office United Nations International Federation Red Cross/Red Crescent National Research Council National Oceanic and Atmospheric Administration U.S. Geological Survey United Nations High Commissioner for Refugees

Table 11 ACA: Fifteen Most Cited Agency Authors, 1994-2011 (Unique Instances) CITES 152 92 89 87 85 82 81 79 76 63 59 54 54 45 41

AUTHOR, YEAR Federal Emergency Management Agency, 1997 United Nations Development Programme, 1991 Centers for Disease Control and Prevention, 1991 World Bank, 1993 World Health Organization, 1989 Intergovernmental Panel on Climate Change, 1996 United Nations International Strategy for Disaster Reduction, 2002 Department of Homeland Security, 2006 United Nations, 1984 International Federation Red Cross/Red Crescent- 1996 U.S. Government Accountability Office, 2003 National Research Council, 1994 U.S. Geological Survey, 2003 U.S. National Oceanic and Atmospheric Administration, 2001 U.S. Census Bureau, 2000

41

Table 12 ACA: Top Fifteen Authors (Combined) Ranked by Sigma, 1994-2011 SIGMA 2.42 1.98 1.87 1.62 1.49 1.37 1.37 1.36 1.31 1.29 1.27 1.21 1.19 1.19 1.18

AUTHOR, YEAR N.N. Ambraseys, 1982 J. Macrae , 1992 D.E. Alexander, 1993 M. Duffield, 1996 R.K. McGuire, 1976 M.B. Anderson, 1996 T.S. Murty, 1977 B. Gutenberg, 1949 T.E. Drabek, 1990 G. Grunthal, 1993 W.L. Waugh, 1996 D.J. Varnes, 1978 E. Auf Der Heide, 1989 K. Aki, 1980 U.S. Dept. of Homeland Security, 2006

Note. Sigma measures possible novel scientific contribution, taking into account betweenness centrality and burst frequency (Chen, Ibekwe-SanJuan, & Hou, 2010).

of authors usually associated with “emergency management” as well as authors associated with climate change. A close-up of the upper left quadrant, which includes part of the largest grouping of authors is shown in Figure 8. Within this area are found the authors generally associated with EDM, but there are secondary groupings also visible. The upper section contains a significant cluster of public administration authors (containing most of the authors identified by Comfort et al. (2012)), as well as U.S. government agencies. The lower area is composed largely of authors from other social sciences, such as sociology and geography, as well as U.N.-related bodies. Two authors are indicated with centrality values of 0.1 or greater: D.E. Alexander (0.18), and N.N. Ambraseys (0.15), indicating these two authors act as important gateways between the two main halves of the network. Results of the qualitative examination and labeling of the various author groupings found in the visualization is presented in Figure 9. As can be seen, there appears to be an underlying organization of multiple regions where authors can be associated with particular areas of study/interest. These areas, at their broadest, represent three primary inputs into the field: the medical and health sciences, the social sciences, and the earth sciences. Within these broad areas further narrowing by topic/subject can be seen. Areas do overlap, particularly as one attempts to narrow the topic/subject area, and the borders are indistinct. The evolution of the network is shown in Figure 10. A large core of authors were already established by the beginning of the study period. Development of areas related to study of natural

42

hazards, complex emergencies, human systems, and vulnerability are established generally earlier than the areas of disaster medicine/public health, and traditional emergency management. These areas show greatest growth in the latter years of the study period.

Figure 7. Author Co-citation Analysis (ACA) merged network, 1994-2011. Size of labels and nodes are proportionate to the number of unique citations received by the author. Remember that link and node center color indicate specific years: dark blue to light blue = 1994 to 1999; light green to dark green = 2000 to 2006; and light yellow to dark orange = 2007 to 2011. Notice that the nodes and links in the lower (predominately green) section, corresponding to scientific hazard research, were generally established and cited mostly in the years prior to 2004. Much of the upper half (predominately yellow/orange/brown colors), corresponding to sociology, human ecology, and public administration inputs to the discipline, shows greatest citation development after 2004.

43

Figure 8. Close-up of core concentration of authors in ACA network. Link transparency has been increased so nodes can be seen more clearly. e upper center/left contains a large number of authors (Birkland, Comfort, Kapucu, Lindell, Waugh, etc.) closely associated with public administration and emergency management, particularly as applied in the United States. Also in close proximity are several key U.S. government entities associated with emergency management: FEMA, GAO, and Centers for Disease Control and Prevention (CDCP).

44

Figure 9. ACA network visualization with topic/subject/concept labels. e labels were determined by identifying particular authors’ subject areas, contributions, and/or works, using internet searches where necessary. It became apparent early on in the process that there is an underlying organization to the network, with authors generally grouped according to their interests or associations. e expression of the organization is not perfect, for many authors have multiple areas of interest. is fact may also account for why only one cluster (the entire ACA network) could be identified by CiteSpace II. Interestingly, CiteSpace II, identified the word “disaster” as the most likely descriptor for the networkcluster, based on analysis of the citing articles’ keywords, using log-likelihood ratio.

45

1994

1998

2002

2007

2009

2011

Figure 10. Evolution of ACA network, 1994-2011. Gray areas are unformed parts of the network. Light tan depicts network links formed in previous years. Dark tan/brown shows network links formed in that year.

46

Document Co-Citation Analysis (DCA) Table 13 shows the articles and books with the highest citation totals during the study time period. Table 14 shows the results when ordered by sigma value. . A great many of the works listed when citation totals are considered are well-known, such as At Risk and Disasters by Design, which top the list. Other works, such as those by Turner et al., Cornell, Guzetti, and Wells and Coppersmith, are possibly less well known. When sigma value is considered, there is a significant difference, though again the overall difference in values between works (apart from Cornell) is relatively small. The Table 13 DCA: Fifteen Most Cited References, 1994-2011 CITES 149a 71 41 34 30 29

28 26 25 25 25 25 23 23 22

CITED REFERENCE Blaikie, P., & Wisner, B. (1994). At Risk: Natural Hazards, People’s Vulnerability and Disasters Mileti, D.S. (1999). Disasters by Design: A Reassessment of Natural Hazards in the United States Cornell, C.A. (1968). Engineering seismic risk analysis. Bulletin of the Seismological Society of America (58), 1583-1606. Tierney, K.J. (2001). Facing the Unexpected: Disaster Preparedness and Response in the United States Cutter, S.L. (2003). Social vulnerability to environmental hazards. Social Science Quarterly (84)2, 242-261. Wells, D.L., & Coppersmith, K.J. (1994). New empirical relationships among magnitude, rupture length, rupture width, and surface displacements. Bulletin of the Seismological Society of America (84), 974-1002. Turner, B.L., et al. (2003). A framework for vulnerability analysis in sustainability science. Proceedings of the National Academy of Sciences (100)14, 8074-8079. Cutter, S.L. (1996). Vulnerability to environmental hazards. Progress in Human Geography (20)4, 529-539 Guzzetti, F. (1999). Landslide hazard evaluation: a review of current techniques and their application in a multi-scale study, Central Italy. Geomorphology (31)1-4, 181-216 Hewitt, K. (1999). Regions of Risk: Geographic Introduction to Disasters. Hewitt, K. (ed.) (1983). Interpretations of Calamity from the Viewpoint of Human Ecology United Nations Development Programme (2004). Reducing Disaster Risk: A Challenge for Development United Nations International Strategy for Disaster Reduction (2004). Living with Risk: A Global Review of Disaster Reduction Initiatives U.S. House Select Bipartisan Committee (2006). A Failure of Initiative... Pelling, M. (2003). e Vulnerability of Cities: Natural Disasters and Social Resilience

is includes both editions of At Risk (the 1994 first edition with Blaikie as first author, and the 2004 second edition with Wisner as first author). a

Table 14

47

DCA: Top Ten Cited References Ranked by Sigma, 1994-2011 SIGMA

CITED REFERENCE

4.08

Cornell, C.A. (1968). Engineering seismic risk analysis. Bulletin of the Seismological Society of America 58, 1583-1606.

1.48

United Nations Development Programme (2004). Reducing Disaster Risk: A Challenge for Development

1.22

Cannon, T. (1994) Vulnerability analysis and the explanation of "natural" disasters. In A. Varley (ed.) Disasters, development and the environment , 13-30

1.18

Mccrae, J., et al. (1997). Conflict, the continuum and chronic emergencies: A critical analysis of the scope for linking relief, rehabilitation and development planning in Sudan. Disasters 21(3), 223-243

1.09

Carrara, A. et al. (1995). GIS technology in mapping landslide hazard. In A. Carrara and F. Guzetti (eds.) Geographic Information Systems in Assessing Natural Disasters (Advances in Natural and Technological Hazards Research #5), 135-176

1.09

Pielke, R.A., Gratz, J., et.al (2008). Normalized hurricane damage in the United States, 1900-2005. Natural Hazards Review 9, 129-42

1.09

Waugh, W.L. (2006). Collaboration and leadership for effective emergency management. Public Administration Review 66(S1), 131-140

1.09

Comfort, L.K. (1999). Shared risk: Complex systems in Seismic response

1.09

Wise, C.R. (2006). Organizing for homeland security after Katrina: Is adaptive management what’s missing? Public Administration Review 66 (3), 302-318

1.09

McKenzie, D. (1972). Active tectonics of the Mediterranean region. Geophysical Journal of the Royal Astronomical Society 30 (2), 109-185

noticeable lack of At Risk on the list may be an artifact of combining the two editions in the analysis, which may have impacted the burst strength. At Risk has the highest centrality (0.30) observed in any of the three analyses, marking it as a key work. It is quite possible that the two lists are like two different lenses, which offer slightly different perspectives, and that both total citations and sigma might best be used in conjunction. This matter will be taken up at the beginning of Chapter V. Figure 11 shows the central part DCA network visualization, and Figure 12 shows a detail of part of the super-cluster at the center. The entire network is quite dispersed, and a great many clusters (many of which are too far separated to be displayed in the visualization image shown) lie unconnected to each another, and unconnected to the center super-cluster. This has implications that will be discussed in

48

Chapter V. The center super-cluster is dominated by Blaikie et al.’s At Risk (centrality = 0.30) and Mileti’s Disasters by Design (centrality = 0.09). Again, as in the ACA, articles appear to have thematic groupings, with the articles associated with traditional EM occupying a different location than those on vulnerability and risk analysis, for example. These thematic areas are shown in Figure 13, with a closeup of this section of the cluster shown in Figure 14. Even though the network is more dispersed, the same general thematic and subject matter areas can be found in the DCA that were identified in the ACA. Figure 15 shows the evolution of the DCA network over the study period. Again, like the ACA, areas associated with hazard study develop in the early years of the study period, with emergency management areas developing in the last few years of the study.

Figure 11. Document Co-citation Analysis (DCA) merged network, 1994-2011. Notice the dispersion of the network, with many clusters of articles not linked to the central super-cluster (link strength is below the analysis threshold of ccv=0.30). e visual dispersion seen, combined with the number of clusters found (= 95) indicates significant clusters of research potentially lying outside the awareness of others in the field, even those who are conducting research in the same area. e visualization is dominated by the center super-cluster, which includes the two largest nodes: At Risk (center) and Disasters by Design (right center).

49

Figure 12. Close-up of lower right quadrant of center cluster in DCA visualization. Similar to what was seen in the ACA visualization, a large number of public administration scholars’ emergency management articles are clustered within one particular area of the visualization.

50

Figure 13: Super-cluster at center of DCA visualization, with labels. Again, there is a general underlying logic to the network, though in detail some similar topics are scattered in different locations. For example, the cluster containing articles by Fuchs et al. (2007) and Keiler et al. (2006) on avalanche risk assessment might be expected to be on the same side as Landslides. However, as Fuchs et al.’s paper concerns the empirical expression of vulnerability in avalanche risk analysis, this may explain why it is located closer to articles related to risk and vulnerability.

51

Figure 14. Close-up of densest part of DCA network, with labels. Area contains a majority of the applied social science input into the field. Again, a slight separation in the organization appears between the articles involving emergency/crisis management more to the lower right, and those that deal with fundamental disaster concepts such as vulnerability, sustainability, and resilience to the left center.

52

1994

2004

2009

1998

2007

2010

Figure 15. Evolution of the Document Co-citation Network, 1994-2010. A large proportion of the network formed in the second half of the study period, between 2004 and 2010.

53

Journal Co-Citation Analysis (JCA) Table 15 displays the journals with the highest citation totals in the JCA. Natural Hazards (608) and Disasters (390) top the list, with International Journal of Mass Emergencies and Disasters (132) being the highest ranked of the journals specifically targeting emergency management. The Journal of Homeland Security and Emergency Management (not on the list) was cited 60 times, with a more recent average year of citation (2004) than the Australian Journal of Emergency Management, which was also cited 60 times, with a slightly earlier average year of citation (2000) . Journals devoted to scientific hazard study are well represented, with Natural Hazards and Earth System Sciences (2003) and Natural Hazards Review (2000) having the most recent average year of citation on the list. Disasters is the journal with the highest sigma value, as is shown in Table 16. The International Journal of Mass Emergencies and Disasters and Public Administration Review figure more prominently when sigma is considered, though as in the previous analyses, the difference in sigma between journals is relatively small. The entire JCA visualization is shown in Figure 16. Once again, an underlying structure appears, with hazard-related journals appearing more in the upper part of the visualization and social science and health/medical journals in the lower section, with further organization by thematic area possible. Two journals have significant centrality values: Disasters (0.15) and Lancet (0.11). As has been seen in the previous analyses, there is consistency in the thematic groupings: a variety of earth science-related disciplines surround Natural Hazards; disaster medicine is represented in the area to the left of Disasters; complex emergencies, aid, and humanitarian relief is represented just below Disasters; and journals related to EDM occupying the lower right quadrant of the visualization (Figure 17). Just above this section is another rather distinct area occupied by diverse fields from both natural and social sciences, as represented by their journals, which suggests they belong to a slightly different thematic group. This area is shown in Figure 18. The visualization, labeled with all of these thematic groupings, will be presented and discussed in Chapter V. The evolution of the JCA network is shown in Figure 19. The upper half of the visualization was already well-established at the beginning of the study period. The area corresponding to complex emergencies developed mostly during the early years under study, though some development of the disaster medical section is seen. This area, along with the EDM sections of the visualization, developed largely in the last few years of the study period.

54

Table 15 JCA: Twenty-five Most Cited Journals/Book Series, 1994-2011 (Unique Instances) CITES 608 390 294 265 222 202 175 168 164 164 160 158 152 151 146 144 141 134 132 129 127 126 123 116 115

YEAR 1992 1990 1977 1986 1966 1990 2003 1962 2000 1991 1979 1995 1994 1989 1993 1994 1993 1991 1984 1960 1988 1994 1991 1993 1998

JOURNAL/BOOK SERIES TITLE Natural Hazards Disasters Science Bulletin of the Seismological Society of America Nature Geophysical Research Letters Natural Hazards and Earth System Sciences Journal of Geophysical Research Natural Hazards Review Pure Applied Geophysics Tectonophysics Engineering Geology Disaster Prevention and Management Journal of Geophysical Research-Solid Earth Geomorphology (Zeitschrift Fur Geomorphologie) Risk Analysis Bulletin of the American Meteorological Society International Journal of Geophysics International Journal of Mass Emergencies and Disasters Monthly Weather Review Earthquake Spectra Water Resources Research JAMA-Journal of the American Medical Association Climatic Change Journal of Hydrology

55

Table 16 JCA: Top Ten Journals/Book Series Ranked by Sigma, 1994-2011 SIGMA 2.79 1.95 1.74 1.48 1.45 1.27 1.23 1.21 1.21 1.20

JOURNAL/BOOK SERIES Disasters Bulletin of the Seismological Society of America Nature Journal of Geophysical Research-Solid Earth Institute of Development Studies Bulletin Tectonophysics International Journal of Mass Emergencies and Disasters Public Administration Review Bulletin of Volcanology Journal of Geophysical Research

56

Figure 16. Journal Co-citation Analysis (JCA) merged network, 1994-2011. The JCA includes journal articles and books. Labels are proportionate to total number of citations. The axis of the network is formed by the two journals cited most frequently: Natural Hazards and Disasters. Disasters and Lancet (just to the left of Disasters), are also nodes with high betweenness centrality, indicating that they are significant “access points” to parts of the network. Disasters is linked to journals across the visualization, and links the human science areas to the hazard sciences. Lancet connects other areas to the medical/health aspects of disasters.

57

Figure 17. Close-up of lower right section of JCA visualization. Once again a large number of emergency management, public administration, political science, and management-related journals/books are clustered in one general area. Almost all of the co-citation links in this area were formed between 2007 and 2011.

58

Figure 18. Close-up of center right section of JCA visualization. Above the emergency management/public administration area is this area which is similar but not the same as the lower section. Here is where diverse fields such as climatology, hydrological science, atmospheric science, sociology, geography, ecology, environmental studies, international development, and economics come together in studying the complex relationships between climate change, hazards, humans, and disasters. It is an area where risk, vulnerability, and adaptation appear to be central concepts.

59

1994

1998

2001

2007

2009

2010

Figure 19. Evolution of JCA network, 1994-2010.

60

CHAPTER V: DISCUSSION AND CONCLUSIONS General Findings of the Analyses The results of three analyses of the EDM bibliographic dataset show an underlying structure to the EDM body of knowledge that can be considered to represent a discipline. It is a discipline that is extremely broad in terms of its multi-disciplinary inputs, but relatively small in terms of its citation production patterns. Inputs to the field come from the basic and applied earth sciences, the basic and applied social sciences, and the health and medical sciences. The low density of the networks produced and the general lack of extremely high centrality or sigma scores could be interpreted as it has by some like Comfort et al. (2012) as showing that no one perspective dominates the field. Although this is correct, the large number of contributing disciplines, and the dispersion seen in the networks, can also mean there is a large amount of “latent” knowledge present in the field. Latent knowledge is knowledge relevant to a field that exists outside the awareness of researchers (Chen, Kuljis, & Paul, 2001). The same invisible colleges spoken of by Crane (1972), responsible for disseminating scientific and academic knowledge, if poorly linked to one another, become isolated pockets of knowledge, and academic and scientific progress can slow. Results of the analysis also show that some caution should be exercised in the use of sigma as an alternative, rather than complimentary method of finding key works in the field. In EDM, the total citation totals appeared to represent these key works somewhat better than the results produced by sigma. A possible explanation for this would be that because the network visualized in this study is larger and less dense, smaller sigma values will be obtained (due to the fact that large, low density, networks will have fewer high centrality nodes) than those obtained when pruning algorithms are applied to the network (Chen et al.’s (2012) study of regenerative medicine, for example, used the Pathfinder algorithm) Thus, for EDM, the citation totals might therefore be a better metric at the present time, supplemented by the sigma value, for identifying significance, until further study of the utility of sigma in EDM is completed. Research Hypotheses and Questions There were five research hypotheses proposed in this study. The five hypotheses, and the results of the research are: 1. Co-citation analysis and KDViz are approaches ideally suited, so long as methodological limitations are understood, to provide unique views of the possible intellectual structures and relationships within EDM that have not previously been detected. The visualizations produced by KDViz have, in fact, presented a view of the entire field and its body of knowledge not previously available. It has brought out an intellectual structure, and relationships within that structure, not previously observed. The relationship of hazard sciences to areas traditionally thought to be EDM has been brought out in these results. Results are consistent with what little citation analysis has been done in the field, such as the clustering in the ACA of public administration authors in Comfort et al.’s (2012) research.

61

2. The sources (authors, articles, and journals) that are the foundation of EDM are vastly larger in size and scope; are more diverse in their multidisciplinary origins; and are more poorly integrated into the field than possible realized.

As seen in the size and density of the visualized EDM networks, this hypothesis has been shown true. The body of knowledge visualized includes fields diverse as geophysics, seismology, atmospheric sciences and climatology, civil engineering, structural engineering, sociology, geography, human ecology, international development, public administration, political science, business, management, medicine, public health, epidemiology, and others. These diverse disciplines are linked to one another through a very large but dispersed network of disciplinary and multidisciplinary journals, such as Disasters, Natural Hazards, International Journal of Mass Emergencies and Disasters, Disaster Prevention and Management, Public Administration Review, Lancet, and Journal of the American Medical Association, among others. 3. Analysis and visualization will reveal an identifiable disciplinary structure to a larger knowledge domain than what is commonly thought of as EDM. The larger knowledge domain will refer to that which has been called many names, but in this thesis will be called the discipline of Disaster Studies and Sciences. EDM do not represent the entirety of the field, but instead are subdomains or specialties within the larger knowledge domain of Disaster Studies and Sciences. Writers like Jensen (2010) have suggested the term “Emergency Management” be used to refer to the entire field of disaster-related studies. It is believed that these analyses and visualizations offer evidence against such conclusions. The results show that the entire discipline is larger and more inclusive than the topics normally considered to be part of emergency management. The entire discipline has been shown to include elements of the basic and applied earth sciences, basic and applied social sciences, and medicine and public health. Emergency Management and its related-specialties (crisis management, response management, business continuity, etc.) are but one arm of a larger discipline. This topic will be further discussed in the next section, where the title “Disaster Studies and Sciences” will be proposed as representing the nature of the entire discipline. 4. Uncertainty in the field regarding a professional body of knowledge in EDM will be reflected in the visualizations. Visualizations are hypothesized to show structure for EDM as an academic discipline within the larger field of Disaster Studies and Sciences, but less so for evidence of a structure of the practical application of that knowledge, which should be expected in the knowledge structure of a profession. Professions should show a distinction between the basic science of the field that underlies the profession, and the knowledge that is part of the practice of the profession. In medicine, for example, there is a discrete knowledge structure for the branches of basic medical sciences and for the branches of clinical medical practice. Such structure is anticipated to be poorly realized, if at all, in EDM.

62

This hypothesis has been generally confirmed. The body of knowledge revealed in the body of knowledge appears to represent the academic body of knowledge within the discipline of Disaster Studies and Sciences, including the specialty of Emergency Management. The articles, books, and authors found in the analysis (with the possible noticeable exception of seismic engineering, where the work of several engineers are prominent) appear to generally be the product of the academic/scientific members of the discipline, not the professional practitioners. Thus the academic foundation of emergency management as a body of knowledge appears well-established, the link between this body and applied research into its professional practice appears weaker, but is in need of further study and analysis. These results would appear to confirm that the concerns of authors like Neal (1993) and Quarantelli (1993) of how research and practice could be integrated are still valid. One could make an argument that the prominent presence of FEMA, DHS, CDCP, and other governmental bodies, represent the integration of research and practice. As pointed out by Reid (1997), however, in her analysis of terrorism research, some concern should also accompany the significant involvement of government bodies in supporting or promoting research and/or practice, for it potentially jeopardizes the independence and objectivity of the knowledge produced. 5. When compared to other attempts to identify a professional EDM body of knowledge, such as FEMA’s yearly Body of Knowledge Survey, KDViz will show a gap between the actual knowledge domain that exists and the knowledge domain presented by the survey. It is this author’s contention that this hypothesis has also been shown true by revealing a greater scope and depth to the discipline than found in FEMA’s Body of Knowledge Survey by including more of the discipline’s authors and literature, including hazard sciences and other specialties, not captured by the Body of Knowledge survey. Perhaps more concerning is some divergence between core works as revealed in this analysis and what is revealed in the surveys. In the 2006-2009 surveys, At Risk made it into the top of the list only once, and in the 2011 survey it was selected by only 2 of the 56 higher education programs responding (Cwiak, 2009a, 2011). Yet At Risk is far and away the most cited work in the discipline, as revealed in this analysis. Other works, such as Reducing Disaster Risk: A Challenge for Development, also figure more prominently in this analysis than they do in the surveys. This discrepancy could be explained in two ways: 1) the surveys represent a specifically American point of view of emergency management; and 2) the surveys represent works specific to EM rather than the underlying discipline of disaster studies of which EM is a part. Both of these explanations can be true simultaneously, and there is evidence within each of the co-citation network visualizations to support the view. It is hoped this KDViz study might broaden the understanding of the larger discipline beyond any one national view or any one specialty. Also, using the results of KDViz analysis in conjunction with the Body of Knowledge Survey, it is possible to begin specifying within the discipline’s body of knowledge those works that are fundamental works foundational to disaster studies from those specific to academic EM and the professional practice of EM. This in fact would begin to give to professional EM more of the disciplinary structure it currently lacks. In addition, KDViz provides a means to relatively easily perform a yearly bibliographical analysis of the discipline and its various specialties, including EM.

63

There were also three primary research questions posed in this study: 1. Is EDM a suitable area for application of co-citation analysis and KDViz? If it is, what does it reveal about the possible structure and organization of EDM? Co-citation analysis and KDViz has shown itself suitable for revealing the underlying structure of EDM, which is shown to be a complex discipline with inputs from a large number of scientific and academic disciplines. 2. Does the analysis reveal evidence of a significant academic and/or professional EDM body of knowledge; and how does KDViz compare to other attempts to determine the key works, journals, and trends in the EDM body of knowledge, such as the FEMA Higher Education Program Body of Knowledge Project? As discussed above, co-citation and KDViz appears superior to methods currently utilized to identify the significant authors, works, and journals in the field, both historically, and on an ongoing basis. The possibilities present in the method can also be further refined through further research. 3. How do the findings of the analysis relate to questions of disciplinary and professional status in EDM? The present research supports a view that there is a body of knowledge connected to an academic discipline that for the purposes of this paper, is called Disaster Studies and Sciences. Within this larger discipline there is a specialty of Emergency Management, with a distinct academic body of knowledge. This body of knowledge, however, is not yet sufficiently linked to a professional body of knowledge, and understood by practitioners as such, for Emergency Management to be rightly considered a profession. Emergency Management, however, can correctly be viewed as an emerging profession. Creating a Disciplinary Framework All of the co-citation networks visualized in this study have shown evidence of a remarkably similar underlying structure. Taken together, this entire structure is asserted here to represent the discipline of Disaster Studies and Sciences. Using the qualitative analysis of journal groupings found in the JCA visualization, it is possible to identify and label this structure. Figure 20 shows the JCA labeled as a representation of the discipline and some of its specialty areas. The discipline is composed of two branches, here named the Hazards Branch and the Human Dimensions Branch. Within each branch are a variety of specialties and subspecialties, many of which are shown in Figure 20 as well as Figure 21, which removes the visualization to provide a more conceptual view. The structure presented here may provoke some disagreement, as it moves certain disciplinary inputs and their authors, including sociology, geography, and human ecology, for example, away from Emergency Management, and gives to them their own area of the discipline, the Human Systems-Hazard-Environment Interactions specialty, which is close to the Integrative Core of primary concepts, theories, and technologies. This structure, repeated in the ACA, DCA, and JCA, does appear to better represent the intellectual structure than the

64

wholesale inclusion of all things, and all authors, to the realm of Emergency Management. The inclusion of complex emergencies and humanitarianism may not please some in the discipline; while the inclusion of hazard-oriented studies and sciences, may be seen by others as a step backward (see Quarantelli, 2005b), rather than as the reconciliation between two estranged siblings this author believes it to be. The framework offered here is also a preliminary working model, one which it is hoped will spark discussion and additional research to refute or refine the structure. This study is, however, believed to be an important initial step in attempting to find order and structure in a field and its body of knowledge that has so far defied such efforts. It is further hoped that the methods of co-citation and KDViz used in this thesis will be undertaken by others to bring further fruits to the discipline. Recommendations for Further Study The present study represents only an initial step in terms of the analysis and visualization possible using the discipline’s bibliographic records. The author is currently working on new visualizations with the current dataset using different thresholds, and this time including network pruning algorithms to examine the field’s disciplinary structure when nodes and links are pared down to the most salient. Also to be investigated is the extent to which certain network measures, like centrality and sigma, increase when the number of nodes and links are pared. This study has utilized bibliographic data provided by WoS, and future research should investigate the results when another database, such as Scopus, is used. The dataset used in this analysis can also be broken down into journal groupings, or individual journals (such as only using Disasters or Journal of Homeland Security and Emergency Management, for example) to more fully explore the co-citation relationships established within these specific journals, which may further refine the structure of the specialties within Disaster Studies and Sciences. The results of the present study and the lists of authors, journals, and works, can also be broken down according to the specialty (a list of authors and works important to Disaster Medicine, or Emergency Management, for example, can be separated out from the aggregated results of the entire discipline). This can be a useful method of more specifically identifying the key parts of the body of knowledge relevant to the different specialties, while at the same time keeping the specialties aware of the knowledge possessed by one another. Finally, the method can be applied to bibliographic data on a yearly basis to record and observe the development of the discipline and its emerging research fronts. As has been hopefully demonstrated in this study, numerous intriguing directions are possible.

65

Figure 20. Possible disciplinary structure of Disaster Studies and Sciences. is structure is suggested by the JCA network. ere are two branches, nine divisions, and multiple specialties and subspecialties. Emergency/crisis management is one such specialty. Labels are based upon the general location of journals and works with similar themes, though it is not intended to be definitive or absolute. Substantial overlap between areas exists.

66

Figure 21. Preliminary framework for the discipline of Disaster Studies and Sciences. This is based upon the JCA visualization but goes beyond by adding additional possibilities for specialties, subject areas, and related academic disciplines. e framework is a working model intended to serve as a starting point for further discussion, exploration, and research.

67

LIST OF REFERENCES American Society of Civil Engineers. (2008). Civil engineering body of knowledge for the 21st century: Preparing the civil engineer for the future (2nd ed.) (Report of the Body of Knowledge Committee of the Committee on Academic Prerequisites for Professional Practice). Retrieved September 12, 2012, from http://www.asce.org/uploadedFiles/ Leadership_Training__New/BOK2E_(ASCE_2008)_ebook.pdf Blanchard, B.W. (2007, April 18). Emergency management top 50 reading list for collegiate educators. Retrieved September 2, 2012, from http://www.training.fema.gov/emiweb/ edu/docs/readinglist/Body%20of%20Knowledge%20EM%20HiEd%20Project%20Managers%20Conc eption.doc Blanchard, B.W. (2008, February 5). Bibliography of emergency management & related references onhand. Retrieved September 20, 2012, from http://training.fema.gov/EMIWeb/edu/docs/Wayne%20Bibliography.pdf Börner, K., Chen, C., & Boyack, K. W. (2003). Visualizing knowledge domains. In B. Cronin (Ed.), Annual Review of Information Science & Technology (pp. 179-255). Retrieved August 15, 2012, from http:// nwb.cns.iu.edu/papers/arist02.pdf Boyack, K.W. (2004). Mapping knowledge domains: Characterizing PNAS. Proceedings of the National Academy of Sciences 101(Suppl 1), S192-S199. doi: 10.1073/pnas.0307509100 Boyack, K.W., Klavans, R., & Börner, K. (2005). Mapping the backbone of science. Scientometrics, 64(3), 351-374. Retrieved September 25, 2012, from http://scimaps.org/exhibit/docs/05-boyack.pdf Chen, C. (2003). Mapping scientific frontiers: The quest for knowledge visualization (Kindle ed.). Berlin, DE: Springer. Available from http://www.amazon.com Chen, C. (2004). Searching for intellectual turning points: Progressive knowledge domain visualization. Proceedings of the National Academy of Sciences 101(Suppl 1), S303-S310. doi: 10.1073/pnas.0307513100 Chen, C. (2005). Measuring the quality of a network visualization. In Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries, p. 405. doi: 10.1.1.92.6476 Chen, C. (2006a). CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. Journal of the American Society for Information Science and Technology, 57(3), 359-377. doi:10.1002/asi.20317 Chen, C. (2006b). Information visualization: Beyond the horizon (2nd ed. /Kindle ed.). Berlin, DE: Springer. Available from http://www.amazon.com Chen, C. (2012, May 24). CiteSpace II (Version 3.1 R3 64 bit WebStart JVM XMX 1024MB) [Computer software]. Philadelphia, PA: Drexel University. Available from http://cluster.ischool.drexel.edu/~cchen/citespace/download.html Chen, C., Hu, Z., Liu, S., & Tseng, H. (2012). Emerging trends in regenerative medicine: a scientometric analysis in CiteSpace. Expert Opinions in Biological Therapies, 12(5), 593-608. doi:10.1517/14712598.2012.674507 Chen, C., Ibekwe-SanJuan, F., & Hou, J. (2010). The structure and dynamics of cocitation clusters: A multiple-perspective cocitation analysis. Journal of the American Society for Information Science and Technology 61(7), 1386-1409. doi: 10.1002/asi.21309 Chen, C., Kuljis J., & Paul, R. J. (2001). Visualizing latent domain knowledge. IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 31(4), 518-529. Retrieved October 16, 2012, from http://144.118.25.24/bitstream/1860/1954/1/2006175204.pdf Chen, C., & Paul, R. J. (2001). Knowledge domain's intellectual structure. Computer, 34(3), 65-71. Retrieved September 4, 2012, from dspace.library.drexel.edu/bitstream/1860/1935/1/2006175222.pdf Comfort, L. K., Waugh, W.L., & Cigler, B.A. (2012). Emergency management research and practice in public administration: Emergence, evolution, expansion, and future directions. Public Administration Review, 72(4), 539-547. doi:10.1111/j.1540-6210.2012.02549.x

68

Crane, D. (1972). Invisible colleges: Diffusion of knowledge in scientific communities. Chicago: University of Chicago Press. Cwiak, C. L. (2009a). 2009 body of knowledge report: The practitioner’s viewpoint. (Report of the FEMA Emergency Management Higher Education Program). Retrieved May 15, 2012, from http://training.fema.gov/emiweb/edu/docs/readinglist/2009%20 Body%20of%20Knowledge%20Practitioners.htm Cwiak, C. L. (2009b). Strategies for success: The role of power and dependence in the emergency management professionalization process (Doctoral dissertation, North Dakota State University, 209). Dissertation Abstracts. Retrieved August 2, 2012, from http://www.ndsu.edu/fileadmin/emgt/ Carol_Cwiak_Fisertation_10_29_009.doc Cwiak, C. L. (2011). 2011 body of knowledge report. (Report of the FEMA Emergency Management Higher Education Program). Retrieved September 15, 2012, from http://training.fema.gov/EMIWeb/edu/docs/ readinglist/Cwiak%20-%20Body%20of%20 Knowledge%20Report% 202011.doc Darlington, J.D. (n.d.). The profession of emergency management: Educational opportunities and gaps. Retrieved August 29, 2012, from http://www.training.fema.gov/emiweb/edu/prof_em.pdf Drabeck, T. E. (2007, August). Social problems perspectives, disaster research and emergency management: Intellectual contexts, theoretical extensions, and policy implications (Paper of lecture presented at the annual meeting of the American Sociological Association, New York City, New York). Retrieved October 5, 2012, from http://training.fema.gov/ EMIweb/edu/docs/Drabek%20%20Quarantelli%20Theory%20Award%20Lecture%20-%20Social%20Problems%20.doc Dwivedi, Y. K., Lal, B., Mustafee, N., & Williams, M. D. (2009). Profiling a decade of Information Systems Frontiers' research. Information Systems Frontiers, 11(1), 87-102. doi:10.1007/s10796-0089147-7 Evetts, J. (2003). The sociological analysis of professionalism: Occupational change in the modern world. International Sociology, 18(2), 395-415. Retrieved August 10, 2012, from https://docs.google.com/open?id=12uza7TSRgUfmqWClg0OTPu1_9VhuQxxAfeEsx5prOvDA661dym2aA0mpvKK Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108-111. Retrieved October 3, 2012, from http://garfield.library.upenn.edu/papers/science1955.pdf Greenwood, E. (1957). Attributes of a profession. Social Work, 2(3), 45-55. doi:10.1093/sw/ 2.3.45 Harzing, A.W. (2010). The Publish or Perish book: Your guide to effective and responsible citation analysis (Kindle edition). Available from http://www.harzing.com/pop.htm International Association of Emergency Managers. (n.d.). Emergency program managers: Knowledge, skills, and abilities. Retrieved August 25, 2012, from http://training.fema.gov/EMIWeb/edu/EmergProgMgr.doc International Association of Emergency Managers. (2011). Answers to commonly asked questions about the CEM/AEM examination: A study guide [Brochure]. Retrieved August 25, 2012, from http://www.iaem.com/certification/generalinfo/documents/ CEMAEMStudyGuide03Feb2012.pdf Janssen, M. A. (2007). An update on the scholarly networks on resilience, vulnerability, and adaptation within the human dimensions of global environmental change. Ecology and Society, 12(2), 18. Retrieved August 18, 2013 from http://ibcperu.org/doc/isis/8189.pdf Janssen, M. A., Schoon, M. L., Ke, W. M., & Börner, K. (2006). Scholarly networks on resilience, vulnerability and adaptation within the human dimensions of global environmental change. Global Environmental Change-Human and Policy Dimensions, 16(3), 240-252. doi: 10.1016/j.gloenvcha.2006.04.001 Jensen, J. (2010). The argument for a disciplinary approach to emergency management [Paper]. Retrieved September 2, 2012, from FEMA Web site: http://training.fema.gov/ EMIWeb/edu/ docs/Jensen%20%20Argument%20for%20a%20 Disciplinary%20 Approach% 20to%20EM%20in%20Higher%20Education.doc

69

Johnson, A. T. (2012). Institute of biological engineering: Body of knowledge. Retrieved September 16, 2012 from www.ibe.org/engineering-for-life/body-of-knowledge.htm Kadushin, C. (2012). Understanding social networks (Kindle ed.). New York, NY: Oxford University Press. Available from http://www.amazon.com Kleinberg, J. (2002, July 23-25). Bursty and hierarchical structure in streams. Paper presented at the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Edmonton, Alberta, Canada. doi:10.1145/775047.775061 Klingenberg, E. S. (2009, July 25). Interior architecture: A body of knowledge and a field for research [Paper presented at Registered Interior Design Association seminar, Hong Kong]. Retrieved September 8, 2012, from http://khio.academia.edu/EllenKlingenberg/ Papers/228930/Interior_Architecture__a_body_of_knowledge_and_a_field_for_research Kuhn, T. S. (1996). The structure of scientific revolutions (3rd ed.). Chicago: University of Chicago Press. (Original work published 1962) Larson, M.S. (1979). The Rise of Professionalism: A Sociological Analysis Retrieved November 14, 2012, from http://books.google.com/books?id=N4Ik8Xsrjx0C Lindell, M. K., Prater, C. S., & Perry, R. W. (2006). Chapter 12: Emergency management standards and evaluation. In Fundamentals of emergency management (pp. 346-372). Retrieved September 12, 2012, from http://training.fema.gov/EMIWeb/edu/docs/fem/ Chapter%2012%20%20EM%20Standards%20and%20Evaluation.doc Mane, K.K., & Börner, K. (2004). Mapping topics and topic bursts in PNAS. Proceedings of the National Academy of Sciences 101(Suppl 1), S287-S290. doi: 10.1073/pnas.0307626100 Massage Therapy Body of Knowledge (MTBOK) Task Force. (2010, May 15). Massage therapy body of knowledge (MTBOK) (Version 1.0). Retrieved September 12, 2012, from http://www.mtbok.org/ downloads/MTBOK_Version_1.pdf Neal, D.M., & Institute of Emergency Administration and Planning. (1993).Integrating disaster research and practice: An overview of issues. International Journal of Mass Emergencies and Disasters, 11(1), 5-13. Retrieved September 3, 2012, from http://training.fema.gov/EMIWeb/downloads/IJEMS/ARTICLES/INTEGRATING%20DISASTER%2 0RESEARCH%20AND%20PRACTICE%20AN%20OVERVIEW%20OF%20IS.pdf Osareh, F. (1996a). Bibliometrics, citation analysis, and co-citation analysis: A review of literature I. Libri 46, 149-158. Available from http://www degruyter.com Osareh, F. (1996b). Bibliometrics, citation analysis, and co-citation analysis: A review of literature II. Libri 46, 217-225. Available from http://www degruyter.com Perry, R. W., & Quarantelli, E. L. (Eds.). (2005). What is a disaster? New answers to old questions. Lexington, KY: Xlibris. Phillips, B. D. (2003, October 22). Disasters by discipline: Necessary dialogue for emergency management education [Paper presented at the Creating Educational Opportunities for the Hazards Manager of the 21st Century, Denver Colorado]. Retrieved September 10, 2012, from http://www.training.fema.gov/ EMIWeb/downloads/DenverFinal.doc Purchase, H. C., Andrienko, N., Jankun-Kelly, T. J., & Ward, M. (2008). Theoretical foundations of information visualization. In A. Kerren, J. T. Statsko, J. D. Fekete, & C. North (Eds.), Information Visualization: Human-Centered Issues and Perspectives (Lecture Notes in Computer Science Vol. 4950, pp. 51-70). Retrieved September 2, 2012, from http://www.dcs.gla.ac.uk/publications/PAPERS/9041/theoretical%20foundations%20of%20information %20visualisation.pdf Quarantelli, E.L. (1993). Converting disaster scholarship into effective disaster planning and managing: Possibilities and limitations. International Journal of Mass Emergencies and Disasters, 11(1), 15-39. Retrieved September 3, 2012, from http://training.fema.gov/EMIWeb/downloads/IJEMS/ARTICLES/BRIDGING%20THE%20GAP%20B ETWEEN%20RESEARCH%20AND%20PRACTICE%20THE%20NATURAL%20H.pdf

70

Quarantelli, E. L. (Ed.). (2005a). What is a disaster? A dozen perspectives on the question (Kindle ed.). Available from amazon.com. (Original work published 1998) Quarantelli, E.L. (2005b). A social science research agenda for the disasters of the 21st century: Theoretical, methodological and empirical issues and their professional implementation. In R. W. Perry & E. L. Quarantelli (Eds.), What is a disaster? New answers to old questions (pp. 325-396). Lexington, KY: Xlibris. Reid, E. O. F. (1997). Evolution of a body of knowledge: An analysis of terrorism research. Information Processing & Management, 33(1), 91-106. doi:10.1016/S0306-4573(96)00052-0 Reid, E. F., & Chen, H. (2007). Mapping the contemporary terrorism research domain. International Journal of Human-Computer Studies, 65(1), 42-56. doi:10.1016/j.ijhcs.2006.08.006 Rorissa, A., & Yuan, X. (2012). Visualizing and mapping the intellectual structure of information retrieval. Information Processing & Management, 48(1), 120-135. Retrieved October 3, 2012, from http://www.albany.edu/~ar955283/pubs/preprint_Abebe_Jenny_ IPM_2011.pdf Samoylenko, I., Chao, T.C., Liu, W.C., & Chen, C.M. (2006). Visualizing the scientific world and its evolution. Journal of the American Society for Information Science and Technology 67(11), 1461-1469. Retrieved October 5, 2012, from http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.102.9195&rep=rep1&type=pdf Schmidt, J. A. (2008, November). What is a profession? Structure, 9. Retrieved August 6, 2012, from http:// www.structuremag.org/Archives/2008-11/C-Infocus-Schmidt-Nov-08.pdf Schoonbaert, D., & Roelants, G. (1996). Citation analysis for measuring the value of scientific publications: Quality assessment tool or comedy of errors? Tropical Medicine and International Health, 1(6), 739752. Retrieved September 12, 2012, from http://lib.itg.be/ds963.pdf Sefton, JM., Shea, M., & Hines, C. (2011). Developing, maintaining, and using a body of knowledge for the massage therapy profession. International Journal of Therapeutic Massage and Bodywork, 4(3), 1-12. Retrieved August 2, 2012, from http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3184471/pdf/ijtmb-4-313.pdf Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265-269. Retrieved September 21, 2012, from http://polaris.gseis.ucla.edu/gleazer/296_readings/small.pdf Spiewak, D.L. (2005, July 5). Common body of knowledge: Top 50 reading list as selected by practicing emergency managers. Retrieved August15, 2012, from http://training.fema.gov/EMIWeb/edu/docs/readinglist/ 2005% 20IAEM%20Practitioner's%20EM%20Body%20of%20 KNowledge% 20Survey.doc Swick, H.M. (2000, June). Toward a normative definition of medical professionalism. Academic Medicine, 15(6), 612-616. Retrieved September 12, 2012, from https://aiha.com/ EMSEducationImplementationPlanning/documents/NormativeDefinition.pdf Synnestvedt, M.B., Chen, C., & Holmes, J.H. (2005). CiteSpace II: Visualization and knowledge discovery in bibliographic databases. 2005 AMIA Annual Symposium Proceedings 2005, 724-728. Retrieved September 12, 2012, from http://www.ncbi.nlm.nih.gov/pmc/articles/ PMC1560567/pdf/amia2005_0724.pdf University Consortium for Geographic Information Science. (2006). Geographic information science & technology body of knowledge 2006 quick reference [Brochure]. Retrieved August 2, 2012, from http:// www.gisci.org/Body_of_Knowledge/body_of_knowledge_ref.pdf Van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143. Retrieved September 11, 2012, from http://yunus.hacettepe.edu.tr/~tonta/courses/ spring2011/bby704/Van%20Raan-Scientometrics2005.pdf Vargas-Quesada, B., De-Moya-Anegón, F., Chinchilla-Rodríguez, Z., & González-Molina, A. (2006). Domain analysis by means of the visualization of maps of vast scientific domains. Paper presented at the I International Conference on Multidisciplinary Information Sciences & Technologies, Merida, Spain. Retrieved October 2, 2012, from

71

http://eprints.rclis.org/bitstream/10760/8628/1/Domain_analysis_by_means_of_the_visualization_of_m aps_of_vast_scientific_domains.pdf Vargas-Quesada, B., & de Moya-Anegón, F. (2007). Visualizing the structure of science (Kindle edition). Berlin, DE: Springer. Available from http://www.amazon.com West, A. (2012, May 23). BabelPad (Version 6.1.0.7) [Computer software]. Retrieved September 27, 2012, from http://www.babelstone.co.uk/software/babelpad.html White, H.D., & McCain, K.W. (1998). Visualizing a discipline: An author co-citation analysis of information science, 1992-1995. Journal of the American Society for Information Science 49(4), 327355. doi:10.1.1.95.6296 Wilson, J. L. (2001). The state of emergency management 2000: The process of emergency management professionalization in the United States and Florida (Doctoral dissertation, Florida International University, 2000). Available from Dissertation.com Zhao, R., & Wang, J. (2011). Visualizing the research on pervasive and ubiquitous computing. Scientometrics, 86, 593-612. doi:10.1007/s11192-010-0283-8

72

APPENDIX A Sample of Original Web of Science (WoS) Dataset PT J AU Cigler, BA AF Cigler, Beverly A. TI The "Big Questions" of Katrina and the 2005 great flood of New Orleans SO PUBLIC ADMINISTRATION REVIEW LA English DT Article; Proceedings Paper CT 67th Annual Meeting of the American-Society-for-Public-Administration CY APR 01-04, 2006 CL Denver, CO SP Amer Soc Public Adm ID RISK AB The "big questions" associated with Hurricane Katrina and the great flood of New Orleans lie at the intersection of the natural and humanshaped environments. The interactions dominating the intersection of the two environments are found in the social-political-economic system, culture and history, intergovernmental relations, and law. The big questions are not whether specific individuals were to blame for the destruction of lives and property, and they do not begin with the slow and inadequate intergovernmental response to the disaster. Instead, the big questions involve the roles of individuals, governments, and private markets in creating so-called natural disasters; whether government, through its lead role in the emergency management system, is incompetent, or whether capability and performance in protecting life and property have been eroded through a long-term "hollowing out" process; and whether Katrina lessons will be learned or merely noted. C1 Penn State Univ, Harrisburg, PA USA. RP Cigler, BA (reprint author), Penn State Univ, Harrisburg, PA USA. EM [email protected] CR ADAMSON F, 2004, FRAMING SECURITY POS Agranoff R., 2001, GETTING RESULTS COLL Allen BL, 2003, URBAN IND ENVIRON, P1 ALLEN BL, 2006, RES WORKSH CTR BIOEN Barry J. M., 1997, RISING TIDE GREAT MI Bonabeau E., 2001, HARVARD BUS REV, V79, P107 Brinkley Douglas, 2006, GREAT DELUGE HURRICA Brookings Institution, 2005, NEW ORL STORM LESS P BURBY R, 1998, COOPERATING NATURE C BURBY RJ, 1984, REGIONAL STATE WATER BURBY RJ, 2005, NATURAL HAZARDS REV, V6, P67, DOI 10.1061/(ASCE)15276988(2005)6:2(67) CARAFANO JY, 2006, LEARNING DISASTER RO CIGLER B, 1990, CITIES DISASTER N AM, P59 CIGLER B, 1990, HDB EMERGENCY MANAGE, P81 CIGLER BA, 1988, MANAGING DISASTER ST, P39 CIGLER BA, 1988, CRISIS MANAGEMENT CA, P5 CIGLER BA, 2006, PUBLIC MANAGER, V35, P3

73

COLTEN CE, 2001, TRANSFORMING NEW ORL Comfort LK, 2006, URBAN AFF REV, V41, P501, DOI 10.1177/1078087405284881 Cowdrey Albert E., 1977, LANDS END HIST NEW O CUTTER SL, 2005, AM GEOPHYS UNION, V86, P381 CUTTER SL, 2005, GEOGRAPHY SOCIAL VUL Cutter SL, 2003, SOC SCI QUART, V84, P242, DOI 10.1111/15406237.8402002 Dreier P, 2006, URBAN AFF REV, V41, P528, DOI 10.1177/1078087405284886 Dyson Michael E., 2006, COME HELL HIGH WATER Fischetti M, 2001, Sci Am, V285, P76 FRITZELL P, 1978, AM WATER RESOURC NOV, P523 Hacker JS, 2004, AM POLIT SCI REV, V98, P243 Hall I P, 2000, Respir Res, V1, P6, DOI 10.1186/rr3 HANDWERK B, 2005, NATL GEOGRAPHIC 0902 Harrison Robert W., 1961, ALLUVIAL EMPIRE, VI Hayes B, 2005, AM SCI, V93, P496, DOI 10.1511/2005.56.3470 HAYNES SR, 2007, 40 HAW INT C SYST SC Houck O., 2006, TULANE ENV LAW J, V19, P1 Kelman Ari, 2003, RIVER ITS CITY NATUR KING RO, 2005, RL32972 C RES SERV LEATHERMAN SP, 2006, WATER RESOURCES IMPA, V8, P6 LINDBLOM CE, 2000, MARKET SYSTEM WHAT I MCCARTHY JE, 2006, RL3310M C RES SER McPhee J., 1989, CONTROL NATURE MCQUAID J, 2002, TIMES PICAYUNE Moynihan Donald P., 2005, LEVERAGING COLLABORA Mueller John, 2005, INT STUDIES PERSPECT, V6, p[208, 217], DOI 10.1111/j.1528-3577.2005.00203.x Multi-hazard Mitigation Council, 2005, NAT HAZ MIT SAV IND National Academy of Public Administration (NAPA), 1993, COP CAT BUILD EM MAN Perrow C., 1984, NORMAL ACCIDENTS LIV POWELL WW, 1990, RES ORGAN BEHAV, V12, P295 REED DJ, 2004, PHYSICAL GEOGR, V25, P4, DOI DOI 10.2747/02723646.25.1.4 RELYEA HC, 2006, 98505 C RES SERV Rickel S., 2005, OUR TOXIC GUMBO RECI Shallat Todd, 1994, STRUCTURES STREAM WA SHARP RE, 2006, ISSUES SCI TECHNOLOG SLOVIC P, 1986, RISK ANAL, V6, P403, DOI 10.1111/j.15396924.1986.tb00953.x Snook SA, 2000, FRIENDLY FIRE ACCIDE SNOWDEN JO, 1980, GEOLOGY GRATER NEW O STEPHENSON WD, 2007, HOMELAND SECURITY AF, V3 STREEVER B, 2001, SAVING LOUISIANA BAT The White House, 2006, FED RESP HURR KATR L TRAVIS J, 2005, SCIENCE, V309, P57 VILEISIS A, 1999, DISCOVERING UNKNOWN Viscusi WK, 1997, ECON J, V107, P1657, DOI 10.1111/1468-0297.00248 Weick K. E., 1995, SENSMAKING ORG

74

NR TC Z9 PU PI PA SN J9 JI PD PY VL SU BP EP PG WC SC GA UT ER

WHORISKEY P, 2006, WASHINGTON POST 0331 ZEDLEWSKI SR, 2006, BUILDING BETTER SAFE *NATL COMM TERR AT, 2004, 9 11 COMM REP FIN RE *OFF INSP GEN, 2006, PERF REV FEMA DIS MA *PEW RES CTR PEOPL, 2005, 2 3 CRIT BUSH REL EF *US DEP JUST, US GOV INT DOM TERR *US DHS, 2004, NATL INC MAN SYST *US DHS, 2004, NAT RESP *US HOUS, 2006, FAIL IN *US SEN, 2006, HURR KATR NAT STILL 72 13 13 WILEY-BLACKWELL MALDEN COMMERCE PLACE, 350 MAIN ST, MALDEN 02148, MA USA 0033-3352 PUBLIC ADMIN REV Public Adm. Rev. DEC 2007 67 S 64 76 13 Public Administration Public Administration 245MJ WOS:000251938900008

75

APPENDIX B Sample of Final Dataset (Author Co-citation Analysis Version) PT J AU CIGLER, BA AF CIGLER, BEVERLY A. TI THE "BIG QUESTIONS" OF KATRINA AND THE 2005 GREAT FLOOD OF NEW ORLEANS SO PUBLIC ADMINISTRATION REVIEW LA ENGLISH DT ARTICLE; PROCEEDINGS PAPER CT 67TH ANNUAL MEETING OF THE AMERICAN-SOCIETY-FOR-PUBLIC-ADMINISTRATION CY APR 01-04, 2006 CL DENVER, CO SP AMER SOC PUBLIC ADM ID RISK AB THE "BIG QUESTIONS" ASSOCIATED WITH HURRICANE KATRINA AND THE GREAT FLOOD OF NEW ORLEANS LIE AT THE INTERSECTION OF THE NATURAL AND HUMANSHAPED ENVIRONMENTS. THE INTERACTIONS DOMINATING THE INTERSECTION OF THE TWO ENVIRONMENTS ARE FOUND IN THE SOCIAL-POLITICAL-ECONOMIC SYSTEM, CULTURE AND HISTORY, INTERGOVERNMENTAL RELATIONS, AND LAW. THE BIG QUESTIONS ARE NOT WHETHER SPECIFIC INDIVIDUALS WERE TO BLAME FOR THE DESTRUCTION OF LIVES AND PROPERTY, AND THEY DO NOT BEGIN WITH THE SLOW AND INADEQUATE INTERGOVERNMENTAL RESPONSE TO THE DISASTER. INSTEAD, THE BIG QUESTIONS INVOLVE THE ROLES OF INDIVIDUALS, GOVERNMENTS, AND PRIVATE MARKETS IN CREATING SO-CALLED NATURAL DISASTERS; WHETHER GOVERNMENT, THROUGH ITS LEAD ROLE IN THE EMERGENCY MANAGEMENT SYSTEM, IS INCOMPETENT, OR WHETHER CAPABILITY AND PERFORMANCE IN PROTECTING LIFE AND PROPERTY HAVE BEEN ERODED THROUGH A LONG-TERM "HOLLOWING OUT" PROCESS; AND WHETHER KATRINA LESSONS WILL BE LEARNED OR MERELY NOTED. C1 PENN STATE UNIV, HARRISBURG, PA USA. RP CIGLER, BA (REPRINT AUTHOR), PENN STATE UNIV, HARRISBURG, PA USA. EM [email protected] CR ADAMSON F, 2004, FRAMING SECURITY POS AGRANOFF R, 2001, GETTING RESULTS COLL ALLEN BL, 2003, URBAN IND ENVIRON, P1 ALLEN BL, 2006, RES WORKSH CTR BIOEN BARRY JM, 1997, RISING TIDE GREAT MI BONABEAU E, 2001, HARVARD BUS REV, V79, P107 BRINKLEY D, 2006, GREAT DELUGE HURRICA BROOKINGS INSTITUTION, 2005, NEW ORL STORM LESS P BURBY RJ, 1998, COOPERATING NATURE C BURBY RJ, 1984, REGIONAL STATE WATER BURBY RJ, 2005, NAT HAZARDS REV, V6, P67, DOI 10.1061/(ASCE)15276988(2005)6:2(67) CARAFANO JY, 2006, LEARNING DISASTER RO CIGLER BA, 1990, CITIES DISASTER N AM, P59 CIGLER BA, 1990, HDB EMERGENCY MANAGE, P81 CIGLER BA, 1988, MANAGING DISASTER ST, P39 CIGLER BA, 1988, CRISIS MANAGEMENT CA, P5 CIGLER BA, 2006, PUBLIC MANAGER, V35, P3

76

COLTEN CE, 2001, TRANSFORMING NEW ORL COMFORT LK, 2006, URBAN AFF REV, V41, P501, DOI 10.1177/1078087405284881 COWDREY AE, 1977, LANDS END HIST NEW O CUTTER SL, 2005, AM GEOPHYS UNION, V86, P381 CUTTER SL, 2005, GEOGRAPHY SOCIAL VUL CUTTER SL, 2003, SOC SCI QUART, V84, P242, DOI 10.1111/15406237.8402002 DREIER P, 2006, URBAN AFF REV, V41, P528, DOI 10.1177/1078087405284886 DYSON ME, 2006, COME HELL HIGH WATER FISCHETTI M, 2001, SCI AM, V285, P76 FRITZELL P, 1978, AM WATER RESOURC NOV, P523 HACKER JS, 2004, AM POLIT SCI REV, V98, P243 HALL I P, 2000, RESPIR RES, V1, P6, DOI 10.1186/RR3 HANDWERK B, 2005, NATL GEOGRAPHIC 0902 HARRISON RW, 1961, ALLUVIAL EMPIRE, VI HAYES B, 2005, AM SCI, V93, P496, DOI 10.1511/2005.56.3470 HAYNES SR, 2007, 40 HAW INT C SYST SC HOUCK O, 2006, TULANE ENV LAW J, V19, P1 KELMAN A, 2003, RIVER ITS CITY NATUR KING RO, 2005, RL32972 C RES SERV LEATHERMAN SP, 2006, WATER RESOURCES IMPA, V8, P6 LINDBLOM CE, 2000, MARKET SYSTEM WHAT I MCCARTHY JE, 2006, RL3310M C RES SER MCPHEE J., 1989, CONTROL NATURE MCQUAID J, 2002, TIMES PICAYUNE MOYNIHAN DP, 2005, LEVERAGING COLLABORA MUELLER J, 2005, INT STUDIES PERSPECT, V6, P[208, 217], DOI 10.1111/J.1528-3577.2005.00203.X MULTI-HAZARD MITIGATION COUNCIL, 2005, NAT HAZ MIT SAV IND NAPA, 1993, COP CAT BUILD EM MAN PERROW C, 1984, NORMAL ACCIDENTS LIV POWELL WW, 1990, RES ORGAN BEHAV, V12, P295 REED DJ, 2004, PHYSICAL GEOGR, V25, P4, DOI 10.2747/0272-3646.25.1.4 RELYEA HC, 2006, 98505 C RES SERV RICKEL S., 2005, OUR TOXIC GUMBO RECI SHALLAT T, 1994, STRUCTURES STREAM WA SHARP RE, 2006, ISSUES SCI TECHNOLOG SLOVIC P, 1986, RISK ANAL, V6, P403, DOI 10.1111/J.15396924.1986.TB00953.X SNOOK SA, 2000, FRIENDLY FIRE ACCIDE SNOWDEN JO, 1980, GEOLOGY GRATER NEW O STEPHENSON WD, 2007, HOMELAND SECURITY AF, V3 STREEVER B, 2001, SAVING LOUISIANA BAT THE WHITE HOUSE, 2006, FED RESP HURR KAT TRAVIS J, 2005, SCIENCE, V309, P57 VILEISIS A, 1999, DISCOVERING UNKNOWN VISCUSI WK, 1997, ECON J, V107, P1657, DOI 10.1111/1468-0297.00248

77

NR TC Z9 PU PI PA SN J9 JI PD PY VL SU BP EP PG WC SC GA UT ER

WEICK KE, 1995, SENSMAKING ORG WHORISKEY P, 2006, WASHINGTON POST 0331 ZEDLEWSKI SR, 2006, BUILDING BETTER SAFE *NATL COMM TERR AT, 2004, 9 11 COMM REP FIN RE *OFF INSP GEN, 2006, PERF REV FEMA DIS MA *PEW RES CTR PEOPL, 2005, 2 3 CRIT BUSH REL EF DOJ, US GOV INT DOM TERR DHS, 2004, NATL INC MAN SYST DHS, 2004, NAT RESP HOUSE SEL COMM, 2006, FAIL OF INITIATIVE USSENATE, 2006, HURR KATR NAT STILL 72 13 13 WILEY-BLACKWELL MALDEN COMMERCE PLACE, 350 MAIN ST, MALDEN 02148, MA USA 0033-3352 PUBLIC ADMIN REV PUBLIC ADM. REV. DEC 2007 67 S 64 76 13 PUBLIC ADMINISTRATION PUBLIC ADMINISTRATION 245MJ WOS:000251938900008

78