Applications of High-Performance Knowledge-Based Technology Marion G. Ceruti, Senior Member, IEEE Space and Naval Warfare Systems Center San Diego, CA 92152-5001
Craig Anken United States Air Force Research Laboratory Rome, NY 13440
Albert D. Lin Be-Bee, Inc. San Diego, CA, 92127
Stuart H. Rubin, Senior Member, IEEE Space and Naval Warfare Systems Center San Diego, CA 92152-5001
Abstract This paper describes the technology developed in the Defense Advanced Research Projects Agency’s (DARPA) High-Performance Knowledge-Base (HPKB) Program that was transferred to follow-on applications. These applications include the following: 1. A data-mining project using naïve Bayesian networks for the U.S. Marine Corps; 2. A Theatre Ballistic Missile Reasoner project, using Bayesian networks for the U.S. Air Force; 3. An application of course-of-action technology for the U.S. Army, and 4. On-going research projects to mine knowledge from the World Wide Web. The HPKB program objectives are presented and each transferred technology is described in relation to its target application. Future trends for knowledge-based and datamining technology are discussed.
1 Introduction The purpose of this paper is to document the technology developed in the Defense Advanced Research Projects Agency’s (DARPA) High-Performance Knowledge-Base (HPKB) Program that was transferred to applications in follow-on projects. This paper provides the following information: 1. an overview of each target application; 2. a description of each transferred HPKB technology area; 3. a discussion of the technology’s contribution to the application, and 4. uses of HPKB-sponsored technology. Knowledge is power. The second industrial revolution, now in full swing, promises unprecedented increases in productivity with knowledge-based systems. Advances in data mining technologies promise to overcome the so-called “knowledge-acquisition bottleneck.” The HPKB program has sponsored research that addresses mining knowledge from the Web – including mixed-media mining. (See, for example, [18] and [22].) Similarly, Rubin et al. are researching how to mine text efficiently using an object-oriented approach. If successful,
the resulting large and eminently practical knowledge bases (KB’s) will drive the intelligent engines of the 21st century. KB’s, inexact computing, optronics, distributed databases, and human-factors computing will become much more prevalent than they are today. This paper explains how a confluence of nascent computer technologies will play a fundamental role in the tomorrow’s knowledge-based military. DARPA in general, and programs such as HPKB in particular, are on the forefront of the efforts to define and extend the state of the art. The paper is organized as follows. Section 2 contains a description of the DARPA HPKB program. Section 3 describes the use of a naïve Bayesian network algorithm to support a data-mining project for the U.S. Marine Corps. Section 4 details an application of Bayesian network technology to a Theatre Ballistic Missile Reasoner for the U.S. Air Force. Section 5 is about the knowledge-base technology that supports an army commander’s ability to decide courses of action (COA’s). Section 6 suggests directions for future technology transfers. Section 7 is the summary.
2 HPKB Program Overview The HPKB program was a two-year program that began in 1997. This program’s objective was to produce the technology needed to enable system developers to construct rapidly (within months) large KB's (consisting of between 10K to 100K axioms, rules, and/or frames) that provide comprehensive domain coverage, that are reusable by multiple applications, and that are maintainable in rapidly changing environments [2], [4], [15]. To meet this objective, the approach consisted of three major steps. 1. Building foundation knowledge. This involved selecting the knowledge representation scheme, assembling theories of common knowledge, and defining domain-specific terms and concepts with which to populate the large, comprehensive KB’s for particular domains of interest. Building foundation
knowledge required the knowledge representation expert to select, compose, extend, specialize and modify components from a library of reusable ontologies, common domain theories, and generic problem-solving strategies. 2. Acquiring domain knowledge. This means constructing and populating a complete KB by using the foundation knowledge to generate domain-specific knowledge acquisition, data mining, and information extraction tools. This complete KB enabled collaborating teams of domain experts, who were not all computer experts, to extend efficiently the foundation theories, to define additional domain theories and problem-solving strategies, and to acquire domain facts for populating a comprehensive KB covering the domain of interest. Machine learning also contributed to domain knowledge acquisition in this step of the approach. 3. Efficient problem solving. This was accomplished by providing efficient inference and reasoning procedures to operate on a complete KB, and by providing tools and techniques to select and to transform knowledge from a complete KB into optimized problem-solving modules tailored to the unique requirements of an application. Two knowledge-base integration teams developed and integrated the new products and technology of this effort. One team, consisting of private companies such as Teknowledge, Cycorp and Kestrel, constructed an integrated development environment based on Cycorp’s integrated KB, Cyc, to include run-time environment and generic problem solvers for information retrieval, situation assessment, and scheduling. The other team consisted of Science Applications International Corp. (SAIC) [15], Knowledge System Laboratory (KSL) of Stanford University, Stanford Medical Informatics (SMI), SRI International, and the University of Southern California (USC) Information Science Institute (ISI). A key objective in these integration efforts was to produce a system robust enough to solve the HPKB challenge problems, to be developed by Alphatech, Inc.; Information Extraction & Transport, Inc.; and Pacific-Sierra Research Corp., especially to test the capabilities of the integration environment and its components. The application domains for these challenge problems included the areas of crisis management and battlefield awareness, such as a knowledgebased assessment of various COA’s. These goals required the rapid development of large-scale KB’s and integration strategies that were accomplished by the integration teams and their supporting technology developers.
have produced practical application in areas such as analyzing medical outcomes, detecting credit card fraud, predicting customer purchase behavior, predicting the personal interests of Web users, and optimizing manufacturing processes. They also have led to a set of fascinating and important scientific questions about how computers might learn automatically from past experience [18]. This section covers an HPKB-developed Bayesian network technology that was applied to a data-mining project, the outcome of which was aimed at data classification. The Advanced Data Fusion project was a two-part effort, one part of which was assigned to the Command and Control (C2) Department of Space and Naval Warfare Systems Center, San Diego (SPAWARSYSCEN). The purpose of the C2 part was wartime-event prediction by mining data from the U.S. Marine Corp’s Urban Warrior exercises [3]. These predictions involved a classification task to sort the data initially into two groups, data that predict that an attack is likely and data that do not. SRI International, an HPKB contractor, developed an approach to data classification using naïve Bayesian networks. SRI International’s classifier, the Tree Augmented Naive (TAN) Bayesian classification algorithm has the advantages of robustness and polynomial computational complexity [3]. Bayesian networks have some drawbacks that SRI has addressed in the TAN algorithm. In ordinary naive Bayesian networks, the variables (data) are assumed to be conditionally independent given the class. Logically, this is not always true. For example, enemy troops may be observed at location X and their tanks, at location Y. With Naive Bayesian networks, one assumes that these events are independent. However, both events may be part of the overall enemy battle plan. In the TAN algorithm, the trees provide edges that represent correlation between the variables [3]. Bayesian networks with tree augmentation, are a suitable technology for data-mining classification and event prediction for reasons described in [3] and [17]. The operational concept, the data-mining environment, and the techniques that use Bayesian networks for classification also are described in [3] and [17]. Two key steps in the research plan are to implement machine learning to train Bayesian networks to classify the data and to test the trained networks using additional command and control data. The technique is under consideration to support a Marine battlefield commander in predicting attacks.
3 Wartime Attack Prediction
4 Theatre Ballistic Missile Reasoner
The HPKB program has generated technologies that contribute to data-mining projects. The field of data mining addresses the question of how best to discover general regularities and improve the process of making decisions [18]. The fields of data mining, sometimes called “knowledge discovery from databases,” and machine learning already
4.1
Problem Definition
Time-Critical Targets (TCT’s) are short-lived opportunity targets identified by the Joint Forces Air Component Commander (JFACC) [16] staff requiring immediate response. Shooter selection and tasking for TCT’s often exceed opti-
mal timeframes because no single sensor can detect and characterize them consistently. Because of this, all TCT’s present a major timing challenge to the command-andcontrol process, system, and weapon capabilities. Theatre Ballistic Missile (TBM) Defense presents an imposing TCT challenge to U.S. forces operating against them. TBMs can carry high explosive warheads, including Weapons of Mass Destruction (WMD), and are targeted against high-priority friendly or civilian assets. A good example of a ground TCT associated with TBM Defense is the TBM Transporter Erector Launcher (TEL). It is very difficult to locate TBM launchers. Naturally, finding and disabling before launch is the preferred time to negate a ballistic missile attack. Unfortunately, the TEL and missile position cannot be identified confidently until the actual launch. Thus, when a TEL is located an immediate response is necessary. Often, this will affect attacks on pre-planned targets [8]. Dissecting the TBM threat is a complex, multi-dimensional intelligence problem. To attempt to address this problem analysts need to decompose and correlate the “who and what (equipment and units), where (infrastructure), when, why, and how (operations)” to develop potential enemy COA’s. The goal here is to give the JFACC the capability to deny missile launch instead of attacking TELs after missile launch. As the TEL advances from garrison through the post launch phase, the time available to find the TEL, task assets against it, and attack it becomes shorter, thus making the task more pressing. The entire process of finding targets, tasking assets, and attacking and killing must occur within this brief period. Enhanced decision support is needed to help commanders determine COA’s within the time frame required [7].
4.2 TBM Reasoner The TBM Reasoner will support the process of analyzing the adversary theater-missile force and environment in a specific geographic area. It will consist of three basic parts: movement analysis, prediction and COA assessment. The tool is designed to support commanders and staff in their planning and decision-making at the joint and service levels. It will assist the commander and staff in selectively applying and maximizing combat power at critical points in time and space. This will be accomplished by determining the likely adversary TBM force COA. A KB, consisting of Intelligence Preparation of the Battlespace (IPB) products useful in the identification of TBM vehicles, will be developed. The KB will consist of terrain models, road networks, operational areas, areas of interest, battlespace effects, and other domain-relevant knowledge. The KB will also include suspected hide sites and launch sites and will encode adversary order of battle and models of adversary behavior.
A key capability of the TBM Reasoner is to provide exploitation of sensor data to infer states and locations of enemy TBM objects as well as transition probabilities of these objects from one location to another. One of the major problems associated with this capability is that often the sensor data are impoverished, noisy and incomplete. Whereas traditional approaches to KB reasoning are adequate to represent TBM IPB products, they do not support noisetolerant, probabilistic reasoning. Conversely, traditional approaches to stochastic modeling provide principled Bayesian approaches to probabilistic reasoning, but do not provide modeling or inference tools that scale well to complex problems such as representing and reasoning about TBM IPB products.
4.3 Dynamic Bayesian Networks The core of the TBM Reasoner will be based on work performed for DARPA and the Air Force Research Laboratory under the DARPA HPKB program. KB development for the TBM Reasoner tool will utilize a powerful specification language for representing TBM IPB products based on a probabilistic frame system for knowledge representation that integrates logic-based and probabilistic representations in a uniform Bayesian framework [16]. New algorithms for approximate reasoning using Dynamic Bayesian Networks (DBN’s) will be utilized to enable efficient inference over rich probabilistic behavior models, such as those that can be specified using probabilistic frame systems, such as that mentioned above. A DBN models the evolution of a set of random variables over time. DBN’s will allow the user to represent multiple TBM vehicle behaviors while the number of state variables remains tractable. Using these techniques, the user can specify more elaborate models of TBM IPB behavior, compile these models into DBN’s, and exploit new algorithms for approximate inference in DBN’s to identify efficiently the TBM TCT’s from sensor data [12], [14].
5 Course-of-Action Technology The U.S. Army develops plans using a doctrinal procedure called the Military Decision-Making Process (MDMP) [9]. This process involves many stages: 1. Mission Receipt, 2. Mission Analysis, 3. COA Development, 4. COA Analysis, 5. COA Comparison, 6. COA Briefing and Approval, 7. Operation-orders (OPORD) Preparation, 8. Execution Preparation, 9. Execution Monitoring. The present practice of the planning relies heavily on human “brainstorming,” manual transfer of sketches to maps, and the extensive consultation of manuals to find the appropriate data. These procedures are time consuming and the accuracy is questionable. In stages 4 and 5, the Army planners face two problems. First, the decision making depends on evaluation of
complex alternatives, and second, the current military planning systems do not include tools to critique the generated plans automatically. During the briefing stage, the background understanding and reasoning that contributed to producing COA’s are difficult to transfer to the commander, who needs this understanding to make the most informed decisions. Moreover, the briefing officers find it difficult to portray wargaming insights, such as the identification of decisive points in a battle. The two-fold goal of the COA effort is as follows: 1. Automatically critique COA’s for ground-force operation and suggest refinement. 2. Train and empower military staff. Each technology, described below, was created to achieve the project goals collaboratively. Ontologies – These are collections of terminologies that consist of a systematic account of concepts in the KB. They are needed for many domain tasks. All participants in this project contributed to the ontology development. Rule Bases – These are used to assess the suitability, feasibility, acceptability, and correctness of a COA, identify the strengths, weaknesses, and decisive point(s) of a COA, scheme of maneuver, command and control, and enabling
COA Sketch Tool (Teknowledge)
COA Simulation (UMass)
Translator (TFS-SAIC)
Geographic Reasoner (NWU)
COA Statememt Editor (AIAITeknowledge)
tasks. Knowledge and insights gained in the COA formulation process can be entered into the rule base to assist in the next COA task. COA Sketch – This consists of a sketch tool, developed by Teknowledge, Inc., that is used to draw military unit symbols, areas, tactical missions, maneuver, and products that document IPB. It is based on a Geographic Information System (GIS). The drawings of the tool can be translated automatically into knowledge representation based on CycL. Northwestern University (NWU) also contributed a sketching tool with multimodal interaction (speech and “lightpen”) that enables natural interfaces for COA creation. CycL – This is a formal language that is modeled after the language of first-order predicate calculus. However, it is a far more complex and expressive language [5]. COA Statement – The University of Edinburgh’s Artificial Intelligence Applications Institute (AIAI) and Teknowledge Federal Systems (TFS) collaboratively created a translator that converts textual COA descriptions into formal knowledge representations using CycL [6]. The translator can operate independently of the sketch tool, or it can integrate its results with the output of the sketch tool.
Translator (AIAI)
Expert (ISI)
COA Critiquer (ISI)
Disciple (GMU)
COA Critiquer (GMU)
Protege (SMI)
COA Critiquer (TeknowledgeCycorp)
Knowledge Acquisition
Fig. 1. COA Architecture
Critiquers
Query Interface
Geographical Reasoner – NWU’s reasoning engine provides human-like spatial reasoning on a qualitative spatial vocabulary formed from spatial concepts in the battlespace. Analogical Critique – The analogical critiquer, developed by NWU, allows subject matter experts (SMEs) to apply lessons learned in a human-like way. The results of these lessons learned can be stored in a library of cases. GMU Critiquer – George Mason University’s (GMU’s) knowledge acquisition and critiquer emphasizes identifying the strengths and weaknesses of a COA. Teknowledge/Cycorp Critiquer – This critiquer employs multiple Cyc-based problem-solving methods to highlight doctrinal problems in a human-generated COA and to determine the soundness and feasibility of that COA. ISI Critiquer – USC ISI contributed two critiquers. The EXPECT critiquer develops and uses middle-level theories of plan evaluation and critiquing. It generates English explanations without hand-coded templates. The other critiquer provides case-based critiquing of COA decisive points. UMASS Simulator – The University of Massachusetts (UMass) developed a war-game simulator to simulate COA’s in capturing the flag. Fig. 1 depicts the architecture of the HPKB COA system. The system can take inputs from a text COA statement or COA sketch. The translators convert the inputs into CycLbased knowledge representations. The geographical reasoner generates terrain features, avenues of approach, and a combined-obstacle overlay from the inputs and makes them available to the critiquers. The knowledge-acquisition tools learn domain rules from the SMEs. These tools include ISI’s EXPECT, GMU’s Disciple, and Protégé, which was developed by SMI. The critiquers use these domain theories, along with geographical reasoning results and the war game simulation output, to solve problems. The critiquers critique the COA and suggest refinements in plain English. The user can locate and evaluate these explanations through the CAT GUIs. The community-wide collaboration on ontology development enables the interoperability of many knowledge-based critiquers. The shared domain theories of the integrated system also empower the problem-solving capabilities of each critiquer. The multi-modal interface and output GUI allow the Army planners, who do not have knowledge engineering backgrounds or much computer training, to operate without assistance from AI experts. The COA system was evaluated by a series of challenge problems [11] generated by Alphatech, Inc. who also designed metrics for the evaluation. The integrated COA system produced a score of 82.4% correct on recall, 75.01% on precision, and a total score of 78.92% correct.
The HPKB COA system, called the “Integrated Course of Action Critiquing and Elaboration System (ICCES),” will be delivered to the U.S. Army’s Battle Command Battle Laboratory. Plans are in progress to expand the system and to improve its robustness so the critiquers can operate the system in the laboratory 24 hours per day and seven days per week. The user interface, including the sketching tool, statement editor, IPB input, and the meta-critiquer will display simultaneously the sketch, statement, and in separate windows. The fusion of textual COA statements with COA sketches will be improved to handle all possible statement and sketch inputs. The target end users of the system are the U.S. Army planners who can use the integrated tools to help generate better COA’s more rapidly. The tools also can be used to train students and junior staff. Three companies now provide ICCES. Alphatech supplies challenge problem extensions; Logica Carnegie (LGCI) is extending their CADET software; and Teknowledge is extending the HPKB critiquing software.
6 Directions for Future Technology Transfer 6.1 Research at SPAWARSYSCEN, SD Knowledge-base technology and data mining are two related technologies that are planned for deployment in the information systems of future Naval surface ships within the next decade. This requirement has contributed to a considerable interest at SPAWARSYSCEN, San Diego, in the areas of data mining and knowledge management. For example, Rubin, et al. [21] are working on algorithms for mining object classes. This contrasts with the first-generation mining algorithms that were based on the assumption that the data contained only numeric and symbolic features, and that no text, image features, or raw sensor data were present [18]. This project will test the automatic mining of search-control knowledge in the context of mining information for a pattern recognition problem of military significance. On a theoretical level, we expect results to show that the representation of knowledge must be subject to evolution, which in turn implies the need for self-reference. The Incompleteness Theorem [10] imposes certain limitations. In particular, the role of chaos in data mining is fundamental. HPKB technology developer Mitchell advocates mixedinitiative data mining in which human experts collaborate more closely with computers to form hypotheses and test them against the data [18], [22]. DesJardins and co-workers also implemented a mixed-initiative construction of application-specific KB’s in the HPKB program [2]. The mixedinitiative approach is an evolutionary paradigm. Rubin proved [20] that this mixed-initiative approach represents an on-going process of randomization – one in which the hu-
man intervention evolves to higher and higher levels until, in the theoretical limit, only a chance device can provide the requested human input. In this context, mixed-media mining [18], [22] is seen as the fusion of mining results. The work of Rubin et al. on object mining is directed toward making inroads here, although, it may be years before a practical mixed-media mining tool is commercially available and deployed on ships. The present activity of this research project involves the augmentation of data-mining problems with new features [5], a process that improves the accuracy of the predictions. Quinlan has noted that the extraction of features that relate to winning chess games is quite difficult [19]. To overcome this difficulty, whether the domain of interest is a chess game or an actual military battle, one can apply randomization theory to the problem. Software is under development to enable the SME to specify and record domain-specific features. We expect that the inclusion of such features will improve the mining accuracy by at least an order of magnitude. We believe that this technology is consistent with what Mitchell [18], [22] calls “combining supervised and unsupervised learning.” We term this random programming because the SME is programming novel features into the system that are random with respect to the data set [20] in that a more-compressed algorithm or expression could not predict these features. Thus, random programming can cause long programs or data sets become more concise. Random programming, unlike conventional programming, minimizes all manner of repetitive constructs. Many researchers are concerned with only mining in the large. As the Clementine mining-tool makers have acknowledged, nothing more than an ordinary PC is necessary for some serious mining. Humans tend to aggregate knowledge (e.g., in visual object recognition) and future Naval capabilities will require learning algorithms to do the same. Once again, our forthcoming object-oriented hierarchical-learning algorithms are suggested.
6.2 DARPA Rapid Knowledge Formation The human-machine interaction and machine-learning strategies are emphasized in DARPA’s Rapid Knowledge Formation (RKF) program, the follow-on DARPA project to HPKB [1]. The goal of RKF is to enable subject-matter experts working in distributed teams to enter knowledge directly into KB’s efficiently without the intervention of a knowledge engineer [1].
7 Summary In this paper, we discussed several applications of highperformance knowledge-based technology as sponsored by the DARPA HPKB program. These applications demon-
strate the transition potential of HPKB technology to benefit all of the armed forces. These technologies, such as Bayesian networks, have considerable potential as data-mining applications. Data mining is a key technology that is evolving rapidly to the point where it should become a key component of an intelligent machine. This machine gradually will replace the “engineer in the loop” in mining. This trend also is evident in the COA technology and in the RKF program in which SMEs aim to rely less and less on knowledge engineers. The system will evolve to supervise itself in a user-friendly state that allows knowledge input from SMEs who are not computer-literate users. Such is the promise of scaling in the large. The technology for achieving this is well within reach. Therefore, we need to focus on promising theory in support of ever-more promising applications. The role of the theorist and empiricist are intertwined as never before.
Acknowledgments The authors thank the Defense Advanced Research Projects Agency and the Office of Naval Research for their financial support of this research. This work was produced by U.S. government employees as part of their official duties and is not subject to copyright. It is approved for public release with an unlimited distribution.
References [1] M. Burke, Rapid Knowledge Formation (RKF) Program Description, http://dtsn.darpa.mil/iso/programtemp.asp?mode=331 [2] M.G. Ceruti, “Application of Knowledge-Base Technology for Problem Solving in Information-Systems Integration,” Proceedings of the DOD Database Colloquium '97, pp. 215-234, September 1997. [3] M.G. Ceruti and S.J. McCarthy, “Establishing a DataMining Environment for Wartime Event Prediction with an Object-Oriented Command and Control Database,” Proceedings of the IEEE International Symposium on Object-oriented Real-time Distributed Computing, ISORC2K, March 2000. [4] P.Cohen, R. Schrag, E. Jones, A. Pease, A. Lin, B. Starr, D. Gunning, and M. Burke, “The DARPA High Performance Knowledge Bases Project,” AI Magazine, vol. 19, no.4, Winter 1998. [5] Cycorp, Introduction to the Cyc Knowledge Server, http://www.cyc.com/cyc-course-intro.html [6] Cycorp, Features of CycL, http://www.cyc.com/cycl.html. [7] Defense Intelligence Agency, Theatre Missile defense Intelligence Preparation of the Battlespace TTP (Draft), DIA Missile and Space Intelligence Center, Redstone, AL, July, 1998.
[8] Department of the Air Force, Combat Air Forces Concept of Operations for C2 against Time Critical Targets, HQ ACC/DRAW, Langley AFB, VA, July 1997. [9] Department of the Army, Washington DC, FM 101-5, Staff Operations and Organizations, 31 May 1997 [10] K. Godel, On Formally Undecidable Propositions of Principia Mathematica and Related Systems, B. Meltzer (Translator) Dover Publisher, New York, reprinted in February 1992. [11] E. K. Jones, R. Tenney and B. Rosenberg, HPKB CP Technical Note #1: What is a Challenge Problem? http://www.iet.com/users/schrag/ALPHATECH/ memo.html [12] D. Koller and X. Boyen, “Exploiting the Architecture of Dynamic Systems,” Proceedings of the Sixteenth National Conference on Artificial Intelligence, July 1999. [13] D. Koller, X. Boyen and N. Friedman, “Discovering the Hidden Structure of Complex Dynamic Systems,” Proceedings of Uncertainty in Artificial Intelligence, 1999. [14] D. Koller and A. Pfeffer, “Probabilistic Frame-based Systems,” Proceedings of the Fifteenth National Conference on Artificial Intelligence, July 1998. [15] A.D.Lin and B. H. Starr, “HIKE (HPKB Integrated Knowledge Environment) – An Integrated Knowledge Environment for HPKB (High Performance Knowledge Bases) Proceedings of the IEEE Knowledge and Data Engineering Exchange Workshop, KDEX-98, pp. 35-41, November 1998. [16] Logica Carnegie Group, JFACC - Joint Forces Air Component Commander, http://www.cgi.com/web2/govt/jfacc.html [17] S. J. McCarthy and M.G. Ceruti, “Advanced Data Fusion for Wartime Event Correlation and Prediction,” Proceedings of the AFCEA Federal Database Colloquium '99, pp. 243- 249, September 1999. [18] T.M. Mitchell, “Machine Learning and Data Mining,” Comm. of the ACM, vol. 42, no. 11, pp. 30-36, November 1999. [19] J. R. Quinlan, Programs for Machine Learning, Morgan Kaufmann Pub., New York, 1992. [20] S.H. Rubin, “Computing with Words,” IEEE Trans. On Systems, Man, and Cybernetics, vol. 29, no. 4, pp. 518-524, August 1999. [21] S.H. Rubin, M.G. Ceruti, and R. Rush, “Knowledge Mining for Decision Support in Command and Systems,” Proceedings of the AFCEA Federal Database Colloquium and Exposition 2000, San Diego, CA, September 2000. [22] S. Thrun, C. Faloutsos, T. Mitchell, and L. Wasserman, “Automated Learning and Discovery: State of the Art and Research Topics in a Rapidly Growing Field,” AI Magazine, vol. 20, no. 3, pp. 78-82, Fall 1999.
Dr. Marion G. Ceruti (M’98, SM’99) is a scientist at the Space and Naval Warfare Systems Center, San Diego, Code D4121, 53560 Hull Street, San Diego, CA 92152-5001; Tel. (619) 553-4068,
[email protected]. She received her Ph.D. in 1979 from UCLA. Her professional activities include information systems research and analysis for command and control decision-support systems. She was the technical coordinator for the SPAWARSYSCEN, San Diego’s contracts for the HPKB program, and is the present technical coordinator for several DARPA RKF contracts. Dr. Ceruti is the author of over 50 publications in science and engineering. She is a senior member of the IEEE and a member of several other scientific and technical organizations. Mr. Craig Anken is a senior computer scientist and technical advisor for the Intelligent Information Branch at the United States Air Force Research Laboratory, 525 Brooks Road, Rome, NY 13440; Tel. (315) 330-4833,
[email protected]. He received his B.S. in Mathematics from Hobart College in 1983 and his M.S. in Computer Science from the State University of New York in 1990. Mr. Anken's professional activities include information systems research and development for Air Force command and control decision-support systems. He was technical coordinator for the U.S. Air Force's contracts for the HPKB program. Mr. Albert Lin is a Principal Software Engineer at Science Applications International Corporation. He received his B.S.E.E. degree from Feng Chia University in Taiwan in 1984 and his M.S. degree in electrical and computer engineering from San Diego State University in 1990. Mr. Lin was a technical lead for the HPKB project at SAIC. His professional interests include distributed knowledge systems, data mining, and object-oriented modeling, analysis and design. Mr. Lin is co-founder and Chief Software Architect at Be-Bee, Inc., where he may be contacted at 16835 W. Bernardo Drive, Suite 101, San Diego, CA, 92127; Tel. (858) 485-9996, ext. 14,
[email protected]. Dr. Stuart H. Rubin (M’88, SM’00) is an engineer at the Space and Naval Warfare Systems Center, San Diego, Code D73C, 53560 Hull Street, San Diego, CA 92152-5001; Tel. (619) 553-3554,
[email protected]. He is an associate professor of computer science at Central Michigan University in Mt. Pleasant, MI. He received his Ph.D. from Lehigh University, Bethlehem, PA. Dr. Rubin’s research interests include biometric authentication, fuzzy information mining, Internet-based tutoring, knowledge-based and knowledge discovery systems, randomization, and software reuse. He has published over 80 articles and one book. Dr. Rubin is a senior member of the IEEE and a member of several other scientific and technical organizations.
