A CONTROLLED NATURAL LANGUAGE BASED KNOWLEDGE REPRESENTATION APPROACH FOR AGENT BASED SIMULATION RESULTS Harshal G. Hayatnagarkar G Murali Krishna Tata Consultancy Services Ltd email:
[email protected] email:
[email protected]
ABSTRACT
D R
A
The scientific process encourages practitioners to keep record of their experiments for various aspects such as objectives, setup, observations, and insights. Computer simulation experiments unlike their physical counterparts pose a unique challenge – Scale. Due to increasing computing power and sophisticated software, scientists can perform large number of experiments on variety of scenarios. Keeping track of different aspects of these experiments in both human and machine amenable form is difficult and sometimes even unyielding. Maintaining a machine amenable form is important for automation because it aids in unearthing the insights users are looking for. Semantic electronic notebooks try to address this need by offering features to associate metadata with data. However, scientists are limited by the metadata schemas and user interface capabilities of such notebooks, while they may prefer a more traditional approach of using a natural language. In this paper, we see this two-fold challenge as a knowledge representation problem and discuss an approach based on controlled natural language to develop electronic notebooks for computer simulations. We discuss insights, capabilities, and limitations of this approach for simulations of an agent-based traffic model.
important role they play in conducting research, guidelines and best practices have emerged across disciplines to maintain quality of such notebooks. Over time, practicing the scientific method has become more complex because of several reasons such as advancements in science and technology, interdisciplinary and global collaboration, and more recently computational science (Wikipedia 2016a). To deal with such a complex process, electronic lab notebooks (ELNs) offer many enhancements such as logging, tracking, querying, reliable reuse and backup. Many offer collaboration features such as sharing, access control, data integration (e.g. budgets, calendars, inventory), and process integration (e.g. reviewing) (Elliott 2004, Rubacha et al. 2011). Recently, these ideas are studied by a new branch of information science sprung namely lab informatics (Wikipedia 2016c, Wood 2007). Computational science approach is different from traditional empirical and theoretical approaches to practice science, in that, a scientist builds mathematical models, executes experiments, gathers data and analyzes them, entirely using simulation. Computer simulations make up large part of such experiments, and hence, are known as virtual experiments (Carley 1999, Finkelstein et al. 2005). Such simulations are used to generate hypotheses instead of proving them. Thus the simulation science method specializes the scientific method by following all the necessary steps of scientific method, such as identifying simulation objectives, modeling and specifying the simulation experiment, executing the simulations, analyzing the simulation results followed by tuning the experiments. Such tuning may change any aspect of simulation, from parameters to the objectives. However, unlike their physical counterparts, simulation experiments are generally performed in large number and with more complexity due to rapid advances in computing power. This increase in scale of experiments poses three challenges: a) It is difficult to document such large number of simulations. b) Due to such large number, it is imperative to ensure human as well as machine amenable documentation. c) Discovering analysis techniques that the user community finds useful. Scientists running simulations capture simulation results in a broad spectrum of forms, from unstructured
FT
KEYWORDS Agent-Based Systems, Controlled Natural Language, Knowledge Representation, Simulation Results Analysis, Lab Informatics
INTRODUCTION The scientific method is an iterative process that includes making observations, thinking of interesting questions, formulating hypotheses, developing testable predictions, gathering data on these predictions and refining the hypotheses, and this rigor presupposes meticulous documentation. Historically, laboratory notebooks acted as primary record of the research and also serve as chronological record, organization tool, store of data and insights related to observations, knowledge sharing documents, and as a legal proof of the research. Given the
Figure 1: Traffic grid simulation in Netlogo
FT
few observational variables. Users can experiment with the traffic grid dynamics by changing control variables such as grid size, number of cars on the roads, speed limit, and power status of traffic lights. Number of stopped cars, average wait time of cars, and average speed of cars are observed to understand effect of chosen values of control variables on the overall dynamics of the grid, as rendered by three plots.
D R
A
text to highly structured text, tables and other forms. The scientists then analyze these results, either automatically for structured data or manually for unstructured data. However, when faced with scale challenge, currently scientists resolve to as much automation as possible, by shifting to and by relying on structured data. For unstructured data especially text, many scientists develop a kind of short-hand notation, which is very much restricted to individuals or very small closely working groups. Such notations usually capture aspects such as qualitative insights and annotations, and are kept informally in separate documents. Such issues limit what can be linked, extracted, tracked, and verified, and perhaps automated using tool-chains or processes. In addition, it creates silos and barriers in the collaboration. Such practices are time consuming, limiting and error prone. Even if the observations and insights are meticulously saved in a notebook, retrieving them becomes cumbersome. With growing number of experiments, the whole process becomes a bottleneck. Semantics is another important aspect in this process. When users find that their tools cannot be adapted for scientific jargon, and they must mentally translate these tools’ jargon, a barrier is established between the users and the tools. Recently, semantic electronic notebooks offer some help here (Coles et al. 2013, Adams et al. 2013). In an ideal scenario, users should have lab notebooks and life-cycle management tools that are integrated. However, considering the complexity of such tools, these can be developed separately, while being aware of the larger problem. In this paper, we explore a knowledge-representation approach based on controlled natural language(CNL), for agent-based simulations. We began this by conducting several simulation experiments using an agent-based model of traffic grid system, and recorded our observations using natural language in a notebook. As we found patterns in notebook entries, we first hypothesized and then transferred the contents of this notebook into a controlled natural language. Our approach provided a means to capture information not only in a formal manner but also in a natural, user-friendly, and collaborative way. In the subsequent sections, we discuss details of each of the major steps of our exploration. TRAFFIC GRID SIMULATION
To understand the process of recording in a scientific lab notebook for agent-based simulation, we chose to conduct experiments using a traffic grid model (Wilensky 2003), which is part of a popular agent-based modeling and simulation toolkit namely Netlogo (Tisue and Wilensky 2004) shown in figure 1. The model simulates dynamics of a traffic grid system, which comprises a road network, traffic lights and cars. The model exposes a few control variables and renders a
Figure 2: Simulation purpose, parameters and observational variables We conducted simulation experiments by changing control parameters especially number of cars and speed
grid related terms like traffic, car, traffic light etc. We bucketed them as per their nearness to each other, and assigned a unique color to each bucket so as to visually recognize these keywords, such as green for statistics, brown for traffic grid domain, violet for complexity, and khaki for interesting but undecided (with superscripts $, #, @, and ? signs respectively) as shown in figure 3. The coloring helps in many ways, the most significant is pattern identification, such that which keywords are collocated, or participate in describing a construct. For example, domain keywords dominate purpose and other constructs of the context, statistics and complexity related keywords dominate observations and insights. Based on the closeness of keywords with each other and their affinity to constructs in a context, we could see that each bucket represents a domain. To convey domain ideas, constraints, etc., we need an expressive enough language. In a language oriented perspective, we thus believe each of these domains represents a language by itself. In discussion section, we elaborate this idea further.
D R
A
FT
limit, to understand the effect of these changes to the traffic. We articulated such motivations in the notebook, becoming part of the life-cycle of an experiment. With the spectrum of values for each control variable, we could observe rich dynamics in the model, concluding that the model is not only sufficiently simple to grasp but also complex enough to be interesting. This exercise allowed us to observe how the complexity inherent to notebooks emerges out. Given that the logging required us to consider numerical values, text, and snapshots of rendered plots, we used a word processing software for logging. Excerpts of this notebook are shown in figures 2, 3, 4, and 5. In the remaining of section, we see how these excerpts bring out need for more such features for a notebook.
Figure 3: Multiple domain languages are inherent in notebooks Figure 2 describes context of the model such as its purpose (motivation), input and output parameters, and insights. While describing these constructs, we identified few keywords such as concepts, objects, actions. Some of these keywords are related, for example traffic
Figure 4: Temporality
Mixing of text, numbers, equations, diagrams, tables, and images makes notebooks rich and expressive. In figure 5 it can be seen that the notebook entry has text, table, and images to convey the observations. Natural language allows users to convey ideas like temporality (such as before, after, around, and eventually as shown in figure 4). Figure 5 shows a batch simulation where observations refer to each other. In our observation, we described the first experiment with many details, and for subsequent experiments describe only additions or deviations from this first experiment to avoid repetition. This kind of logging assumes inherent cross-linking based on position an experiment or an entry in the notebook. We expect that such scenarios are common for batch experiments. In the next section, we describe our first steps to identify the solution space.
USING A CONTROLLED NATURAL LANGUAGE
D R
A
From the traffic grid simulation experiment, we observed that a scientific notebook would need a host of syntactic and semantic features. Syntactic features, which are associated with form and format of information, include support for text with formatting, numbers, equations, tables, images, and plots. To implement semantic features which are associated with meaning of information, a notebook should be aware of multiple domains, such traffic grid, statistics, and complexity. Ontology support can cover classification (taxonomy and typology) (Marradi 1990), and meronomy semantics. Such a support can also enable inferences using first-order logic based reasoners. However, a few features require to go beyond first-order logic, such as support of statistical operations for statistics, and temporal logic (Manna and Pnueli 1981) with time-series analysis for temporality. Complexity domain requires integration of ideas from other domains to connect to its own abstractions, such as emergence and tipping point use network theory and temporality. In presence of such multiple domains, collaboration among experts becomes necessary, hence it is desired to have such collaboration features in notebooks. In a nutshell, we realized that a knowledge representation system (Davis et al. 1993) with these additional features can make a good notebook. To select a knowledge representation system that fits our needs, we surveyed existing systems and frameworks (Boose 1989, Cadoli and Donini 1997, Trentel-
FT
Figure 5: Reference structure in a batch experiment
man 2009). As we expected, most systems in these surveys had some level of ontology and reasoning support. Out of all systems we surveyed, (Shapiro 2000) and (Fuchs et al. 2006a) support a natural language based knowledge acquisition. To reduce ambiguity and context dependent meaning, (Wasow et al. 2005) recommends fusion of formal and natural languages. We came across such an approach by (Fuchs et al. 2006a) which uses Attempto Controlled English (ACE), a restricted, subset of English, is computer-translatable. (Schwitter 2010, Kuhn 2014) discuss alternatives to ACE, but we chose ACE as the CNL because it is less complex, more predictable, open source, and has many tools written around it such as AceWiki (Kuhn 2008), creating an ecosystem. We learned that ACE has been already used in similar context of knowledge representation (Fuchs et al. 2008; 2006b), for writing clinical guidelines (Shiffman et al. 2010). We chose AceWiki software (Kuhn 2008; 2009), which is a semantic wiki, based on a subset of ACE, with reasoning, sentence composition and collaboration support. AceWiki organizes its language constructs such as nouns, verbs, associations, and others as articles. A user has to map domain ontology to these constructs. Articles can contain sentences and comments. A sentence can be either an assertion, or a rule, or a question. Comments are ignored during processing. AceWiki combines features from an ontology language OWL (Antoniou and Harmelen 2009) for modeling, a semantic reasoner (Wikipedia 2016d) for inferences, and SWI Prolog (Wielemaker et al. 2012) for CNL processing. The wiki supports two reasoners namely HermiT (Horrocks et al. 2012) and Pellet (Sirin et al. 2007). For our exercise, we chose Pellet as it provided better reasoning support over HermiT. Pellet also gave better performance. We also retained default OWL language profile OWL 2 Full for AceWiki, due to expressiveness of this profile. CNL processing is done by an external program namely ‘Attempto Parsing Engine’(APE) (Norbert et al. 2005) which is a part of the ACE ecosystem. These features are wrapped into a Java-based web application. We required to define entities such as Simulation, Agent based simulation, and Traffic simulation, along with setting up relationships among them, as shown in figures 6, 7, and 8 respectively. The sentences which are too complex to be reasoned by the reasoner are rendered by AceWiki in red color, (as pointed by an arrow) in figure 8. Even though AceWiki provides a collaborative semantic environment, it can use only a subset of ACE, restricted by its sentence composition user interface. Even with full ACE (directly, without AceWiki) we could not recreate the traffic grid notebook due to inherent complexity of the notebook. Hence, for AceWiki we carved out a simpler version from the original notebook for traffic grid model. We populated AceWiki with a few experiments based on this version of the notebook that we
DISCUSSION AND FUTURE WORK In previous sections, we discussed our motivation for a notebook for simulation scientists, observations by writing a manual notebook ourselves, and intuition about a CNL-based approach towards realizing a notebook with required features. In this section, we discuss and evaluate our experience of using AceWiki, perform matchand-gap analysis for our needs, and pick up interesting takeaways for future work. Mapping a Model to AceWiki CNL Constructs
Figure 6: AceWiki article - Simulation
Here we need to map domain modeling (Wikipedia 2016b) ideas to CNL constructs.
Object property Rules
Singleton instances
Experiments Loading Time 12 7 50 7 100 8 500 8
D R
A
created. Logs of two representational experiments are shown in figures 9 and 10. We created an article ‘Simulation dashboard’, shown in figure 11. The users’ queries related to the simulation results are logged here. AceWiki renders the query results automatically using the provided (in this case, Pellet) reasoner. In addition, to test out scalability, we synthetically generated more experiment data. We evaluated AceWiki performance for loading data and pages, and reasoning, for large number of experiments. AceWiki used Pellet reasoner version 2.3.1, and default OWL2 Full profile for maximum expressiveness. AceWiki was running on a server with Intel Core i7 2600k processor, 16 GB RAM, Fedora 24 64-bit Linux operating system, Oracle Java Runtime 8 update 92, and Apache Tomcat 8.0.36 application server. We had given 8GB of dedicated heap to Tomcat server. For AceWiki client, we used the same machine with Firefox 47.0.1. Results of this performance evaluation are shared in table 1. Reasoning Time
14s 360s crashed after 1200s crashed after 1200s
Table 1: Performance of AceWiki for loading data, and reasoning in Simulation dashboard page In spite of such a short exercise, AceWiki offered an interesting perspective into if and how a CNL-based approach can be considered by simulation science community for maintaining notebooks. In following section, we elaborate this perspective further.
CNL construct noun
Examples
verb (verb + nouns) or of-construct Verb + noun sequence If statements
completes within Has exactly 200 Cars
FT
Figure 7: AceWiki article - Agent based Simulation
Modeling construct Concept, entity or class Action Data property
proper noun
Simulation, Car
Has a Motivation If a Simulation studies formation-of a Traffic Jam then the Simulation studies a Traffic Jam the Wall-clock(shared by all simulations)
Table 2: Mapping of modeling constructs to CNL constructs We first share limitations of our approach, and then move on to describe advantages. Limitations AceWiki does not provide many features that are mentioned in traffic grid simulation section, such as formatting of text, embedding tables and media, and full expressiveness of ACE itself. AceWiki also currently does not support consistency checking across statements within an article. For example, a statement every traffic simulation has at least 1 car puts cardinality constraint on an instance of Traffic simulation, which goes undetected. One modeling related issue was about how a simulation variable be represented, instantiated and referred within the constraints of AceWiki. As we understand, a variable in AceWiki can be either a noun or a proper noun or an of-construct. Each choice offers a trade-off depending upon what the choice gets translated into,
Figure 8: AceWiki article - Traffic simulation
inside the AceWiki framework, as discussed below:
A
Figure 10: AceWiki article - Traffic simulation experiment 2
FT
Figure 9: AceWiki article - Traffic simulation experiment 1
D R
• A noun becomes an OWL class which needs to be instantiated. In most simulations, a property is desired to be a noun so that when it is defined at the concept level such as Traffic Simulation, then for each Traffic Simulation, one just needs to assign the values. However, such an assignment means an instantiation with a unique identifier, which can be referred from elsewhere in the wiki. In the context of CNL, this instantiation should happen inline and transparently. No such feature is offered by AceWiki, and hence one has to look towards other two possibilities. Instead, in AceWiki one can instantiate, but the instantiation is not reflected in answers to the queries, because it lacks unique identification.
• A proper noun becomes a singleton instance (a singleton instance is the only instance of its own class). It is shared by its referring instances, which means that once you alter its value in a simulation, then all other simulation instances referring to it reflect that change. For modeling variables, this choice cannot, thus be made when each simulation needs its own
Figure 11: AceWiki article - Simulation dashboard
references. In addition, a proper noun does not appear in answers to queries such as Which variable causes a traffic jam?. • The of-construct becomes an object property, which has a domain (to which this property is attached), and a range (a data-type of this property, such as numerical, string, or an object). AceWiki Geography demo wiki makes use of this approach to model ‘president-of’ property of a country (Attempto 2013), and can be queried easily and naturally. However, in case of simulations, representing and querying for this construct is sufficient for literal value variables, but not for association. For
We chose AceWiki for prototyping so that we could experiment, learn, adapt, and improve our ideas. Without following advantages, we had to come up with something our own, perhaps close to the AceWiki. • AceWiki provided to us unified interface to define ontology for simulation experiments, specify details for each experiment, ask queries, and get query results. In absence of such unified interface, we would not have gotten ability to reason so effectively using just CNL. • User interface of AceWiki provides guided sentence composition. • Web-based interface for collaboration is quite useful. • AceWiki has relatively modular architecture, which allowed us to swap one OWL reasoner by another.
• Many simulations may need some analysis to be done as part of reasoning. For example, finding minimum of certain values, or to find most correlated pair of variables and so on. If the environment can support such operations from within, users may find it easier and friendlier. It is imperative that such operations to be offloaded to a data analysis platform. • A simulation unfolds dynamics of its model over time. Thereafter the user analyzes this temporal data for pattern. Supporting temporal concepts such as before, after, and eventually is essential for this environment to remain relevant for such users. A combination of temporal logic and timeseries analysis, we believe, can reason about time sequences as shown in 5. • For features like consistency checking and temporality, we believe that formal methods languages and related tools like model checkers and theorem provers can be useful. We may either evaluate a theorem prover namely Reasoner for ACE Fuchs and Schwertel (2003) from ACE ecosystem, or add support for others such as Z (Bowen 1996), TLA (Lamport 1994), SAL (Bensalem et al. 2000), or Alloy (Jackson 2002).
• We observed that the users may want to refer to parts of an experiment or complete experiment from other experiments, as shown in figure 5. Hence, the environment should have a mechanism to enable the users this kind of referencing.
A
• AceWiki can export and import data, which facilitate to generate experiment data by a simulation engine. However, currently the AceWiki can load this data only during startup, and does not provide an online service to inject data into a running instance of AceWiki.
• The environment should guide its users to follow the life-cycle process, and provide necessary feedback for course correction.
FT
example, queries such as Which Condition causes a Traffic Jam? cannot be written using of-construct, even though simulations explicitly record A condition that Traffic Lights were Off causes a Traffic Jam.
Future work
D R
• AceWiki supports many formats for exporting data to be consumed by external applications such as .owl files for OWL editors.
We started with an environment which supports controlled natural-like language, reasoning, and collaboration. Given the gaps and challenges we faced during the notebook exercise, we intend to reduce some of these barriers as discussed below. • As we discovered in the exercise, seamless integration of multiple domain languages is very important. In future, we desire to explore possibilities for supporting such languages in a systematic manner. • Of these modeling languages, we believe that simulation life-cycle process language is the most important because it forms the basis on which experiments can be described. Hence, we envision that this language should directly be built in the environment.
• Improving richness of the specification with text formatting, tables, and media that form part of the languages. • To support large number of experiments, we intend to scale up performance of environment. The traffic grid experiment helped us in realizing the complexity that is inherent in simulation experiments, however, in order to ensure identification of all challenges, we would like to use a real-life case study as our next experiment. CONCLUSION In this paper, we presented a novel application combining knowledge representation with a controlled natural language (ACE in our case) and its accompanying reasoning feature in the context of lab notebooks for agentbased simulation experiments. We began with our motivation which is role of lab notebooks, features expected
ACKNOWLEDGEMENTS We would like to acknowledge Dr. Madhav Marathe (Director of Network Dynamics and Simulation Science Laboratory, Biocomplexity Institute of Virginia Tech), Dr. Swaminathan Natarajan (Principal Scientist, TCS Research), and Mr. Kishore Padmanabhan (Vice President, TCS Ltd) for their inspiration and support to work on this idea. REFERENCES
Carley K.M., 1999. On generating hypotheses using computer simulations. Tech. rep., DTIC Document. Coles S.J.; Frey J.G.; Bird C.L.; Whitby R.J.; and Day A.E., 2013. First steps towards semantic descriptions of electronic laboratory notebook records. Journal of cheminformatics, 5, no. 1, 1. Davis R.; Shrobe H.; and Szolovits P., 1993. What is a knowledge representation? AI magazine, 14, no. 1, 17. Elliott M.H., 2004. Are ELNs Really Notebooks? Scientific Computing & Instrumentation, 6. Finkelstein N.; Adams W.; Keller C.; Kohl P.; Perkins K.; Podolefsky N.; Reid S.; and LeMaster R., 2005. When learning about the real world is better done virtually: A study of substituting computer simulations for laboratory equipment. Physical Review Special Topics-Physics Education Research, 1, no. 1, 010103. Fuchs N.E.; Kaljurand K.; and Kuhn T., 2008. Attempto Controlled English for knowledge representation. In Reasoning Web, Springer. 104–124. Fuchs N.E.; Kaljurand K.; and Schneider G., 2006a. Attempto Controlled English Meets the Challenges of Knowledge Representation, Reasoning, Interoperability and User Interfaces. In FLAIRS Conference. vol. 12, 664–669.
D R
A
Adams N.; Haller A.; Krumpholz A.; and Taylor K., 2013. A semantic lab notebook-report on a use case modelling an experiment of a microwave-based quarantine method. In Proceedings of the 3rd International Conference on Linked Science-Volume 1116. CEURWS. org, 46–58.
Cadoli M. and Donini F.M., 1997. A survey on knowledge compilation. AI Communications, 10, no. 3, 4, 137–150.
FT
from an ideal electronic notebook for simulation science, and deficiencies in current available software. We discovered that abilities to write in as natural a language as possible, and to reason about the contents of notebooks are largely missing. We then shared our experience of manually maintaining such a notebook for a simple yet rich enough agent-based simulation. We identified the features a notebook would enable users to handle simulation experiments with rigor, that not only provides users with intuitive capabilities, but also provide reasoning support. After considering various options, we chose AceWiki and migrated our notebook to it as much possible. We elaborated our experience of using AceWiki, including insights, strengths and limitations of our approach. Finally, we conclude with possible future work, for example extending support for domains and reasoning, and scaling up performance.
Antoniou G. and Harmelen F.v., 2009. Web Ontology Language: OWL, Springer Berlin Heidelberg, Berlin, Heidelberg. ISBN 978-3-540-92673-3, 91–110. doi: 10.1007/978-3-540-92673-3 4. URL http://dx.doi. org/10.1007/978-3-540-92673-3_4. Attempto, 2013. AceWiki - Geography Demo Wiki. URL http://attempto.ifi.uzh.ch/webapps/ acewikigeo/. Bensalem S.; Ganesh V.; Lakhnech Y.; Munoz C.; Owre S.; Rueß H.; Rushby J.; Rusu V.; Saıdi H.; Shankar N.; et al., 2000. An overview of SAL. In Proceedings of the 5th NASA Langley Formal Methods Workshop. Boose J.H., 1989. A survey of knowledge acquisition techniques and tools. Knowledge acquisition, 1, no. 1, 3–37. Bowen J., 1996. Formal specification and documentation using Z: A case study approach, vol. 66. International Thomson Computer Press.
Fuchs N.E.; Kaljurand K.; and Schneider G., 2006b. Attempto Controlled English Meets the Challenges of Knowledge Representation, Reasoning, Interoperability and User Interfaces. In FLAIRS 2006. Fuchs N.E. and Schwertel U., 2003. Reasoning in attempto controlled english. In International Workshop on Principles and Practice of Semantic Web Reasoning. Springer, 174–188. Horrocks I.; Motik B.; and Wang Z., 2012. The HermiT OWL reasoner. In In Proc. Jackson D., 2002. Alloy: a lightweight object modelling notation. ACM Transactions on Software Engineering and Methodology (TOSEM), 11, no. 2, 256–290. Kuhn T., 2008. Acewiki: A natural and expressive semantic wiki. arXiv preprint arXiv:08074618. Kuhn T., 2009. How controlled English can improve semantic wikis. arXiv preprint arXiv:09071245. Kuhn T., 2014. A survey and classification of controlled natural languages. Computational Linguistics, 40, no. 1, 121–170.
Lamport L., 1994. The temporal logic of actions. ACM Transactions on Programming Languages and Systems (TOPLAS), 16, no. 3, 872–923.
Wielemaker J.; Schrijvers T.; Triska M.; and Lager T., 2012. SWI-Prolog. Theory and Practice of Logic Programming, 12, no. 1-2, 67–96. ISSN 1471-0684.
Manna Z. and Pnueli A., 1981. Verification of concurrent programs. part i. the temporal framework. Tech. rep., DTIC Document.
Wikipedia, 2016a. Computational science — Wikipedia, The Free Encyclopedia. URL https://en.wikipedia.org/w/index.php?title= Computational_science&oldid=728726122. [Online; accessed 8-July-2016].
Norbert R.; Fuchs E.; Schwertel P.B.U.; Fuchs N.E.; Kaljur K.; Rinaldi F.; and Schneider G., 2005. A Parser for Attempto Controlled English. Rubacha M.; Rattan A.K.; and Hosselet S.C., 2011. A review of electronic laboratory notebooks available in the market today. Journal of the Association for Laboratory Automation, 16, no. 1, 90–98. Schwitter R., 2010. Controlled natural languages for knowledge representation. In Proceedings of the 23rd International Conference on Computational Linguistics: Posters. Association for Computational Linguistics, 1113–1121. Shapiro S.C., 2000. SNePS: A logic for natural language understanding and commonsense reasoning. Natural language processing and knowledge representation: language for knowledge and knowledge for language, 175–195.
Wikipedia, 2016c. Laboratory informatics — Wikipedia, The Free Encyclopedia. URL https://en.wikipedia.org/w/index.php?title= Laboratory_informatics&oldid=714490247. [Online; accessed 8-July-2016]. Wikipedia, 2016d. Semantic reasoner. URL https://en.wikipedia.org/w/index.php?title= Semantic_reasoner&oldid=728824399. [Online; accessed 10-July-2016]. Wilensky U., 2003. NetLogo traffic grid model. Northwestern University, Evanston, IL: Center for Connected Learning and Computer-Based Modeling. URL http://ccl.northwestern.edu/netlogo/ models/TrafficGrid. Wood S., 2007. Comprehensive laboratory informatics: A multilayer approach. American Laboratory, 39, no. 16, 20.
D R
A
Shiffman R.N.; Michel G.; Krauthammer M.; Fuchs N.E.; Kaljurand K.; and Kuhn T., 2010. Writing Clinical Practice Guidelines in Controlled Natural Language. In N.E. Fuchs (Ed.), Proceedings of the Workshop on Controlled Natural Language (CNL 2009). Springer, Berlin / Heidelberg, Germany, Lecture Notes in Computer Science, vol. 5972. ISBN 9783-642-14417-2, 265–280.
Wikipedia, 2016b. Domain model. URL https://en.wikipedia.org/w/index.php?title= Domain_model&oldid=722714741. [Online; accessed 10-July-2016].
FT
Marradi A., 1990. Classification, typology, taxonomy. Quality & Quantity, 24, no. 2, 129–157.
Sirin E.; Parsia B.; Grau B.C.; Kalyanpur A.; and Katz Y., 2007. Pellet: A practical OWL-DL reasoner. Web Semantics: Science, Services and Agents on the World Wide Web, 5, no. 2, 51 – 53. ISSN 1570-8268. doi:http://dx.doi.org/10.1016/j.websem. 2007.03.004. URL http://www.sciencedirect.com/ science/article/pii/S1570826807000169. Software Engineering and the Semantic Web. Tisue S. and Wilensky U., 2004. Netlogo: A simple environment for modeling complexity. In International conference on complex systems. Boston, MA, vol. 21, 16–21. Trentelman K., 2009. Survey of knowledge representation and reasoning systems. Tech. rep., DTIC Document. Wasow T.; Perfors A.; and Beaver D., 2005. The puzzle of ambiguity. Morphology and the web of grammar: Essays in memory of Steven G Lapointe, 265–282.
