Roman Lukyanenko Jeffrey Parsons and Yolanda Wiersma .... Corporate settings. â« Health IS ... See: Lukyanenko and Parsons 2013b; Lukyanenko and.
Impact of Conceptual Modeling on Information Completeness Roman Lukyanenko Jeffrey Parsons and Yolanda Wiersma Florida International University Memorial University Presented: Dec 2014 @ ICIS in New Zealand
FACULTY OF BUSINESS ADMINISTRATION
WWW.BUSINESS.MUN.CA
Outline • • • • • • •
Background and Motivation Research Problem: IQ in UGC Theoretical Propositions Field Experiment Contributions Future Work Note: this presentation is based on: Lukyanenko et al. 2014
Conceptual Modeling and Information Completeness
2
Background and Motivation • Traditionally, IS are used in well-controlled information production settings • Increasingly, organizations turn to data produced outside org. boundaries Social media, crowdsourcing facilitate usergenerated content (UGC): • Various forms of digital information produced by members of the general public – often casual content contributors (the crowd) – rather than by employees or others closely associated with an organization
Conceptual Modeling and Information Completeness
3
Harnessing UGC • UGC supports decision making and operations Businesses • better understand customers, develop products
Health care • telemedicine, doctor reviews
Government • public services, disaster management
Scientific research • citizen science Conceptual Modeling and Information Completeness
4
Example: Citizen Science
eBird.org
Conceptual Modeling and Information Completeness
5
Research Problem: IQ in UGC • Major challenge in making effective use of UGC is crowd IQ E.g., accuracy of a citizen science observation on eBird.org
Gura 2013 Nature Conceptual Modeling and Information Completeness
6
Limitations of existing approaches
• Traditional approach is ‘fitness for use’ • Popular approaches to crowd IQ Educate and train online users Provide data collection instructions “Clean” data post-hoc
• Focus on data consumers Dissuade contributors from providing data Prevent contributors from communicating important situational knowledge Conceptual Modeling and Information Completeness
7
Proposed: Contributor-centric, useagnostic IQ
• IQ from contributors’ perspective Crowd IQ: “the extent to which stored information represents the phenomena of interest to data consumers, as perceived by information contributors” • Use-agnostic, contributor-centric • Cognizant of data consumers
• How to design IS sensitive to contributors? Rethink approaches to conceptual modeling Conceptual Modeling and Information Completeness
8
Proposed: Conceptual modeling as a factor of crowd IQ
• Conceptual modeling “describing some aspects of the physical and social world around us for the purposes of understanding and communication” (Mylopoulos 1992)
Conceptual Modeling and Information Completeness
9
Theoretical Propositions • Classification Accuracy: Class-based information models result in lower information accuracy (more classification errors) when classes defined in an information system do not match those familiar to the information contributor.
• Property Loss. Class-based information models result in property loss when the class that a contributor uses to record an instance does not imply some properties of the instance observed by contributor.
• Dataset completeness. Class-based information models result in lower dataset completeness (e.g., number of instances recorded) when the classes and properties that a contributor intends to record are not defined in an information system.
Conceptual Modeling and Information Completeness
10
Alternative: Instance-based modeling
• Theoretical Foundations Bunge’s ontology: • world is made of individual objects (that people group into classes) • objects / instances possess attributes
Cognitive psychology: • diversity of classifications (even for the same observer)
• Proposed Solution: No a priori modeling of data objects Storing data objects in a flexible (e.g., instance-based) database
Conceptual Modeling and Information Completeness
11
Impact on dataset completeness
• Field Experiment using NLNature Class-based condition (species-only) Instance-based condition
• Hypotheses: H-4.1 More instances observed in the instancebased condition H-4.2 More instances of novel (i.e., not present in existing schema) species in the instancebased condition
Conceptual Modeling and Information Completeness
12
Scientists then use data to:
Observe wildlife
www.nlnature.com
NLNATURE.COM
Conceptual Modeling and Information Completeness
NLNature.com
• • • • • • • •
URL: www.nlnature.com Founded: 2009 All Newfoundland and Labrador ~500 members ~1,700 sightings > 2,500 photos uploaded > 4,000 sightings likes > 450,000 sighting views 14
Conceptual Modeling and Information Completeness
14
Instance-based condition
Conceptual Modeling and Information Completeness
15
Species-only condition
Conceptual Modeling and Information Completeness
16
Results: H-4.1 • Period: June to Dec (6 months) No of users in condition
No of observations
Class-based
42
87
Instance-based
39
390
Condition
Class-based model
avg. p < 0.01* Dataset completeness -instances stored
Instance-based model * Based on permutation test Conceptual Modeling and Information Completeness
17
Results: H-4.2 • Period: June to Dec (6 months) Condition
No of users in condition
No of new species
Class-based
42
7
Instance-based
39
119
Class-based model
avg. p < 0.01* Dataset completeness -novel species stored
Instance-based model * Based on permutation test Conceptual Modeling and Information Completeness
18
Summary of Findings • Conceptual modeling affects dataset completeness Prevailing class-based approaches may result in lower dataset completeness Existing IS may preclude discovery of new classes of instances
• Potential value of the proposed instancebased approach for modeling UGC
Conceptual Modeling and Information Completeness
19
Contributions • Impact of conceptual modeling on information quality Prevailing class-based modeling may have detrimental impact on IQ Antecedents of IQ: "a significant gap in the IS research” (Petter et al. 2013, p. 30)
• Contributor-oriented IQ • Instance-based conceptual modeling More effective ways to harness UGC Exemplar of an “[e]xciting ..work” exploring “new technological environments” (Goes MISQ Editorial 2014, p. vi)
Conceptual Modeling and Information Completeness
20
Future work • Deeper understanding of the impact of modeling on IQ: Impact on Accuracy Interaction between modeling and familiarity • Contributor-oriented IQ management Impact on decision making (in-progress: study with data consumers)
• Beyond citizen science Corporate settings Health IS, telemedicine
Conceptual Modeling and Information Completeness
21
Future work (cont’d) • Extending instance-based approach to conceptual modeling How to combine it with traditional modeling? Do we need “instance-based” grammars? • See: Lukyanenko and Parsons 2013a
How to better manage attribute-based data collection Implications for user interfaces • See: Lukyanenko and Parsons 2013b; Lukyanenko and Parsons 2013c;
Conceptual Modeling and Information Completeness
22
References • Lukyanenko, R., Parsons, J., and Wiersma, Y. (2014). The Impact of •
•
•
Conceptual Modeling on Dataset Completeness: A Field Experiment. International Conference on Information Systems, 19 pp. Lukyanenko, R. and Parsons J. (2013a). Is Traditional Conceptual Modeling Becoming Obsolete? In W. Ng, V.C. Storey, and J. Trujillo (Eds.), International Conference on Conceptual Modeling (ER 2013), Lecture Notes on Computer Science Vol. 8217, Springer, Heidelberg. pp. 61-73. Lukyanenko, R. and Parsons, J. (2013b). Reconciling theories with design choices in design science research. In J. vom Brocke et al. (Eds.), International Conference on Design Science Research in Information Systems and Technologies (DESRIST 2013), Lecture Notes on Computer Science Vol. 7939, Springer Berlin / Heidelberg. pp. 165-180. Lukyanenko, R. and Parsons J. (2013c). Implementing Lightweight Conceptual Modeling for User-Generated Content: Prototype. Workshop on Information Technologies and Systems (WITS 2013). [prototype paper].
Conceptual Modeling and Information Completeness
23
THANK YOU! FACULTY OF BUSINESS ADMINISTRATION
WWW.BUSINESS.MUN.CA