Impact of Conceptual Modeling on Information ...

Impact of Conceptual Modeling on Information Completeness Roman Lukyanenko Jeffrey Parsons and Yolanda Wiersma Florida International University Memorial University Presented: Dec 2014 @ ICIS in New Zealand

FACULTY OF BUSINESS ADMINISTRATION

WWW.BUSINESS.MUN.CA

Outline • • • • • • •

Background and Motivation Research Problem: IQ in UGC Theoretical Propositions Field Experiment Contributions Future Work Note: this presentation is based on: Lukyanenko et al. 2014

Conceptual Modeling and Information Completeness

2

Background and Motivation • Traditionally, IS are used in well-controlled information production settings • Increasingly, organizations turn to data produced outside org. boundaries  Social media, crowdsourcing facilitate usergenerated content (UGC): • Various forms of digital information produced by members of the general public – often casual content contributors (the crowd) – rather than by employees or others closely associated with an organization


3

Harnessing UGC • UGC supports decision making and operations  Businesses • better understand customers, develop products

 Health care • telemedicine, doctor reviews

 Government • public services, disaster management

 Scientific research • citizen science Conceptual Modeling and Information Completeness

4

Example: Citizen Science

eBird.org


5

Research Problem: IQ in UGC • Major challenge in making effective use of UGC is crowd IQ  E.g., accuracy of a citizen science observation on eBird.org

Gura 2013 Nature Conceptual Modeling and Information Completeness

6

Limitations of existing approaches

• Traditional approach is ‘fitness for use’ • Popular approaches to crowd IQ  Educate and train online users  Provide data collection instructions  “Clean” data post-hoc

• Focus on data consumers  Dissuade contributors from providing data  Prevent contributors from communicating important situational knowledge Conceptual Modeling and Information Completeness

7

Proposed: Contributor-centric, useagnostic IQ

• IQ from contributors’ perspective  Crowd IQ: “the extent to which stored information represents the phenomena of interest to data consumers, as perceived by information contributors” • Use-agnostic, contributor-centric • Cognizant of data consumers

• How to design IS sensitive to contributors?  Rethink approaches to conceptual modeling Conceptual Modeling and Information Completeness

8

Proposed: Conceptual modeling as a factor of crowd IQ

• Conceptual modeling  “describing some aspects of the physical and social world around us for the purposes of understanding and communication” (Mylopoulos 1992)


9

Theoretical Propositions • Classification Accuracy: Class-based information models result in lower information accuracy (more classification errors) when classes defined in an information system do not match those familiar to the information contributor.

• Property Loss. Class-based information models result in property loss when the class that a contributor uses to record an instance does not imply some properties of the instance observed by contributor.

• Dataset completeness. Class-based information models result in lower dataset completeness (e.g., number of instances recorded) when the classes and properties that a contributor intends to record are not defined in an information system.


10

Alternative: Instance-based modeling

• Theoretical Foundations  Bunge’s ontology: • world is made of individual objects (that people group into classes) • objects / instances possess attributes

 Cognitive psychology: • diversity of classifications (even for the same observer)

• Proposed Solution: No a priori modeling of data objects  Storing data objects in a flexible (e.g., instance-based) database


11

Impact on dataset completeness

• Field Experiment using NLNature  Class-based condition (species-only)  Instance-based condition

• Hypotheses:  H-4.1 More instances observed in the instancebased condition  H-4.2 More instances of novel (i.e., not present in existing schema) species in the instancebased condition


12

Scientists then use data to:

Observe wildlife

www.nlnature.com

NLNATURE.COM


NLNature.com

• • • • • • • •

URL: www.nlnature.com Founded: 2009 All Newfoundland and Labrador ~500 members ~1,700 sightings > 2,500 photos uploaded > 4,000 sightings likes > 450,000 sighting views 14


14

Instance-based condition


15

Species-only condition


16

Results: H-4.1 • Period: June to Dec (6 months) No of users in condition

No of observations

Class-based

42

87

Instance-based

39

390

Condition

Class-based model

avg. p < 0.01* Dataset completeness -instances stored

Instance-based model * Based on permutation test Conceptual Modeling and Information Completeness

17

Results: H-4.2 • Period: June to Dec (6 months) Condition

No of users in condition

No of new species

Class-based

42

7

Instance-based

39

119

Class-based model

avg. p < 0.01* Dataset completeness -novel species stored

Instance-based model * Based on permutation test Conceptual Modeling and Information Completeness

18

Summary of Findings • Conceptual modeling affects dataset completeness  Prevailing class-based approaches may result in lower dataset completeness  Existing IS may preclude discovery of new classes of instances

• Potential value of the proposed instancebased approach for modeling UGC


19

Contributions • Impact of conceptual modeling on information quality  Prevailing class-based modeling may have detrimental impact on IQ Antecedents of IQ: "a significant gap in the IS research” (Petter et al. 2013, p. 30)

• Contributor-oriented IQ • Instance-based conceptual modeling  More effective ways to harness UGC Exemplar of an “[e]xciting ..work” exploring “new technological environments” (Goes MISQ Editorial 2014, p. vi)


20

Future work • Deeper understanding of the impact of modeling on IQ:  Impact on Accuracy  Interaction between modeling and familiarity • Contributor-oriented IQ management  Impact on decision making (in-progress: study with data consumers)

• Beyond citizen science  Corporate settings  Health IS, telemedicine


21

Future work (cont’d) • Extending instance-based approach to conceptual modeling  How to combine it with traditional modeling?  Do we need “instance-based” grammars? • See: Lukyanenko and Parsons 2013a

 How to better manage attribute-based data collection  Implications for user interfaces • See: Lukyanenko and Parsons 2013b; Lukyanenko and Parsons 2013c;


22

References • Lukyanenko, R., Parsons, J., and Wiersma, Y. (2014). The Impact of •

•

•

Conceptual Modeling on Dataset Completeness: A Field Experiment. International Conference on Information Systems, 19 pp. Lukyanenko, R. and Parsons J. (2013a). Is Traditional Conceptual Modeling Becoming Obsolete? In W. Ng, V.C. Storey, and J. Trujillo (Eds.), International Conference on Conceptual Modeling (ER 2013), Lecture Notes on Computer Science Vol. 8217, Springer, Heidelberg. pp. 61-73. Lukyanenko, R. and Parsons, J. (2013b). Reconciling theories with design choices in design science research. In J. vom Brocke et al. (Eds.), International Conference on Design Science Research in Information Systems and Technologies (DESRIST 2013), Lecture Notes on Computer Science Vol. 7939, Springer Berlin / Heidelberg. pp. 165-180. Lukyanenko, R. and Parsons J. (2013c). Implementing Lightweight Conceptual Modeling for User-Generated Content: Prototype. Workshop on Information Technologies and Systems (WITS 2013). [prototype paper].


23

THANK YOU! FACULTY OF BUSINESS ADMINISTRATION

WWW.BUSINESS.MUN.CA