Computer-Supported Collaborative Questioning ...

Computer-Supported Collaborative Questioning. Regimes of Online Sociality on Quora Răzvan Rughiniș

Ștefania Matei

University Politehnica of Bucharest, Faculty of Automatic Control and Computer Science Bucharest, Romania [email protected]

University of Bucharest Faculty of Sociology and Social Work Bucharest, Romania [email protected]

Alina Petra Marinescu-Nenciu

Cosima Rughiniș



Abstract— We examine online sociality in Quora, a collaborative Q&A platform, through statistical analysis of 248 answers. Quora members interact through technology and with technology, creating different regimes of sociality: a regime of visibility, in which users and technology assemble the ranked list of answers; a regime of recognition, in which members rely on answers and authors’ identity to acknowledge value; a regime of interaction, in which members comment on posted answers. We classify answers in three clusters: Sympa, Inspire and Casual. Answer types are related to the socio-technical creation of sociality regimes. Keywords- collaborative social media; regimes of online sociality, tehnologically mediated sociality, network sociality; Quora

I.

INTRODUCTION

Questioning makes the world go round. The Internet spins it too. But what happens when questioning meets the Internet? What happens when a type of knowledge overtly addressed through questions enters the circuit? Does a new form of knowledge production emerge? Is knowledge transformed in a commodity transacted in successive flows? Do new knowledge-production services shape novel roles and statuses? Quora is an instance of a Q&A online platform, an online space in which question-answering activities maintain alive a contested community of users. Quora is one of the so-called “knowledge markets” functioning in the informational jungle of the Internet. It is designed (1) to afford question-posing and answering, (2) to support collaboration through a social network, and (3) to develop a blogging platform in which members can publish their contributions. Among its terms of service, Quora requires users to register with accurate information, including their real name, thus developing specific rules of interaction in which identity disclosure is made to be relevant. By registering on the website, users have various possibilities of involvement. They may ask questions and wait for others to answer, they may answer others’ questions, they

may edit and suggesting corrections to others’ contributions, or publish a personal blog. Involvement is not limited to the production of the content, but users receive also the status of consumers. Quora allows upvoting, commenting and content sharing, thus incorporating an evaluation system which is selfsustaining. Users have access to statistics about how their activity was received by others: they may view the number of persons who read, upvoted, followed or shared their content. Also, users have a personal profile which include self-styled descriptions of themselves and metrics of their activity. The profile contains quantitative information about users’ followers, ‘followees’ (people followed by the author, counted on Quora profile under the ‘following’ metric), edits, and topics - thus incorporating the past activity in a set of indicators visible at a glance. Access to Quora content (questions, answers and blog posts) is possible both by using a search facility or by receiving a newsfeed generated according to personal preferences. In organizing content, Quora uses an algorithm of classification based on signs of public recognition which relies, among others, on the total number of upvotes received by a particular piece of information, related to the number of views. Our paper takes part in the discussion on forms of sociality characterizing online spaces [1] [2]. We argue that the sociality of online collaborative environments, such as Quora, is defined by the emergence of interactional patterns sustained by technological features of the medium. The paper is organized as follows: we start by discussing alternative views on sociality in online spaces. We then present our methods and data used to describe Quora arrangements. We go on to discuss findings by classifying observed arrangements in: regimes of visibility, recognition, and interaction. We conclude the paper by highlighting that the three regimes we identified are organized on a project-byproject basis [1].

CISTI 2014 | 222

II.

SOCIALITY OF ONLINE COLLABORATIVE SYSTEMS OF KNOWLEDGE PRODUCTION

Sociality of online environments is understood from two main perspectives: a) Community perspective These considerations belongs to scholars who focus their approach on the idea of community-based interaction, especially in relation with platforms relying on collaborative modes of knowledge production [3], [4]. According to this perspective, the sociality of online environments is produced either by patterns of collective working towards a common goal [5] or by exhibiting social markers that appoint to a common context [6]. Users define themselves as members of the group and share a sense of belonging which is maintained by digital objects [7]–[9] and identity symbols. Collaborative environments were considered as communities of practice in which situated learning is produced in interaction with others [10] [11]. b) Network perspective This approach focuses on relationships that are maintained through technological means [12]. In this case the main subject of analysis is not the interaction as such anymore, but types of relationships. A network is usually attested by social ties that define how its functioning is to be understood [13]. Whether networks constitute communities has been a hot debate, starting from the observation that networks allow only integration but not belonging [1]. A network is assumed to favor an exchange of data rather than a mutual experience or a common history [1]. Quora is a an environment whose technical mechanism is designed to foster both a network and a community [14]. Even if it relies on an infrastructure of a network - with systems of followers, following, votes and other similar features - it also integrates methods specific to an online community of practice. Users are allowed to edit the content posted by others and they are allowed to intervene and signal errors or misinformation. Therefore, they are engaged in collaborative interactions with others. Based on this hybrid mode of operation, it is important to study Quora’s regime of sociality focusing on interactional opportunities and relationships. III.

METHODS AND DATA

Our analysis is oriented towards contexts in which mental disorders such as depression, ADHD (attention deficit hyperactivity disorder), schizophrenia, bipolar disorders, OCD (obsessive–compulsive disorder) are called into question. We have chosen this particular subject because these themes allow the creation and distribution of both scientific and experiential knowledge. This analysis is not centered on mental disorder as a topic per se, but on particularities of the space of discussions created around it. We created a database in which we included as units of analysis all the answers to the questions presented in Table 1. Questions have a variable number of answers, votes, and comments. Answers were published from 2010 to 2014. We analyze multiple observable and quantifiable elements of answers. Our dataset includes a first set of variables about

response characteristics: number of votes, number of sharing operations, number of comments, wordcount, whether or not the answer includes pictures/illustrations, whether or not the answer includes links to external documents, whether or not the answer is copy/paste from another source or just a redirect to another page. Secondly, we recorded answerers’ characteristics. In the first instance, we included anonymity criteria as a differentiator (we observed whether or not identity markers are displayed). In cases in which the author wrote under signature, we also considered elements describing the gender of the author, the number of followers, the number of following, the total number of answers and the total number of edits – from authors’ profiles. Thirdly, we recorded the date of answer publication and we computed a variable measuring the popularity of the author as a difference between the number of author followers and ‘followees’ (people followed by the author). TABLE 1. OVERVIEW OF QUESTION-ANSWERING ACTIVITY FOR THE EXAMINED QUESTIONS No. of answers

Average number of votes

Total number of comments to answers

Date of first published answer [by/mm/did]

…depression?

87

17

67

11/02/01

… ADHD?

52

20

82

11/04/26

… schizophrenia?

43

24

83

11/02/13

… bipolar disorder?

28

15

50

11/02/15

… OCD?

21

16

18

10/08/02

… to be depressed?

17

12

10

14/01/01

What does it feel like [ to have ] …

Total sample

248

Our exploratory analysis relies on multilinear regression and cluster analysis. Regression models are estimated for the entire sample, and also for sub-categories of answers: we analyzed in more depth answers coming from authors that have at least one response in addition to the one analyzed (Table 3, Table 5) and answers offered by men or women (Table 4). The assumptions of linear regression analysis (linear association, independence of observations, the normal distribution of residual errors, and the absence of multicollinearity) were met in our data. Besides regression models, we have used cluster analysis to classify answers and identify patterns of engagement with the platform. One of the limitations of the study is that, given the absence of information about anonymous authors (such as number of followers, following, or edits - which were only available on personal profiles), we had to analyze the subsample of nonanonymous authors in models in which we included popularity as a variable. Most of the patterns we describe are valid for those who chose to display their name. Some aspects related with the status of anonymous contributors remain unexplored, but considering that the proportion of such authors is low (as presented in Figure 1 and Figure 2 we consider the analysis to be relevant in understanding Quora sociality. IV.

FINDINGS: REGIMES OF ONLINE SOCIALITY

The findings allow us to argue that Quora works in different regimes, developed through interaction between users

CISTI 2014 | 223

and technology. These regimes are arrangements derived from the structure of opportunities provided by Quora design and the aggregated actions performed by users in this environment. A. Regime of visibility We start our analysis by exploring what matters for an answer to be on the top of the Quora Page, thus attempting a reverse- engineering, within the limits of our evidence 1 , of Quora’s ranking algorithm. Our expectations were that there are four types of influences on answer rank: temporal advantages, social recognition, particularities of the message and identity markers. Bivariate correlations and a linear regression model is presented in Table 2, including the following variables: days past since the answer was posted, number of upvotes2, answer wordcount, and user anonymity). Results indicate that rank is on average better for early answers, longer answers, those receiving a higher number of votes and those whose author is identified. These predictors remain statistically significant when controlling for the others, in the multivariate model. The number of votes is the strongest predictor – but recency, wordcount, and anonymity are still significant, even when controlling for upvotes. TABLE 2. ANSWER VISIBILITY ON QUORA PAGE Position on Quora Page (1=first answer, 2=second answer on page etc.) Bivariate Linear regress. Pearson correl. Beta coeff. (N=166) Days past since the answer -.203** -.203** was posted (169) Number of votes -.347** -.340** (257) Wordcount -.232** -.211** (257) Author is anonymous .197** .217** (No=0, Yes=1) (256) **Coefficient is significant at the 0.01 level (2-tailed). * Coefficient is significant at the 0.05 level (2-tailed).

This observation highlights a regime of visibility assembled from (a) the ranking algorithm put to work in order to classify the content (which is a technological accomplishment) and (b) user interaction with the medium. Through their actions (upvoting, posting as anonymous or not, posting longer or shorter answers) and through the mediation of Quora’s ranking algorithm, participants contribute to creating a flexible page structure for each question. The page structure does not reflect a linear accumulation of answers, but the constitution of a collaborative body of knowledge, composed of heterogeneous answers that are assembled in a list according to authors’ actions and readers’ reactions. B. Regime of recognition The presence of anonymity or, conversely, the presence of authors’ names is not only relevant in establishing the position of an answer on the page but it is also associated with the 1

An important limitation is that Quora’s ranking algorithm takes into account the number of views of an answer, which we did not have access to. 2 The number of comments is strongly correlated with the number of upvotes, thus we did not include it in the model.

number of comments an intervention receives. Figure 1 shows that the most commented answers are those belonging to authors who are not anonymous in their posting. As presented in the distribution, the cases in which answers accumulate more than five comments are rare. The questions we have analyzed, at the intersection of ‘What does it feel like to X?” and ‘Mental Health’, are dominated by answers which did not generate any reactions and by answers with no more than two comments.

Figure 1. Number of comments according to anonymity

A similar situation is observed when analyzing the number of sharing operations per answer (Figure 2). Results illustrate a low level of engagement in circulating the content of an answer to others through Quora’s sharing facility. Less than 10% of answers have more than two sharing operations. This situation might be specific to the topic we have analyzed. Describing the experience of having a mental disorder is not a subject to qualify in communicating to friends and expecting for other to enjoy. It is possible that an analysis of other kinds of Quora topics would present other patterns of sharing.

Figure 2. Number of sharing operations according to anonymity

The regime of recognition is the most conspicuous in guiding participation in Quora. Interactions are based on a mechanism through which acknowledgment is assigned with a material and infrastructural presence. A system of collective evaluation is put to work in order to guide behavior and establish a set of values and signs of reputation. Accordingly, we considered the number of upvotes as a measure of recognition and acknowledgement. On this account, we developed a model to predict the number of upvotes an answer might accumulate by considering both the

CISTI 2014 | 224

argumentative strategies used in the message (the first two variables in Table 3) and the characteristics of the author who wrote it (the last two indicators referring to answerers’ popularity and their history of revision and activity). As the coefficients indicate, elaborated and demonstrative answers are more likely to receive a higher number of votes. In addition, we observe that the answers posted by popular authors (who have more followers than followees) and by persons with a consistent activity on the platform (measured by number of edits) are, also, more likely to be upvoted. TABLE 3. NUMBER OF UPVOTES AS A FUNCTION OF ANSWER AND AUTHOR CHARACTERISTICS Number of upvotes Bivariate Pearson correl. Wordcount

Linear regress. Beta coeff. (N=147) 211**

.291** (149) Answers includes pictures/ .374** .342** illsutration (149) Diference (Followers286** .184* Following) (149) Author edits .219** .181* (149) **Coefficient is significant at the 0.01 level (2-tailed). * Coefficient is significant at the 0.05 level (2-tailed).

Different regimes of upvoting can be observed when comparing the models that predict the number of votes by gender. Results indicate two different forms of voting behavior (Table 4). The first one is characteristic to male answers. In this case, we observe a pattern similar with the general mode of votes’ accumulation: consistent messages coming from popular and Quora competent male-users are more likely to receive a higher number of votes. A contrasting situation appears for female answers, case in which neither the content of the message, nor the previous activity is significant to predict the number of votes: only popularity matters. In interpreting this result we have to consider the number of female answers in the sample which might be a reason for lack of statistical significance. Available data indicate, though, that there might be two separate processes that function in the accumulation of votes differentiated by gender. TABLE 4. NUMBER OF UPVOTES AS A FUNCTION OF ANSWER AND AUTHORS CHARACTERISTICS BY GENDER Number of upvotes

Wordcount Answers includes pictures/ illsutration Diference (FollowersFollowing) Author edits

Male answers

Female answers

Bivariate Pearson correl.

.326** (88) .539** (88) .256* (88) .235* (88)

Linear regress. Beta coeff. (N=88) .219*

Bivariate Pearson correl. (N=61)

Linear regressBe ta coeff. (N=61)

N/S

N/S

.436**

N/S

N/S

.225*

.379** (61) N/S

.499*

.183*

**Coefficient is significant at the 0.01 level (2-tailed). * Coefficient is significant at the 0.05 level (2-tailed). TABLE 5. NUMBER OF SHARING OPERATIONS AS A FUNCTION OF ANSWER AND AUTHORS CHARACTERISTICS

Number of sharing Bivariate Pearson correl. Wordcount

Linear regress. Beta coeff. (N=148) .199**

.266** (149) Answers include pictures/ .312** .283** illsutration (149) Diference Followers287** .184* Following (149) Author edits .260** .226** (149) **Coefficient is significant at the 0.01 level (2-tailed). * Coefficient is significant at the 0.05 level (2-tailed).

Beside the number of votes, the number of sharing operations is another form of signaling recognition, but it functions under a different logic. While the ‘upvote’ is a measure that incorporates various criteria of appreciation in a single integrated dimension, the ‘sharing operation’ might be interpreted as a measure of worth of the message to be transmitted and made known to others according to a specific line of interest in the topic. Both votes and sharing operations are actions performed related to a piece of communication but, compared to votes, sharing operations have more generative power, in that they may produce independent outcomes. Table 5 contains a regression analysis explaining the number of sharing operations based on the situation we encountered in our sample. Results are similar with the ones obtained when considering the number of upvotes. In both cases the same variables are statistically significant. The number of sharing operations depends on the length and on the content (pictures/illustrations) of the message, but also on the popularity (more followers than followees) and on the activity of the author (number of edits). While the predictive capacity of messages’ features (wordcount and including pictures or illustrations) seems to decrease in the model for sharing operations compared to the model for upvotes, the influence of author’s editing experience becomes more important. The implication is that votes and sharing operations, even if both signs of recognition, are differently integrated within circuits of knowledge transfer and reckoning. While votes are more content-focused, social operations are more oriented towards credibility - in which reputation markers become relevant in the appraisal of the answer. By interpreting results as a whole, we can observe that the regime of recognition displays a “Mathew effect” [15]. The results suggest that answers coming from popular users receive a higher number of votes and are integrated in more sharing operations. Considering that votes and sharing operations are resources used to become popular, we might say that the emergence of popularity on Quora is partly based on the following principle: the popular users get more popular.

N/S

CISTI 2014 | 225

C. Regimes of interaction By considering that actions of voting answers and sharing them are modes of signaling recognition, writing comments is a more personal form of engagement which might be used to signal a detailed recognition, to provide full feedback or to start a debate on the subject, all of them not allowed by a simple vote or sharing operation. That comments are a different action is also visible in Table 6. Compared to the models for upvotes and for sharing operations (Table 3 and Table 5) - in which both message and author’s characteristics count – when predicting the number of comments, popularity and author edits seem to be irrelevant. In this case only variables referring to the content of the article remain statistically significant. This difference indicates that comments are part of a different regime than votes and sharing operations. Therefore, the regime of interaction resides in more personal forms of engagement that starts from the direct experience of reading. This is a form of engagement which makes use of the technical infrastructure - but it is also detached from its metrics. The space of opportunities provided by comments is much larger than for votes and sharing operations.

TABLE 7. OVERVIEW OF ACTIVITY PER YEAR (ANALYSIS OF ANSWERS) Average number of votes Total number of comments Total number of sharing Wordcount (average) % Answerer is anonymous % Answer includes pictures/illustrations % Answer is copy/paste from another source or just a redirect to another source

2011 22 39 4 422 27% 9% 3%

2012 19 103 29 424 28% 17% 6%

2013 23 100 47 473 35% 19% 9%

By selecting the years with more than 5 answers per topic, both the average number of votes and the length of the message remain constant, on average, in the last three years. However, an interesting evolution appears when analyzing the number of comments: answers posted on 2012 or 2013 receive a higher number of reactions than those posted on 2011. It means that older answers do not benefit from the temporal advantage of early bird publication, observation confirmed by the number of votes and sharing operations which are not higher for the old postings.

TABLE 6. NUMBER OF COMMENTS AS A FUNCTION OF ANSWER AND AUTHORS CHARACTERISTICS

Number of Comments Bivariate Pearson correl. Wordcount

Linear regress. Beta coeff. (N=148) .327**

.374** (149) Answers include pictures/ .220** .187* illsutration (149) Diference Followers230** N/S Following (149) Author edits .163* N/S (149) **Coefficient is significant at the 0.01 level (2-tailed). * Coefficient is significant at the 0.05 level (2-tailed).

V.

Figure 3. Number of commented answers by author’s gender

DIFFERENTIATED PATTERNS OF ENGAGEMENT

The temporal analysis of activity illustrates some patterns of engagement which are different from year to year. For example, the proportion of anonymous answers, of answers that include pictures/illustration and of answers with no original content tend to slowly increase with the passing of time (Table 7). The same pattern is observed when analyzing the total number of sharing operations which is higher in 2013 than in 2011 or 2012. This result may illustrate a change in patterns of behavior on Quora (possibly influenced by changes in technology, or in the community of users). It may also reflect different modes of relating to a question based on its date of publication: for older questions, that have already accumulated many meaningful answers, it could be easier to avoid redundancy by including visual illustrations. Answers from 2013 correspond also to questions posted earlier, so the high percent of impersonal messages from the last year might appear due to a process of saturation, as answers accumulate. Further empirical research could clarify these two possibilities.

When focusing on engagement as a function of gender, we observe that the most commented answers belong to male authors (Figure 3). Sharing is also more frequent for answers authored by men (Figure 4). This situation might be interpreted by taking into consideration that gender correlates with article length: men offer longer answers in our sample, on average. When controlling answer length in a multivariate regression model, gender is no longer a significant predictor either for the number of comments, or for the number of sharing operations.

Figure 4. Number of sharing operations by answerer’s gender

CISTI 2014 | 226

VI.

ANSWER TYPES

We conducted a cluster analysis (Table 8) in order to explore types of answers according to the following criteria: number of votes, wordcount, number of author’s followers, number of author’s followees and number of authors’ addressed topics. We distinguish three types of answers: Sympa, Inspire and Casual. Sympa answers receive an average number of votes, have an average length of approx. 200 words, and come from persons with a well developed network of popularity, who have on average more followees then followers. Inspire consists of longer answers (with an average number of 1114 words) that are characterized by a high level of upvotes. They belong to authors who on average have more followers than followees and with a medium number of topics in which they are interested in. Casual answers are short. They receive a low number of votes and come from authors who did not develop a consistent network of followers or followees. We can say that all these answers are developed according to specific regimes of sociality. Sympa is a relevant outcome for the regime of recognition and Inspire is a prototype for the regime of visibility. Still, these regimes are neither separated nor working independently from one another. TABLE 8. TYPES OF ANSWERS: K-MEANS CLUSTER RESULTS Number of upvotes Wordcount Author followers (F) Author following (f) Number of author topics Number of answers (N) Difference (F-f)3

Sympha 16 235 505 547 204 12 -42

Inspire 28 1114 189 87 68 17 102

Casual 8 175 44 32 50 118 12

VII. CONCLUSION Our statistical analysis of patterns of answering on Quora indicates that there are different regimes of engagement at the intersection of the technological infrastructure and users’ participation. We have identified regimes of visibility (on which a non-chronological order of presenting content is sustained), a regime of recognition (which includes a system of evaluation relying on authors’ identity metrics), and a regime of interaction (based on a more personal form of engagement though comments). Gender seems to be a relevant distinction when comparing types of user engagement: men write, on average, longer answers (excluding anonymous contributions) and, as a result, receive more comments. Regimes of online sociality are useful as analytical lines of inquiry in patterns of users’ engagement with a technological environment. The identified regimes show that the sociality created by Quora is one which function on a project-by-project logic [1]. It is a logic based on short-termed commitment to specific tasks in which relations are based on “focused, fast and over-loaded social ties” [1]. This corresponds to the structure of opportunities developed by Quora. On one hand, the succession of answers based on a problem-solving approach and the low number of comments favor contributors’ ability to 3

shift quickly between tasks. On the other hand, the regime of recognition based on a system of followers and followees favor accumulation of contacts rather than engaging in thorough interaction with others. Still, these results might be influenced by the quantitative approach we followed. Further exploration should complete the structural patterns of interaction with in-depth micro-level analysis of specific sequences. ACKNOWLEDGMENT This article has been supported by the research project “Sociological imagination and disciplinary orientation in applied social research”, with the financial support of ANCS / UEFISCDI with grant no. PN-II-RU-TE-2011-3-0143, contract 14/28.10.2011 REFERENCES [1] A. Wittel, “Towards a network sociality,” Theory, Cult. Soc., vol. 18, no. 6, pp. 51–76, 2001. [2] J. van Dijck, “Facebook as a Tool for Producing Sociality and Connectivity,” Telev. New Media, vol. 13, no. 2, pp. 160– 176, 2012. [3] E. Lehtinen, K. Hakkarainen, L. Lipponen, M. Rahikainen, and H. Mukkonen, Computer Supported Collaborative Learning: A Review. JHGI Giesbers. [4] F. Su and C. Beaumont, “Evaluating the use of a wiki for collaborative learning,” Innov. Educ. Teach. Int., vol. 47, no. 4, pp. 417–431, 2010. [5] R. Deaconescu and Ștefania Matei, “The Negotiation of Knowledge and Knowing: The Challenge of Using Wiki Technology in Computer-Supported Collaborative Learning,” in The 19th International Conference on Control Systems and Computer Science CSCS19, 2013. [6] X. Ding, T. Erickson, W. Kellogg, and D. Patterson, “Informing and performing: investigating how mediated sociality becomes visible,” Pers Ubiquit Comput, vol. 16, pp. 1095–1117, 2012. [7] R. Rughiniș and Ștefania Matei, “Digital Badges: Signposts and Claims of Achievement.,” in United States of America: 15th International Conference on Human-Computer Interaction., 2013. [8] R. Rughiniș, “Talkative Objects in Need of Interpretation. ReThinking Digital Badges in Education,” in AltCHI 2013, ACM SIGCHI Conference on Human Factors in Computing Systems, 2013, pp. 1–10. [9] R. Rughiniș, “Badge Architectures in Engineering Education. Blueprints and Challenges,” in 5th International Conference on Computer Supported Education CSEDU 2013, 2013. [10] P. Hildreth and C. Kimble, Knowledge Networks: Innovation through Communities of Practice. London: Hershey: Idea Group Inc, 2004. [11] C. Rughiniș and Ștefania Matei, “Learning Through Massively CoAuthored Biographies. Making Sense of Steve Jobs on Wikipedia through Delegated Voice,” in The 19th International Conference on Control Systems and Computer Science CSCS19, 20013. [12] E. Nicolle and D. Boyd, “Sociality through Social Network,” in in The Oxford Handbook of Internet Studies, W. Dutton, Ed. Oxford: Oxford University Pres, 2013, pp. 151–172. [13] L. Garton, C. Haythornthwaite, and B. Wellman, “Studying OnLine Social Networks,” in in Doing Internet Research, S. Jones, Ed. 1999, pp. 75–105. [14] G. Wang, K. Gill, M. Mohanlal, M. Zheng, and Zhao Ben, “Wisdom in the Social Crowd: an Analysis of Quora,” in International World Wide Web Conference, 2013. [15] R. Merton, “The Mathew Effect in Science. Science,” Science, vol. 159, pp. 56–63, 1968.

The F-f difference was not included as criterion for the Cluster classification.

CISTI 2014 | 227