USING TWITTER 1 Using Twitter to Informally Assess ...

Running head: USING TWITTER

1

Using Twitter to Informally Assess Graduate Students A Conference Paper shared at the 2016 AERA Annual Conference Christina Yuknis, PhD Gallaudet University

Abstract This study examined the use of Twitter in a graduate-level course to informally assess student progress toward course objectives. Social media has been gaining traction in higher education, and this study examines the effectiveness of its application. Findings indicate that students initially expressed reticence but quickly engaged more in using social media for class purposes. Twitter was useful in assessing specific types of course objectives overall, but that it is more effective on a week-by-week basis for instructional planning than as a formal or summative type of assessment.

USING TWITTER

2 Introduction

Politicians are increasingly using social media to engage with their constituents. It has been said that President Obama skillfully used social media, particularly Twitter, to engage younger voters and win both of his presidential elections (Rutledge, 2013; Thomas, 2011). As such, it is important when teaching about policy to ensure that students know how to skillfully navigate Twitter in order to advocate and engage with their elected representatives and increase their civic participation. Twitter is a social networking website and microblogging platform that connects people and organizations to each other through the use of short messages, called tweets, which are a maximum of 140 characters. Connections are made through following other Twitter users, tagging users to bring others into a conversation, and using hashtags to identify conversations. This project presents the results of implementing social media in two graduate courses on education policy as a way of informally assessing content knowledge. Faculty use of social media Faculty use of social media has been of interest to researchers in trying to understand the types and ways social media is used (Chen & Bryer, 2012; Lewis & Rush, 2013; Moran, Seaman, & Tinti-Kane, 2011), differentiating professional and personal uses (Veletsianos, 2012; Veletsianos & Kimmons, 2013), concerns about using social media (Chen & Bryer; Veletsianos & Kimmons), and how they incorporate social media into their courses (Chen & Bryer; Moran et al.). The most common social media site used by faculty is Facebook (Chen & Bryer, Moran et al.), although other sites are used as well including LinkedIn, YouTube, Blackboard, Blogger, Elluminate, SecondLife, and Twitter.

USING TWITTER Faculty members have expressed concerns around maintaining privacy, establishing boundaries, considering intersections between online and offline professional identities, and considering the ethics of using social media (Chen & Bryer; Veletsianos; Veletsianos & Kimmons), and, in the case of the participants in Veletsianos and Kimmons’ study, preferred to maintain social media for personal use. Despite these hesitations, some faculty members do find usefulness in building online communities via Twitter (Lewis & Rush, 2013; Velestianos, 2012). Veletsianos explored the ways in which scholars use social networking sites professionally, specifically Twitter. Fortyfive scholars from various disciplines who actively tweet were included in the sample. After analyzing the most recent 100 tweets of these scholars, seven themes emerged: sharing information and resources, expanding learning beyond the classroom, requesting assistance, sharing life activities, managing digital identities, connecting and networking, and highlighting social presence on other networks. Findings indicated that scholars use Twitter in complex and multi-faceted ways, and there is a blurring of professional and personal lives. Finally, Moran et al. (2011) found that social media use is not limited to professional networking. Approximately 68% of faculty they surveyed reported using social media in their classes. Online videos, podcasts, and blogs were the most commonly used social media used in classes. Facebook and Twitter were rarely used as part of a course. Student perceptions Gikas and Grant (2013) examined the perspectives of undergraduate students on their experiences using mobile devices for educational purposes. They found that students discussed the advantages of quick information access, constant connectivity, multiple learning paths, and situated learning as advantages to using mobile devices. Specifically, students shared that

3

USING TWITTER

4

posting comments to Twitter was easier than logging back into the course discussion board, as they were already using Twitter. Integration of the course content with social media was more natural. Students also shared that Twitter provided opportunities to interact with their professors and other researchers informally, and they discussed three disadvantages: anti-technology instructors, device challenges, and device-as-distraction. On the other hand, when Ebner, Lienhardt, Rohs, and Meyer (2010) studied social media use in graduate students, they found students initially resistant to posting private information, and thus changed their directives to focus on course-only work. Despite this reluctance, they found that the students increased their level of non-course communication throughout the study. Instructional use Perhaps this Ebner et al. (2010) finding is related to how the instructor initially establishes the use of social media in the classroom. For example, Junco, Elavsky, and Heiberger (2013) found that when faculty members engaged with students on Twitter, student engagement with the course also increased. This suggests that the ways in which faculty implement social media have an impact on the outcomes. Additionally, when Twitter was a required part of the course, course engagement online also increased (Ebner et al., 2010; Junco, Heiberger, & Loken, 2011; Junco et al., 2013). When Twitter is a required part of the course, then the frequency of tweets by students is a data point of interest. Ebner et al. (2010) found that the number of entries and discussion about course content increased over time, but both Junco et al. (2011 & 2013) found relatively flat engagement until required assignments were due. At that point, the number of tweets increased dramatically. None of these studies set a limit on the minimum number of tweets students were expected to send. However, Kassens-Noors (2012) did establish a baseline for the number of

USING TWITTER

5

tweets expected, and she found that students did not meet the expectation. No rationale was offered for why this occurred. Influence on engagement and learning outcomes The influence of Twitter use on student engagement and learning outcomes is inconsistent. In 2010, Ebner et al. found that using Twitter resulted in higher student engagement in course content and in more informal learning. Later, Junco et al. (2013) had mixed results in their study of social media use. They found that students required to use Twitter had higher engagement and collaboration than students required to use Ning, which is a platform that allows users to build networks. Additionally, the students using Twitter had significantly higher grades at the end of the semester. Incidentally, these results align with a previous study by Junco et al. (2011). However, the second study indicated that, when given a choice, students opting to use Twitter did not have significant differences in engagement or grades. Kassens-Noor (2012) had three main findings related to using Twitter in one course. Tweeting fostered prolonged interactive engagement and encouraged continual communication with team members. In this way, it worked as an active learning tool. However, due to the constant barrage of messages, it did not support self-reflection as much as personal journals (the control group) did. Finally, since the group using Twitter had ongoing interactive discussions, they were more likely to use responses supplied by others in the group than the group using journals. That group had more time to reflect on their own thinking and writing and less interactive dialogue, thus their responses were more self-generated. Faculty engagement with social media for personal and professional purposes is increasing (Ebner et al., 2010). As faculty become more comfortable with the possibilities opened by social media, particularly Twitter, they are beginning to use it in their instruction. In

USING TWITTER

6

all of these studies, it appears that Twitter has mixed results regarding usefulness for promoting student engagement with course content and supporting higher achievement (Ebner et al.; Junco et al., 2011; 2013). Success may stem from how the instructor structures the use of Twitter in the course, but since the literature is young, the results are inconclusive at this time. However, the literature does highlight variations in how Twitter is applied in academic contexts, from a group tool to an optional tool to an actual assignment. Significance This study aims to show that the inclusion of social media into education policy courses has a positive influence on course outcomes through the informal assessment of content knowledge. Educators bemoan that they are left out of policy decisions impacting their classrooms (Rizzolo & Behrstock-Sherratt, 2013; StudentsFirst, 2012), but they often do not know how to engage policymakers in meaningful dialogue. Taking advantage of the availability of social media to developing skills builds the capacity for educators to have a seat at the policy table. Additionally, social media is here to stay, and students are often savvy users of it. Social media promotes collaboration among individuals, which is a skill desired within the workplace (Robles, 2012). Finally, and perhaps the most relevant, social media provides the instructor with real-time and frequent feedback on student performance which can be used to adjust instruction. How this translates into outcomes and achievement will be beneficial to instructors seeking to incorporate current and relevant technology into their courses. This study aims to promote the inclusion of Twitter into education policy courses, because social media has a positive influence not only on student learning but also students’ course engagement.

USING TWITTER

7 Research Design

Twitter was implemented in two sections of an education policy course taught in consecutive fall semesters. The sample size is small, so descriptive statistics are used to present the quantitative data. Grounded theory methods of coding and analysis are used for the qualitative data (from the social media postings and course evaluations). Research questions The research questions guiding this study were: 1. How frequently do students engage others in education policy discussions outside of their classmates via Twitter? 2. How can tweets be used to informally assess progress on course learning outcomes? Data collection The course is a doctoral-level course in education policy that required a minimum of 10 tweets per week related to the course readings and topics. The use of appropriate tagging (particularly of political figures) and hashtags was taught and encouraged. Additionally, I used Twitter to engage with students and model the types of engagement that I wanted to see from them. The course objectives were: 1. Identify the main tenets of major federal education policies 2. Identify economic, political, and institutional forces that shape school policy and reform efforts 3. Explain how various political ideologies frame and influence the policy process 4. Analyze the consequences of implementing education policies 5. Analyze the implication of policies on equity issues 6. Develop methods that promote change in policies supporting social justice ideals

USING TWITTER

8

A total of 18 unique users participated in the conversation using the course hashtag, and their tweets will form the basis of the analysis. Analysis Students were assigned a specific hashtag to indicate the tweet is part of the course and not part of their own personal tweets (if they are using an existing account). For the analysis, tweets using the course hashtag will be analyzed. A total of 860 tweets were downloaded from the two semesters. The instructor’s tweets were removed, which left 653 tweets that fit this requirement and were used for analysis. To answer the first question, a simple frequency count of how often political figures are tagged either directly or via hashtag on Twitter. Political figures included President Barack Obama, Secretary of Education Arne Duncan, senators, representatives, and school superintendents. Others included were: Diane Ravitch and Henry Giroux, authors whose work we read; the US Department of Education; and #EdChat, #PhDChat, and #Education hashtags. This is by no means an exhaustive list, but it is representative of whom the students attempted to engage in education policy discussions. To answer the second question, tweets were coded by course learning outcomes. An analysis of the tweets in each group was conducted to determine the frequency and depth to which each outcome was addressed by the group of tweets. Results A total of 82 hashtags beyond the required course hashtag were used over the course of the two class sections. The hashtag used the most frequently was #WaitingForSuperman with four uses. Most hashtags were one-offs or were related to a specific purpose, and most were used by students who were already comfortable with Twitter prior to taking this course. Fifteen of the hashtags were used for humorous or sarcastic purposes. Twenty hashtags were related to

USING TWITTER

9

broad education or deaf education topics and ranged from #school to #k12 to #deafed(ucation). There were nine education policy-specific hashtags that included #edureform and #ccss. Seven hashtags had a focus on social justice and included #AERABrownLecture. Eleven were specifically related to course readings or viewings. Finally, 20 hashtags were unrelated to the course content (#sanfran), were specific to current events (#greece), or were specific to websites (#EWopinion). Tagging was used slightly less than hashtags. A total of 68 tags for Twitter users beyond class members were used. @HenryGiroux received the greatest number of tags with 9. The overwhelming majority of tags were to players in the education and education policy fields, with a few that were responses to tweets the students received. Other tagged entities include: @EducationWeek, @ValerieStrauss, @BillGates, @TeachforAmerica, and @GallaudetU. Overall, just one to two percent of tweets for the education policy course included some type of connection to others outside of our class. Students who were already using Twitter and comfortable with that style of communication used the hashtag and tagging features more frequently than students who signed up just for this course. To answer the second question, tweets were categorized by which course outcome they addressed then a qualitative analysis of each category was completed to determine the extent to which students’ tweets demonstrated progress toward course outcomes. Most tweets for the course were not directly related to a course outcome, although they were mostly relevant to education policy. Those tweets were removed from the analysis. Of the remaining tweets, none of them addressed the first course outcome, which was to “identify the main tenets of major federal education policies.” This outcome would not be appropriately addressed through Twitter since it is of a comprehension and recall level, which

USING TWITTER

10

does not lend itself well to Twitter. Twitter discourse is better suited to sharing ideas and having conversations; merely stating the principles of federal policies does not typically generate discussion and thus was not done by the students. However, the remaining outcomes each have tweets that address them. The second course outcome, to “identify economic, political, and institutional forces that shape school policy and reform efforts”, had the highest number of tweets associated with it. Since forces that shape policy are broad and encompass a number of policy actors and ideologies, it follows that students would be able to address this outcome frequently via tweets. Issues and forces covered include, but are not limited to, the role of money in influencing education policy and practice, media portrayal of education (achievement, teachers, minority students), corporate involvement in education, power distribution between social classes. Students asked questions, shared resources, and engaged in dialogue around issues raised in this outcome. The third outcome, to “explain how various political ideologies frame and influence the policy process,” was also largely demonstrated through student engagement on Twitter. Again, understanding the ideology of various policy actors and how that influences defining problems, setting agendas, policy creation, and policy implementation were discussed throughout the tweets. “Analyze the consequences of implementing education policies” is the fourth course outcome, and although student analysis was not lengthy, the quality and depth of analysis can be seen in the 140 character tweets. Students commented on the impact of waivers, creation and selling of textbooks, high-stakes testing, importance of families, use of data for decision-making, teacher credentialing, and quality of research in education. The last two course outcomes, “analyze the implication of policies on equity issues” and “develop methods that promote change in policies supporting social justice ideals,” both had

USING TWITTER

11

many tweets that focused on equity and social justice as they apply to deaf education, although other marginalized groups were included as well. Since the focus of the doctoral program was on critical studies in deaf education, those connections were to be expected. Student tweets questioned wording of articles, common education vernacular, and decisions by policy actors to show their analysis of implications. They questioned common practices, suggested new ways of framing ideas, and offered policy alternatives as ways of promoting change in social justice ideals. These two outcomes were less frequently demonstrated through Twitter use, but they often started conversations that carried over into class time, where they could expand on a line of thinking or questioning. Limitations As with any study, this one is not without its limitations. The primary limitation is that this is a small-scale pilot study of an experimental course, and as such, the results may not be generalizable. Additionally, the analysis shows emerging trends and may be different when applied to other courses or content. However, the underlying concepts of using Twitter to assess student learning do show some promise and warrant further exploration. Discussion Findings indicate that early in the course, students expressed reticence about using social media and engaged it minimally. However, as the course progressed, students engaged in longer conversations, shared resources and articles, and increased their use of tagging. Hashtag use remained relatively low, except by students experienced with Twitter prior to taking this course. Additionally, conversations from social media spilled into the class discussions and vice versa. From an instructor standpoint, it was easy to see which concepts grasped students’ attention, which were difficult to understand, and where attention was needed in class. Social

USING TWITTER

12

media proved to be an easy way to informally assess student learning and to provide immediate feedback. Students also responded positively to the incorporation of social media. The results tentatively suggest that Twitter is a useful tool for informally assessing student progress on course objectives, particularly as the assessment is used to inform instructional planning. The results are tentative as the number of participants is small, and one type of course was used for the study. However, the findings are such that Twitter appears to be a useful tool and replication with additional students and courses may be beneficial. Future research is needed to further evaluate the usefulness of Twitter for informally evaluating a broad range of courses and content.

USING TWITTER

13 References

Chen, B. & Bryer, T. (2012). Investigating instructional strategies for using social media in formal and informal learning. The International Review of Research in Open and Distance Learning, 3(1). Retrieved from: http://www.irrodl.org/index.php/irrodl/article/view/1027/2115 Ebner, M., Lienhardt, C., Rohs, M., & Meyer, I. (2010). Microblogs in higher education – a chance to facilitate informal and process-oriented learning? Computers & Education, 55, 91-100. doi:10.1016/j.compedu.2009.12.006 Gikas, J. & Grant, M. M. (2013). Mobile computing devices in higher education: Student perspectives on learning with cellphones, smartphones & social media. The Internet and Higher Education, 19, 18-26. doi: 10.1016/j.iheduc.2013.06.002 Junco, R., Elavsky, C.M., & Heiberger, G. (2013). Putting twitter to the test: Assessing outcomes for student collaboration, engagement, and success. British Journal of Educational Technology, 44(2), 273-287. doi: 10.1111/j.1467-8535.2012.01284x Junco, R., Heiberger, G., & Loken, E. (2011). The effect of Twitter on college student engagement and grades. Journal of Computer Assisted Learning, 27, 119-132. doi: 10.1111/j.1365-2729.2010.00387.x Kassens-Noor, E. (2012) Twitter as a teaching practice to enhance active and informal learning in higher education: the case of sustainable tweets. Active Learning in Higher Education, 13(1), 9-21. doi: 10.1177/1469787411429190 Lewis, B. & Rush, D. (2013). Experience of developing Twitter-based communities of practice in higher education. Research in Learning Technology, 21, 18598. doi: dx.doi.org/10.3402/rlt.v21i0.18598

USING TWITTER

14

Moran, M., Seaman, J., & Tinti-Kane, H. (2011). Teaching, learning, and sharing: How today’s higher education faculty use social media. Retrieved from: http://files.eric.ed.gov/fulltext/ED535130.pdf Rizzolo, A. & Behrstock-Sherratt, E. (2013, August 16). How to boost teacher voice in policy. Ed Week. Retrieved from http://www.edweek.org/ew/articles/2013/08/16/01rizzolo.h33.html Robles, M. M. (2012). Executive perceptions of the top 10 soft skills needed in today’s workplace. Business and Professional Communication Quarterly, 75(4), 453-465. doi: 10.1177/1080569912460400 Rutledge, P. (2013, January 25). How Obama won the social media battle in the 2012 presidential campaign [Web log]. Retrieved May 14, 2016 from: http://mprcenter.org/blog/2013/01/how-obama-won-the-social-media-battle-in-the-2012presidential-campaign/ StudentsFirst. (2012, February 13). A teacher’s 1501st decision. [Web log]. Retrieved May 14, 2016 from http://www.studentsfirst.org/blog/entry/a-teachers-1501st-decision/ Thomas, K. (2011, June 28). Barack Obama 2012 campaign to go beyond email, text. Huffington Post. Retrieved from http://www.huffingtonpost.com/2011/06/28/barack-obama-2012campaign_n_886280.html Veletsianos, G. (2012). Higher education scholars’ participation and practices on Twitter. Journal of Computer Assisted Learning, 28(4), 336-349. doi: 10.1111/j.13652729.2011.00449.x

USING TWITTER Veletsianos, G. & Kimmons, R. (2013). Scholars and faculty members’ lived experiences in online social networks. The Internet and Higher Education, (16), 43-50. doi: 10.1016/j.iheduc.2012.01.004

15