Immersive Policy Learning: An Interactive Course Experiment - naspaa

Immersive Policy Learning: An Interactive Course Experiment Tanya M. Kelley and Erik W. Johnston Arizona State University

ABSTRACT

An in-course experiment provided undergraduate public policy students with tangible experience in dealing with unfair, discriminatory, intrusive, and arbitrary policies and practices similar to those that legally exist in government. Students were subjected to in-course policies that gave preferential status and enhanced opportunities to some classmates while others were punished or handicapped. Each of the seemingly arbitrary conditions has parallels in U.S. legal, economic, and social systems. The experiment was designed to enhance student learning through an immersion in a simulated policy environment and to offer a personalized experience of dealing with unjust and arbitrary policies. Experimental and control group responses were analyzed with a grounded research approach. The authors found that the immersive environment led to deeper knowledge of the policy situation and an understanding of how to get involved in a policy area to effect change. This study illustrates potential applications for active learning, simulated empathy, and student empowerment.

KEYWORDS Policy learning, participatory experiment, student empowerment, learning environment

In public affairs education, there is often a discussion of both the method and the direction of pedagogy. Along with consideration of what skills a public affairs program helps students to develop, questions also arise as to who students are trained to be, how students are trained to think, and what students are being trained to do in their professional lives. Do public affairs programs produce informed, empowered, and empathetic public servants? From the effectiveness perspective, the argu­ment is that the field should train future admin­istra­ tors to be competent in a variety of facets of public sector delivery (Averch & Dluhy, 1992), and that such competence will fulfill students’ intrinsic, extrinsic, and altruistic motivations for entering this career path (Infeld & Adams, 2011). Additionally, there is a growing call for



JPAE 22 (1), 125–140

learning rooted in ethical concern for the impact of policy choices on peoples’ lives (Lang­ford, 2004) and empathy for the recipients of public services (Lipsky, 2010). Findings from students exposed to service-learning supplements—in which students apply course concepts to realworld situations—show that such experiences not only improve students’ academic learning, but also contribute to their personal and social development (Simons & Cleary, 2006). Jaeger (2003) argues that an emphasis on emotional intelligence in education improves not only academic performance but subsequent job performance as well. These calls for social-emotional learning (Zins, Weissberg, Wang, & Walberg, 2006) are the successors of Dewey’s (1916/2008) emphasis on individual learning as social and interactive Journal of Public Affairs Education 125

T. M. Kelley & E. W. Johnston

processes. There is also a charge to continue the evolution of learning to match the changing external world: There is an urgency to think imaginatively about policy education that is grounded in the complexity of current social prob­ lems and also assessed by rigorous aca­ demic standards. The necessity for new approaches is heightened in light of the sobering statistics regarding economic and social trends and disparities for America’s poorest children and families. (McClintock, 2000, p. 33) Bryer (2014) makes an argument on higher edu­ ­cation that ties together much of the dis­cus­sion of the field. The core of his argument is that students need to be educated in several dimen­ sions of learning. Bryer identifies four dimen­ sions—social connection, political intelligence, social awareness, and economic self-sufficiency (p. 11). This learning can include not allowing constructive conversations about public policy to break down due to differences in political opinion. For students to not only find jobs, but to make a difference in the profession of their choice, the accumulation of knowledge is no longer sufficient. This article presents the results of a cross-course comparison of an in-course experiment con­duct­ ed in an undergraduate public management and administration course at a leading research university in the United States. The experiment’s learning objective was to provide students with a felt experience of how policies and practices commonly observed in social, economic, and public service interactions impact the daily personal and professional lives of individuals in a democratic society. For young learners whose life experiences may be limited to academic settings with relatively well-defined rules and norms of fairness, the concept of unjust, arb­ itrary, or intrusive societal structures, norms, practices, and public policies can seem abstract. Effectively illustrating how such policies can have real impacts on the lives of citizens can strengthen these students’ social-emotional un­ der­­standing of the link between public policy and lived experience. 126

Journal of Public Affairs Education

It is not that students are unable to understand the concepts, but rather that that their under­ standing may be at a macro intellectual level, which can distance students from what they are studying. A delicate line needs to be navigated when teaching students about complex policies that impact the lives of individuals and groups in society: students need guidance in developing a comprehensive understanding of unjust sit­ua­ tions without becoming resigned to power­less­ ness or losing their capacity for empathy (Wald, Krishnamurthy, Johnston, & Vinze, 2014). This experiment was undertaken in a public administration educational context specifically, for the purpose of connecting the classroom to empowered learning and action for students and to strengthen students’ empathy as future public servants. The experimental condition was assessed using a grounded research approach to an experimental and a control cohort assign­ ment on dimensions of policy learning and stu­ dent empowerment. The study results illustrate how the experimental condition affected the way students learned about public policy. AN ALTERNATIVE COURSE STRUCTURE

The purpose of the course discussed in this art­ icle is to provide undergraduate students with a comprehensive introduction to the core con­cepts of public management and administration, which are generally structured as what public ad­ ministration is and what public administration does. This article focuses on an experiment that took place over a 2-week period in an overall course design that uses a hybrid environment and staggered content to establish a culture of active student learning and participation. The one-semester in-person course met twice per week for 1 hour and 15 minutes. While the course is rooted in regular twice-weekly meet­ ings and in-person interactions, there is a strong hybrid component to the course, with online participation facilitated through a wiki. The wiki format changes the dynamic and culture of the course, as students can view each other’s content (though not each other’s grades). The students learn throughout the semester to work

Immersive Policy: An Interactive Course Experiment

together outside the mandatory expectations of regular online discussion boards. The course structure provides the foundation of interaction that the experiment requires. The structure of a typical week in this course is as follows: • A week’s module begins on Thursday. The instructor provides students with an introduction to the concept to be dis­cuss­ ed that week. No preparatory work or read­ings are required beforehand. • Between Thursday and Sunday, students learn additional written and video content via the course wiki and are required to complete a homework assignment on their personal space on the wiki. • Each module’s content on the wiki is dy­ namic and can include multimedia files (e.g., Internet video) and Web content in addition to readings or written notes on the in-course lecture. The wiki content provides enhanced understanding of the lecture material presented the prev­­ ious Thurs­day. This may mean inclusion of cutting-edge research or reports on ad­ mini­strative practice in the public sector, critiques of traditional administrative thought, or relevant news. • Homework assignments are purposely different every time. This is to encourage students to be actively engaged with the material. Examples of different home­ work activities include a do-it-yourself project, collaboratively donating to Kiva (an Internet-based microlending platform), and engaging in a participatory challenge platform. Regardless of the activity, stu­ dents are asked to analyze the homework assignment in the context of the given week’s theme. • On Tuesdays the course has an interactive session that applies the course content to current issues in the news or in practice. The format of the interactive sessions can include discussions with the professor, break-out groups, or group activities.



The logic of the course structure is that as the week progresses, students will receive an in­ creas­ing amount of information and increasing expectations of active participation with the content. The week begins with no work re­quir­ ed on the part of students as they receive an introduction to a public administration con­ cept. Throughout the weekend, the students directly interact with the material and complete individual homework assignments. It is only after this interaction that students are expected to actively participate with the course content in class. The experiment described below was conducted at the beginning of the third week of the sem­ ester, when the habits of listen-read-write-inter­ act were already established in the course culture. A POLICY LEARNING EXPERIMENT

The experiment—providing a direct experience with inequality1—took place in the fall of 2012. As mentioned above, the intent of this experiment was to see how students’ under­ standing of social, economic, and public policies and practices deepened by students being dir­ ectly affected by such policies on a small scale. The experimental condition took place over the course of 2 weeks. In Week 1, in a Thursday session, students were asked to assess news articles for their homework assignment. The experimental condition was not the assignment itself, but the way students were graded for this particular week. The following arbitrary and capricious grading conditions were presented to students at this course session: • Students sitting in the first two rows were permitted to analyze only certain news stories; of the total list of stories, only those with prime numbers were eligible. • Cheek swabs would be collected as a requirement during the next course session in order for students to receive credit for the assignment. • White students (the term white was pur­ posefully included in the instructions) were required to show identification to the teaching assistants to receive credit. Journal of Public Affairs Education 127

T. M. Kelley & E. W. Johnston

• The average score of female students in the course would be adjusted to 77% of the average male score, regardless of in­div­ idual achievement on the assign­ment. • Student athletes not attaining 80% or high­er on the assignment would receive a score of zero. • Students could receive a perfect score on the assignment by donating $25 to fund a course project later in the semester. • Two students (identified by name) had been selected by the professor to receive 100% for helping set up the rules and an extra 20% bonus for excellence. Students completed the assignment and were graded in accordance with the grading rubric presented at the Thursday session. At the next course session, white students were asked to show identification. All students were asked to have the inside of their cheeks swabbed by the TAs (who wore white lab coats) in order to get credit for the assignment. Many students thought that the entire assignment and the incourse activity were an elaborate hoax rather than a serious assignment. However, with the exception of one student, all 80 students enrolled submitted to the cheek swab, and no student refused to show ID. Bottom-Up Student Protest

There was no indication that the full impact of the grading rubric registered with the majority of the students at the time the assignment was first posted. It was not until the next course session, with the reality of the cheek swabs providing a tangible and invasive experience, that opposition became apparent. (As noted before, only one student refused the cheek swab; the remaining course members went through the experience.) The delayed emotional response was not entirely surprising, as students likely thought an explanation would be quickly provided. The absence of an immediate explanation was an intentional design of the experiment, and when directly asked at the end of the course, the 128

Journal of Public Affairs Education

instructor responded, “This course is about public administration, not public management. While we might disagree about some of the policies that are made, it is our job to administer them efficiently and effectively, and I think everyone would agree we did that well.” With the lack of a satisfying explanation for the clearly discriminatory policies, students realiz­ ed that the assignment was not an elaborate ruse, but that the grading rubric was indeed in ef­fect for the assignment. As a result, some of the students organized a union to protest the grading rubric. On the course wiki they created a page to support this organizing effort, and ultimately, approximately half of the course (44 students) joined the union. Interestingly, one of the two students who had been sel­ected for preferential grading also joined the union; the group came to identify him as a “secret weapon.” 2 The wiki page was used to express dismay and anger at the grading, to list reasons why the grad­ ing rules were unjust, and to collect ex­am­ples of other unjust treatments in American history. The page also served as a resource for how students could take action. For example, stu­ dents compiled information about proced­ures to file a formal or informal grievance with the department and collected contact information of people in the department who could help them begin such processes. The members of the union, as part of a negotiation with the instructor, did community service projects to adjust the grading structure for the assignment. Connecting the Experimental and the Real

Given the grading conditions, the follow-through with the cheek swabs and ID require­ments, and the students’ reaction to the experimental assignment, the second week of the experiment was critical. A well-devised and well-executed follow-up was necessary to pro­vide students with insight and empower­ment. If done poor­ly, conflict and disengagement could result for the rest of the semester; a rather large risk con­ sidering that the experiment was conducted in the first few weeks of the course.

Immersive Policy: An Interactive Course Experiment

TABLE 1.

Grading Conditions Matched With Current Policies/Societal Issues

Grading condition Students sitting in the first two rows can analyze only prime news stories.

Policy or societal issue Disparity of opportunity

Disparity in criminal justice system: Certain groups of individuals can be given preferential advantages or disadvantages based on their demographics or financial backgrounds.

To receive credit for the assignment, students must submit to having cheek cells collected via swab at the next course session.

Intrusive public policy

TSA pat-down and/or X-ray practices; random drug testing (https://www.aclu.org/criminallaw-reform/drug-testing); DNA-based investigations.

To receive credit for the assignment, white students must show their IDs to the teaching assistants.

Voter laws

Arizona Senate Bill 1070—Controversial sections of this legislation empowered police to stop individuals based on suspicion of illegal activity and demand to see papers demonstrating citizenship or visas.

The average score of female students in the course will be adjusted to 77% of the average male score.

Unequal pay

If a student athlete does not receive at least an 80% on the assignment, they receive a zero. Students can receive 100% for the assignment if they donate $25 that will be used to support a course project later in the semester. Two students (identified by name) receive 100% for helping set up the rules and an extra 20% bonus for excellence.

Unions

Lilly Ledbetter Fair Pay Act—This legislation is an effort to establish fair pay for workers regardless of their gender. 2011 Wisconsin Protests—Protests occurred following Governor Scott Walker’s effort to limit collective bargaining.

Unknown disadvantages of being poor

Structural and latent costs of being poor—Through fees and other mechanisms, low-income individuals and families face different financial restrictions for the same services available to middle- and upper-income families.

Inequality of influence

Influence of money on political appointments and jobs—Individuals with large financial resources often gain advantages by leveraging that money during times of decision making that lead to dis­ proportionately advantageous outcomes.

The theme of Week 2 of the experiment was politics and public administration. The week began with an explanation that the assignment from the week before had been a purposeful experiment for students to learn about unfair or non-egalitarian public policy. When the logic behind the design of the experiment was presented at the outset of Week 2, students were informed that the unfair grading rubric they had experienced was not created arbitrarily. Rather, each of the grading criteria were



Example

consistent with prevailing societal, economic, and policy conditions currently in practice in the United States. The homework prompt for Week 2 was an opportunity for stu­dents to learn about the actual policies that inspired an unfair grading condition in the ex­periment of the week before. The assignment was dedicated to studying the unfair policy and societal con­ ditions identified through the unfair grading criteria (see Table 1).

Journal of Public Affairs Education 129

T. M. Kelley & E. W. Johnston

This assignment was structured to be completed with small-group research projects detailing each issue’s objectives, impacts, and changes that had occurred. The groups were asked to address the questions listed in Table 2. An interesting aspect of this social experiment was the way that the interactions between the professor and the students continued beyond the original assignment. For instance, the union provided a unique opportunity to extend the process of active learning within the course. Rather than reverting the grades from the unfair experimental condition to a fair and traditional format, the professor decided to use the momentum of the students’ self-organizing action. He worked with the union, using current negotiation approaches, to reconcile the grading from the experimental assignment. For example, he set up an entire module about organized labor and set a period for union negotiation that emphasized the values of the different groups as well as the evaluation of multiple proposals. Ultimately, it was agreed by the professor and the students that in exchange for restoring the full amount of points to female students, the entire course would be responsible for completing 40 acts of community service, such as helping students register to vote or attending a political event (Colbert, 2012). The union was renamed the ASU Action Committee, and their wiki page became a repository for the community actions taken by the course.

TABLE 2.

Homework Questions for Study of Specific Policies or Societal Conditions

A COMPARATIVE YEAR

In 2013, the original professor was on sabbat­i­ cal, and a different instructor taught the same course. The course was taught with the same structure, in coordination with the original pro­fessor, and used modified content from the 2012 wiki. As this was the new instructor’s first teaching experience, the experimental condition was not used, though the learning modules and assignments remained the same. Students’ ex­ per­ience in the politics and public admin­i­stra­ tion modules was thus similar to the other assignments of the 2012 semester. When com­ paring the two course experiences, however, there were noticeable differences between the years. For example, no student union spon­tan­eously formed in 2013 (not surprising, given the ab­ sence of the arbitrary grading scheme), and there were no community-based student actions. The differences in the experience of the two stu­ dent cohorts led us to reflect on how the ex­ perimental condition of the first cohort im­pact­ ed their learning experience of studying policies in the United States. Did the experiment change anything about the way students understood pol­icy or help them to reflect on what they could do to get involved in issues they cared about? The authors understood that there could not be a perfect comparison of the two cohort experiences; there were different instructors and students. However, the consistency of the course and course assignments outside of the experimental condition provided a unique opportunity to examine the potential impacts of the experiment on policy learning. To better reflect on this question, a grounded research analysis was conducted on the homework assignments of the two cohorts. A GROUNDED RESEARCH APPROACH

• What is the issue? • How has it changed over time? • Who is advantaged and disadvantaged? • Is anyone trying to change it? • How can we (the people in this course) get involved?

130

Journal of Public Affairs Education

Grounded research was used, following Tracy’s (2013) and Bernard’s (2000) approach, to assess the assignments from both courses. We analyzed the content of the group assignments of the 2012 and 2013 cohorts using inductive analysis to determine the themes present in the work. Grounded research uses two levels of coding. First-level coding stays consistent with the data and focuses on the “who, what, when”

Immersive Policy: An Interactive Course Experiment

TABLE 3.

Example of Responses to How People in the Course Could Get Involved

Transportation Security Administration

2012 experimental cohort

2013 control cohort

This entire course as a whole can get involved by standing up for our rights, writing our congressman, and getting more educated on what our rights are in airport terminals.

We can’t. It’s federal law and enforced by Congress. People, including this group, feel safer with the TSA than without. Yes, it’s inconvenient, however, as Americans we are resilient people and willing to make any necessary adjustments in order to get out of the Phoenix heat.

• First, due to 49 USC 44902 states that airlines may not transport passengers who do not consent to searches. So our rights are limited already by not giving us a choice. Don’t get me wrong I think people should get searched but to an extent. When a person comes up to the nearly naked screener a passenger can decline going into one but then under the regulations set in place they are can give you a pat down. Then I ask whom or why are these TSA workers allowed to preform (sic) these tasks? They don’t even get a complete background check! • Second, we could always write to our congressman that we as the people are not happy with the regulations that are set in place and maybe give examples on how the regulations can be enhanced to better fit the people. Another example of taking a stand is through this website called the Electronic Frontier Foundation who’s (sic) role is to take action for our freedom. One of many unique features this website has is the ability to fill out a form about on the issues a person may have. On that same website, you can complain to homeland security about your concerns about your rights and they have many other links to explain your rights and how to stand up for them.

Also, in the event that we do have a complaint about the TSA, there are channels to go through to seek resolution including sending a letter of concern to TSA Headquarters explaining problematic situations or ideas for improvement. 

• So my final question is, if Homeland Security has spent over $1.1 trillion dollars on new security measures only at airports then why/how did the underwear bomber make it all the way on to the plane? Here is a funny video proving that the TSA is out of hand.

elements. Second-level coding provides a more analytical view of the data. The qualitative soft­ ware program MAXQDA (MAXQDA 2014, Version 11.0.10) was used for this analysis. Two elements were focused on in this data. First, was there any difference in the way stu­dents communicated about their assigned policies following the experiment? Learning how to absorb information about a policy and explain it to others is a core skill for under­graduate students in public administration and policy.



The majority of the assignment had questions de­ voted to different dimensions of policy learning, such as motivations for policy dev­ elopment, vested interests, and economic and societal impacts on the populace. Of particular interest was if there was any measurable dif­fer­ ence in the homework responses of the cohort that had the direct experience with inequality through the experimental condition. The second element focused on was the stu­ dents’ understanding of their own power and Journal of Public Affairs Education 131

T. M. Kelley & E. W. Johnston

pot­ en­ tial for influence at the time of the assign­ment. The question in the assignment that best captured this theme was “How can we (the peo­ ple in this course) get involved?” This question was of particular interest given the delicate balance between knowledge and em­ power­ment. The hypothesis is that the direct experience with the unfair rules would lead to a more active and empowered student population. It was not assumed that the students in the 2013 cohort did not learn about how policies can impact peo­ple in the United States, but rather that there was a marked difference in their learning exper­ience, which may not have prompted them to action in the same way as the cohort that part­icipated in the experiment the year before. In addition, the experimental cohort from 2012 had the additional experience of forming their own student union. The union formation in­di­ cated bottom-up organization, and the com­ mun­ ity service component emphasized the connection to action. Table 3 is an illustration of the difference between the two cohorts on the

empowerment question. This difference was significant enough to indicate that a thor­ough grounded research analysis would be like­ly to illustrate more differences between the cohorts. Prior to coding, the authors were interested to see if there was any difference in the length of the group assignments. No specific word counts were expected for this assignment in either year; the focus of the assignment was always on addressing the question posed to the group. Table 4 shows the differences in word counts (and multimedia components) between the two cohort years. The 2012 cohort that received the experimental grading condition had consist­ ently higher word counts, with the 2012 cohort averaging 1,716 words and the 2013 cohort averaging 805 words. This difference is statis­ti­ cally significant with a t (18) = 2.87 and p