Getting Started in Teaching and Researching Computer Science in the Elementary Classroom Diana Franklin†, Charlotte Hill†, Hilary Dwyer‡, Ashley Iveland‡, Alexandria Killian‡, Danielle Harlow‡ †Computer Science Department UC Santa Barbara
‡Gevirtz Graduate School of Education UC Santa Barbara
{franklin, charlottehill}@cs.ucsb.edu
{hdwyer, aockey, akillian, dharlow}@education.ucsb.edu science to their curriculum [1], they are unlikely to have had formal training in how to teach computer science. In this paper, we provide practical tips for people who are interested in teaching computer science and who are creating material for computer science curricula.
ABSTRACT The recent growth of interest in computer science has created a movement to more readily introduce computer science in K-12 classrooms. However, little research exists on how to successfully bring computer science to lower grade levels. In this paper, we present advice for researchers and curriculum developers who are getting started working with computer science in elementary schools. Specifically, we focus on practical tips for studies of this nature, developed from our experiences piloting a computational thinking curriculum with 4th-6th grade students. We address issues arising in elementary school classrooms such as recruiting and interfacing with teachers and schools, classroom management strategies, student computer literacy and developmental stages, and curriculum life cycles.
Developing computer science curricula requires knowledge of computer science, understanding of how young children learn, and awareness of how elementary school classrooms function. This requires an interdisciplinary team of experts working together. Computer scientists are needed to identify important computer science content but are unlikely to have experience working with schools, particularly with lower grade levels. Likewise, teachers have extensive knowledge about children, but may be unfamiliar with the content as well as the tools (computers and programming languages). Education researchers bring relevant theory, research methodology, and knowledge about how students learn, but they may not have worked specifically computer science classrooms. To this end, we combined our interdisciplinary experience with research on this age group to compile a set of tips for those seeking to create successful computer science experiences for children.
Categories and Subject Descriptors K.3.2 [Computer and Education]: Computer Science Education
General Terms Design; Experimentation; Human Factors
Keywords K-12; Outreach; Computer Science Education; Curriculum
We narrow our list and discussion to only those ideas most important for getting started. For those familiar with university level computer science teaching, we present nuances of how elementary schools students differ from college students. For those familiar with children and classrooms but not computer science instruction, we present factors that distinguish computerbased instruction from traditional classroom instruction.
1. INTRODUCTION Recent efforts to expand the computer science field have led to a variety of outreach programs and activities, many targeted at underrepresented groups such as girls and students of color (e.g. GirlsWhoCode, BlackGirlsCode, etc.). Since we know that eighth graders’ interest in pursuing a career in science and engineering is a strong predictor of whether or not they will later pursue a science career [18], it follows that introducing students to computer science in elementary school can increase the likelihood that they will pursue careers that require computer science in the future. But, bringing computer science to younger grades comes with challenges. Though many K-8 teachers have added computer
The rest of the paper is organized as follows. We provide background on the educational environment of elementary schools in Section 2, followed by a brief summary of related work in Section 3. Section 4 summarizes our classroom experience. Section 5 presents our tips, from preparation through classroom teaching.
2. BACKGROUND Recent trends in education have made it easier to introduce computer science curricula in elementary school. The amount of technology on school campuses has dramatically increased with the demand of computer assessment practices. With the implementation of Smarter Balanced Assessments, all U.S. students will be required to take national standardized assessments on computers [16]. Thus, many schools now have minimum requirements for Internet bandwidth, operating systems, keyboards, headphones, and screen size.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. SIGCSE’15, March 4–7, 2015, Kansas City, MO, USA. Copyright © 2015 ACM 978-1-4503-2966-8/15/03…$15.00. http://dx.doi.org/10.1145/2676723.2677288
552
faced were due to factors related more to the environment than to the concepts.
While the amount of technology present in schools has recently increased, some argue that this does not necessarily indicate meaningful student learning of computational thinking [7]. Students may spend computer lab time rehearsing multiplication facts, playing math games, or playing games to increase the speed and accuracy of their typing, activities that address skills other than computational thinking. However, an increasing national focus on computer programming has left many schools wanting to integrate programming into computer lab time.
Table 1 shows selected demographics of schools that participated in the pilot to demonstrate the different populations. We refer to these schools as A, B, C, D, E, and F, with A being the first school trial and F being the last. In schools B and E, we collected only student project snapshots. In schools A, C, D, and F, we also observed instruction and filmed students explaining their projects. Each school had a varying number of classrooms, grades participating, start dates.
Another recent educational movement, the “Hour of Code” has attempted to increase the amount of computer programming present on K-12 school campuses [1]. Since December 2012, over 20,000 teachers from kindergarten to 12th grade have implemented some type of computer programming lesson in their classrooms [14]. This is made possible, in part, by student-friendly programming interfaces such as Alice[8], Scratch[14], ScratchJr (K-3)[9], and LaPlaya[12] that have made programming more accessible to younger populations. The increased attention and accessible avenues for coding have left teachers and administrators eager for more curricula in this developing area.
Who was assigned to teach computing varied across schools. In schools A, C, and D, individual classroom teachers taught their own students. In school F, one 4th grade teacher taught computing to all four 4th grade classes. In school B, a technology teacher coordinated the program, but a team of parent volunteers with programming experience taught the curriculum. School E had a programming course taught by a dedicated technology teacher. The schools also differed in the way students accessed the curriculum. In school D, students had their computers on their desks. All other classes went to a dedicated lab for their oncomputer activities.
3. OUR EXPERIENCE We are developing, piloting, and evaluating a modular Scratchbased computational thinking curriculum for upper elementary school students. The curriculum consists of three types of activities: (1) short, pre-populated projects that students can finish within a lab period, (2) off-computer activities for the classroom rather than a lab designed to relate computational thinking concepts to every day life, and (3) an open-ended design-thinking project. We implemented the first module during the 2013-2014 academic year. On-computer activities were completed in a Scratch variant named LaPlaya [12] with a modified environment that introduced only the concepts necessary for digital storytelling: sequential execution, event-based programming, initialization, message-passing, costume changes, and scene changes. We modified the environment to hide blocks not introduced in our lessons to maximize the short, computer lab times (less than 45 minutes). Further modifications were completed after the pilot study [12].
Table 1. Participating school characteristics.
School A B C D
The curriculum was piloted in fifteen 4th – 6th grade classrooms at five schools across California with over 400 students. We performed a design-based research design study [2] in which we collected data from each school and made changes (to the curriculum, interface, or language) where deemed necessary.
% ELL 23% 2% 43% 82%
% Free/ Reduced Lunch 35% 4% 73% 92%
Classes
Grade th
2
4
th
Observed Yes
4
4-5
No
1
th
Yes
th
Yes
th
2
4
4
E
10%
37%
2
6
No
F
32%
40%
4
4th
Yes
At the conclusion of the study, we interviewed each participating teacher for suggestions on all aspects of the study, including the curriculum, tools, and interfacing with our team. The practical tips in this paper are gathered from our observations and data, as well as previous research we used to design our curriculum. The advice in this study can be roughly categorized in three ways: (1) prior research we used when we designed our curriculum that we found to be very successful, (2) advice grounded in prior research that we were unaware of and only discovered as a result of challenges that emerged during our pilot, and (3) advice about situations we were very surprised by, and for which we found no prior research providing solutions or advice.
The initial purpose of the data collection was to research students’ mental models of the concepts involved in the new curriculum. To this end, we collected three types of data. First, in all classrooms, our software collected snapshots of project data. Second, in classrooms with direct observation, Graduate Student Researchers (GSRs) took field notes as to what issues the students encountered each session and why. Third, as students completed projects, GSRs prompted the elementary school students by asking them to share their work and explain what they did. The elementary school students demonstrated and described their programs in short videos collected on iPhones. Observation and videos were used to identify potential problems, and analysis of snapshots was performed to determine the prevalence of problems (and therefore decide whether to make changes). Details of our research study, data collection, and findings can be found in Hill et al[12]. While our intended purpose was to explore elementary student learning, we found that many of the challenges students
4. PRACTICAL ADVICE We start by providing recommendations about early stages of preparation (choosing or developing an interface and language and designing a curriculum), to interfacing with schools and teachers, through delivering content in the classroom.
4.1 Language and environment When teaching a text-based programming language such as C or Java, choosing the programming environment and language are often two completely separate decisions. For visual block-based languages, a popular choice for elementary school students, the
553
environment and language are often entwined because the environment is necessary to program. Therefore, the decision is typically made together.
Technology in Computer Science (ITiCSE), Human Factors in Computing Systems (CHI), and Special Interest Group on Computer Science Education (SIGCSE).
Choosing or creating the language and environment requires special attention to the abilities of the target age group, in our case 4th – 6th grade. Most upper elementary school students have had experience with computers, but vary widely in the amount and type of computer experiences and comfort levels with the physical tasks required by computing, such as typing or clicking and dragging an object across the screen.
4.2 Teacher and Student Materials Once you have identified the language and environment, it is time to choose or create a curriculum. If you are designing a curriculum, there are several considerations to keep in mind. Create an interdisciplinary team of computer scientists, education researchers and teachers. All three components - computer science researchers, education researchers, and classroom teachers – are critical to designing a curriculum. Computer scientists bring content knowledge, education researchers bring research on how students learn, and classroom teachers bring knowledge on how the curriculum will fit into the current norms and expectations in the classrooms.
Choose a language that requires only content at or below target grade level Each school is accountable to state standards for math, literacy, science and social studies. Many states have or are moving towards standards that are aligned with the Common Core State Standards (CCSS)[5,6]. Check the standards for the grade level you are targeting to ensure using the programming language or interface do not require knowledge of math or literacy content above grade level. Math content is especially important: percentages, negative numbers, and fractions are not covered until 3rd-6th grades [6,12].
Reinforce, but do not depend on, other content knowledge Our initial projects integrated subjects taught that year (e.g. missions, California geography, physics [13]). We believed that doing so would increase the value of the curriculum and help teachers justify the time dedicated to it because computing is not a required area of elementary school instruction. Our collaborating teachers were also very positive about this idea. Unfortunately, we found two drawbacks. First, teachers choose varying starting times (early or late in school year) and paces (one 40 minute session a week to two one-hour sessions a week). It is challenging for a teacher to time the computer science content to match when they present other material during the year. Second, projects that depend on material from outside subjects make students’ success contingent on material not related to computing.
Use drop-down menus in interfaces whenever possible Because of the variability in typing skills, drop-down menus should be used in lieu of typing whenever possible. Solomon found spelling and typing difficulties to be common problems for children aged 7-11[17]. Crook found that using the mouse and joystick is much easier than typing for young children [4]. We experienced similar challenges in our classrooms. Often students struggled to type specific strings of characters correctly, leading us to implement drop-down menus as often as possible.
Our solution was to integrate other content areas in ways that reinforced the content, but did not require students to remember any of the other content. For example, instead of providing a planets project with no labels and asking students to fill in the labels and “say bubbles”, we provided the planets project with labels and had students fill in the “say bubbles” with those names. That way, students can focus on programming without being distracted or frustrated by their knowledge of outside subjects.
Avoid downloading and uploading files from a shared computer. Interfaces on browsers and modern operating systems are not well designed for labs configured using identical, parallel file structures for different students on the same computer. Because the parallel file structures differ by only one directory, and this directory may not be obvious in some views, we found this resulted in two common difficulties. First, students sometimes started from a previous student’s project rather than their own; and second, they sometimes uploaded the other student’s project rather than their own. This is because when opening a file or uploading a file, the browser starts in the most recent location for that application. In both cases, the file will be from the last person who sat on the computer. Because the directories had the same files in them, the students only noticed if he or she looked at the folder name and recognized it as not his or her own student number.
Limit or avoid use of student worksheets in computer lab Although math and science activities require students to follow written instructions from books or notebooks and write their responses, written materials do not work as well alongside computers. Depending on the lab configuration and students’ comfort with computer, it can be difficult to manage worksheets and computer activities in the same space. If the lab is far from the classroom, pencils and paper need to be transported back and forth. Also, there may not be space between computers for students to work on paper. Instead, worksheets can be filled out separately from lab and then brought to lab if they are absolutely necessary.
Choose between installation and connectivity-dependence Most platforms are either installed (requiring downloads of project material) or accessible through the browser (requiring connectivity). This choice depends on the attributes of the target computer environment. If connectivity is reliable and plentiful, then an in-browser platform with files that are easy to locate could be preferred. If connectivity is unreliable, then installing the platform and uploading and downloading the files could be preferred (just be careful about downloading as described above and installation issues as described later).
Tailor writing in workshops to student needs Students in 4th grade have learned to read, but there is great variation at their comfort and speed when writing. If desired, writing requirements can be limited by providing options to circle words or answers, drawing pictures, or filling in the blank. Other classrooms may want to use the computer time as an opportunity to practice writing and would design more open-ended responses. Show how your material links to standards in different content areas Elementary school teachers teach lessons across multiple content areas within the span of a single day. Elementary teachers may
Further Resources For more information on the research in languages and interfaces for children, consider looking at conference proceedings from Interaction Design and Children (IDC), Innovation and
554
benefit from knowing how curriculum activities from the computer science curriculum reinforce other topics. Consider aligning your materials to content standards to increase available time for computer science. For example, carefully chosen reflection questions that require students to write in specific ways can be designed to address literacy standards.
is performing the actual installation. In two of our participating schools, the people responsible for installing software for the schools were located off-site. This meant that there was a considerable delay between the request to install software and the actual installation. Any problems installing can result in additional delays. In the other schools, our staff was responsible for installing the software on every computer. Each change required updating every computer.
Provide one-page summary of activity for teacher. Providing teachers a succinct description of the activity allows teachers to quickly identify the information relevant to the lesson while walking around and working with students at their computers.
4.4 In the classroom Managing a class of 25-30 students on computers differs from managing students working on paper and pencil tasks. It can be challenging to keep everyone occupied while assisting individual students. The following recommendations are intended to increase student productivity in the classroom or computer lab. In addition, we present different lab configurations we encountered and how we worked with each.
Further Resources If you are interested in using an existing curriculum for this age group, some free choices are CSTA, ScratchJr, ScratchEd, CS Unplugged, KELP CS, and code.org.
Bookmark web locations for students As discussed earlier, typing a url is difficult for young students. On the first day, we helped students bookmark important locations for our project.
4.3 Working with schools In our project, we worked with a variety of schools, teachers, students, and lab configurations. In this section, we present the lessons we learned across our experiences in schools.
Use a mouse, not just a laptop touch pad Younger students may still be developing the manual dexterity required for clicking and dragging objects, especially when using touch pads. There are two separate problems associated with this. The first is that a student who is trying to click may instead trigger a drag of a very short distance. The second is that a student may have difficulty holding down the button while simultaneously dragging.
Finding schools partners is easier than you may think Finding school partners can be daunting for computer scientists without contacts with local schools. Education departments often have existing relationships with local schools. In addition, we found that schools and districts were eager to learn more. We received e-mails from interested teachers and administrators requesting to use our curriculum. Schools were interested in using our computer programming curriculum during their lab time to provide their students with opportunities to learn computer science. Fifteen classrooms participated in the first year, and we anticipate over fifty classrooms will participate in the second year.
We identified two solutions. The first is to provide a mouse for every student so they do not depend on the touch pad. If this is not possible, students who have trouble dragging can use two hands. The pointer finger on one hand clicks the button while the pointer finger on the other hand glides across the touch pad.
Pilot with your own researchers teaching the curriculum Computing curricula are complex, involving development environments, languages, activities, and students. During the first iteration, members of your team should teach so that you can make rapid changes to any aspect of the environment. Schedule teacher training after this initial pilot so that teachers are trained on the revised material.
Create a supportive collaborative environment Research on college-level pair programming courses found that mixed-gender pairs were the least harmonious, and females had complaints about gender bias [3]. This might be due to a combination of stereotypes and different communication patterns [10]. We observed similar patterns in the elementary classroom: when girls asked a boy for assistance, the boy sometimes took over and fixed it for the girl rather than explaining the concept to the girl. Therefore, we suggest that students be physically next to someone of the same gender or someone with whom they work well. For example, if students are seated linearly, four boys can be seated in a row, then four girls, etc. If students are seated in pairs, students could be paired with a student of the same gender.
Know the context you will be working in In preparation for teacher training, working with elementary school students, or any other implementation, it is important to figure out how and where the material will be taught. Classroom technology and personnel teaching computer activities vary across schools. In just our study, various combinations of classroom teachers, dedicated technology teachers, and parent volunteers taught computer labs. Moreover, reliable Internet and firewall restrictions varied, as well as the types of devices students used (netbooks, PCs, iPads).
Use hands-free visual aid for students to request assistance When students need help and raise their hands, they are unable to make progress. In order to provide you a clear visual cue and allow students to continue working, you can use visual aids such as red and green paper tents.
Find out the process for installing and/or using computers Using software on the web and installing software on the computers may require school level or district permissions. Procedures for granting permission differ across schools and districts.. For web access, you need to learn whether there is a firewall limiting access to websites. Students may not be allowed to access Google or YouTube, so if you have video content, you may need to house it yourself.
Students can create paper tents by placing a green paper on top of a red paper and folding them in half, making a tent with a green paper on the outside and red on the inside. The students keep the green paper on the outside to signal that they are working successfully. When they need help, they place the red on the outside and on top of the monitor, providing a visual cue to the teacher that they request assistance, but leaving their hands free to continue working.
Installing software on lab computers has two barriers. The first is obtaining permission for the software to be installed. The second
555
Use sticky notes on monitors to indicate completed students If students are working on a short, skill-building exercise, then visual cues are also useful to indicate who has completed the exercise. For example, in our curriculum, when students complete the pre-populated projects, they place a sticky note on their monitor and play in “sandbox” area. Placing a sticky note on the monitor allows teachers to easily see who has finished and moved into the “sandbox” or bonus area. Include bonus options Students work at different speeds. When students finish the assignment at different times, early finishers may be bored and ready for additional challenges. We found three categories of “bonus” options to be useful. First, the curriculum can provide extra, optional, more challenging tasks for those who finish early. Second, instructors can pose challenges for existing projects such as completing programming tasks with fewer instructions, with a wider variety of instructions, or adding “bells and whistles” to the project. Finally, students can go to a separate, open-ended activity, either within the environment (such as the sandbox in KELP CS) or a separate environment (Scratch project).
Figure 1: Computers arranged in groups One arrangement was computers arranged in table groups (Figure 1). This arrangement encouraged students to work together and share ideas. One needs to consider how to group students. Placing students with similar skill levels can help teachers provide targeted support to groups while placing students in mixed ability groupings can facilitate peer support. How well students are comfortable assisting their peers will depend on existing classroom norms. This configuration is challenging for full-class demonstrations, so consider have the “front” of the classroom be to the side of everyone, have students turn their chairs, or assemble students away from the tables for presentations.
Create student experts Teachers and researchers can use elementary school students as resources for their peers for basic computing issues. In one of our classrooms, the principal had trained small groups of 4th graders in every class to serve as computer ambassadors. These ambassadors learned basic computing and computer knowledge around the school’s new netbooks. We found that these student experts helped immensely in managing basic technology concerns such as where to find files, typing websites, and general use of their classrooms computers. As a result, teachers and researcher had time to address more conceptual issues. Moreover, the student ambassadors took pride in being classroom resources, and many of their peers were more comfortable getting help from them than other adults in the room. Create hands-free attention signal Elementary school teachers often use a signal to get students’ attention, such as having students raise one hand with a peace symbol or repeat a phrase after the teacher. When in a computer lab, use a signal that requires students to use both hands so that students remove their hands from the keyboard. Examples are placing both hands behind the neck, clapping, or wiggling their fingers
Figure 2: Computers arranged in rows The second arrangement was computers arranged in rows (Figure 2). This arrangement facilitates students’ view of a teacher in the front of the room allows students to work with their neighbors. Because students look over their screens at the board, it is especially important to have them turn off their monitors during group demonstrations.
Turn off monitors or close laptops during discussions, demonstrations, and directions. Because computer screens are dynamic and engaging, they compete for students’ attention during group activities when students are requested to listen to the teacher or a peer. Turning off the monitor removes this competition. Have students direct their chairs to the teacher or gather in the middle of the room on the floor during group time. Even if the computers are closed or screens turned off, students may have difficulty watching demonstrations if their chairs are not turned towards the teacher. Bringing the students together removes them from the computer work, allowing them to focus more on group time..
Figure 3: Computers arranged around perimeter The third arrangement was computers arranged around the perimeter of the classroom (Figure 3). This provides easy access for the instructor to see all of the monitors, but it is very hard for the students to see the instructor. It is imperative that the students turn their chairs for group demonstrations.
Analysis and tips for specific configurations During our pilot study, we encountered three classroom configurations used in the computer labs. Here, we provide discussion of the pros and cons in each of these configurations along with tips to handle any drawbacks.
556
[4] Crook, C. and Bennett, L. 2010. “Does using a computer disturb the organization of children’s writing?,” British Journal of Developmental Psychology, 25, 2, 313–321.
4.5 Providing Feedback Specific and timely feedback has been shown to increase learning and retention [8]. For this age group, the way you determine correctness and the way you express feedback is critical, especially given the variability of comfort with the material. This feedback can be given verbally by in-class instructors or programmed into the programming or project environment (if supported).
[5] Common Core State Standards Initiative. Common Core Standards for Mathematics. www.commoncorestandards.org/Math [6] Common Core State Standards Initiative. Common Core Standards for English Language Arts & Literacy in History / Social Studies, Science, and Technical Subjects. www.commoncorestandards.org/ELA-Literacy
Be flexible about your definition of “correct.” Elementary school students are still developing their reading skills, so having very precise instructions to lead students to a single right answer may not be successful. Instead, look at the overarching goals of the project and develop a broad definition of what it means to succeed at these goals.
[7] Cuban, L., Kirkpatrick, H., and Peck, C. 2001. High access and low use of technologies in high school classrooms: Explaining an apparent paradox. American Educational Research Journal, 38, 4, 813-834.
Be careful about the wording of your feedback. The majority of elementary school students are new to computing. When they struggle at learning, they may believe this is because they are not good at computing, rather than attribute their difficulties to learning something new. Critical language can reinforce possible preconceived notions that they are not meant for computing. For example, in our feedback we adopted the wording of “You are not finished yet” rather than “The answer is incorrect.”
[8] Dann, W., Cooper, S., and Pausch, R.. 2000. Making the connection: programming with animated small world. In Proceedings of Innovation and Technology in Computer Science Education (Helsinki, Finland, July 11-13, 2000), . ITiCSE ’00. ACM: New York, NY. [9] Flannery, L., Silverman, S., Kazakoff, E., Umaschi Bers, M., Bontá, P., and Resnick, M. 2013. Designing ScratchJr: support for early childhood learning through computer programming. In Proceedings of the International Conference on Interaction Design and Children (New York, NY, June 24-27, 2013). IDC '13. ACM: New York, NY.
5. CONCLUSIONS Working in elementary schools is incredibly rewarding: elementary school teachers and students are enthusiastic, and there is the opportunity to make significant contributions through curriculum development or implementation. Working in elementary school classrooms, however, differs from outreach activities – there is more time involved, elementary school teachers may be less comfortable with the material than dedicated outreach providers, and attendance is compulsory rather than voluntary Therefore, it is critical that everyone involved be careful about the languages, development environment, curriculum, and classroom atmosphere. We found that establishing a collaborative team consisting of computer scientists, teachers, and educational researchers facilitated success.
[10] Franklin, D. 2013. Practical Guide to Gender Diversity for CS Faculty, Morgan-Claypool. [11] Hattie, J. & Timperley, H. 2007. The Power of Feedback. Review of Educational Research, 77, 1, 81-112. [12] Hill, C., Dwyer, H., Martinez, T., Harlow, D., Franklin, D. 2015. Floors and Flexibility: Designing a programming environment for 4th-6th grade classrooms. Proceedings of the SIGCSE Technical Symposium. [13] Next Generation Science Standards. http://www.nextgenscience.org/next-generation-sciencestandards [14] Resnick, M., Maloney, J., Monroy- Hernández, A., Rusk, N., Eastmond, E., Brennan, K., and Kafai, Y. 2009. Scratch: Programming for all. Communications of the ACM, 52, 11, 60-67.
6. ACKNOWLEDGMENTS This work is supported by the National Science Foundation CE21 Award CNS-1240985.
7. REFERENCES
[15] Richtel, M. (2014, May 10). Reading, writing, arithmetic, and lately, coding. The New York Times.
[1] Alvarado, C. 2014. CS Ed Week 2013: The hour of code. ACM SIGCSE Bulletin, 46(1), 2-4.
[16] Smarter Balanced Assessment Consortium. 2014. The smarter balanced technology strategy. Olympia, WA.
[2] Barab, S., & Squire, K., (2004) Design-based research: Putting a stake in the ground, The Journal of the Learning Sciences, 13(1) 1-14.
[17] P. Solomon. 1993 Children’s information retrieval behavior: A case analysis of an opac. Journal of the American Society for Information Science, 44, 245–64,.
[3] Choi, K. 2013. Evaluating Gender Significance within a Pair Programming Context, Hawaii International Conference on System Sciences.
[18] Tai, R., Liu, C., Maltese, A., & Fan, X. 2006. Planning early for careers in science, Science, 312, 5777, 1143–1144.
557
