Kara L. Orvis. George Mason University. Consortium Research Fellows Program. January 2003. United States Army Research Institute for the Behavioral and ...
Technical Report 1133
Web-Based Collaborative Learning: An Assessment of a Question-Generation Approach
James Belanich Robert A. Wisher U.S. Army Research Institute Kara L. Orvis George Mason University Consortium Research Fellows Program
January 2003
United States Army Research Institute for the Behavioral and Social Sciences
Approved for public release; distribution is unlimited.
20030303 131
U.S. Army Research Institute for the Behavioral and Social Sciences A Directorate of the U.S. Total Army Personnel Command
ZITA M. SIMUTiS Director Technical Review by J. Douglas Dressel, U.S. Army Research Institute Arthur Graesser, University of Memphis James J. Dees, TRADOC
NOTICES DISTRIBUTION: Primary distribution of this Technical Report has been made by ARI. Please address correspondence concerning distribution of reports to: U.S. Army Research Institute for the Behavioral and Social Sciences, Attn: TAPC-ARI-PO, 5001 Eisenhower Ave., Alexandria, VA 22333-5600. FINAL DISPOSITION: This Technical Report may be destroyed when it is no longer needed. Please do not return it to the U.S. Army Research Institute for the Behavioral and Social Sciences. NOTE: The findings in this Technical Report are not to be construed as an official Department of the Army position, unless so designated by other authorized documents.
REPORT DOCUMENTATION PAGE 1. REPORT DATE (dd-mm-yy) January 2003
2. REPORT TYPE Interim
3. DATES COVERED (from. . . to) March 2001 - August 2002 5a. CONTRACT OR GRANT NUMBER
4. TITLE AND SUBTITLE
Web-based collaborative learning: An assessment of a 5b. PROGRAM ELEMENT NUMBER
question-generation approach
633007 6. AUTHOR(S)
5c. PROJECT NUMBER
James Belanich and Robert A. Wisher (U.S. Army Research Institute);
A792
Kara L. Orvis (George Mason University)
5d. TASK NUMBER 620 5e. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) U.S. Army Research Institute for the Behavioral and Social Sciences Advanced Training Methods Research Unit 5001 Eisenhower Avenue Alexandria, VA 22333-5600
8. PERFORMING ORGANIZATION REPORT NUMBER
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS{ES) U.S. Army Research Institute for the Behavioral and Social Sciences 5001 Eisenhower Avenue
10. MONITOR ACRONYM
Atm: TAPC-ARI-II Alexandria, VA 22333-5600
ARI 11. MONITOR REPORT NUMBER Technical Report 1133
12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution is unlimited. 13. SUPPLEMENTARY NOTES
14. ABSTRACT (Maximum 200 words):
Students linked into a learning environment over the Internet may learn topics better by collaborating to create questions and answers. In research reported here, students used a learning aid for collaborative question generation called Army TEAMThink, a commercial program modified for Army use under a TRADOC Delivery Order contract. Research was done at three US Army schools to assess the quality or doctrinal correctness of questions and answers generated by students and to measure any learning benefit. Students first completed a tutorial on how to write effective multiple-choice questions. Next, students wrote questions and reviewed questions written by other students. Based on the feedback from the reviews, authors were allowed to modify their own questions. Finally, students took a test of the questions that had been developed by students using the learning aid. Army subject matter experts judged that most of the questions developed were considered acceptable and could be repurposed for use in course exams. A majority of the question feedback was constructive, indicating that the collaborative process was helpfiil. SUidents who went through the process scored higher on a test of novel questions than those who did not use Army TEAMThink. They also scored higher than students who went through the process on a different topic from the test topic, demonstrating a moderate learning effect. The general finding of this research about a collaborative question-generation approach is that instructors can accumulate quality multiple-choice ■ questions and monitor student comprehension, and students have an additional opportunity for better learning. 15. SUBJECT TERMS
training, educational technology, distributed learning, distance learning, collaboration, online instruction, web-based instruction, question generation, questions SECURITY CLASSIFICATION OF
16. REPORT Unclassified
17. ABSTRACT Unclassified
18. THIS PAGE Unclassified
19. LIMITATION OF ABSTRACT
Unlimited
20. NUMBER OF PAGES
31
21. RESPONSIBLE PERSON (Name and Telephone Number) James Belanich 703/617-2362
This Page Intentionally Left Blanl
