The effect of integrating constructivist and evidence

Nurse Education Today 42 (2016) 1–8

Contents lists available at ScienceDirect

Nurse Education Today journal homepage: www.elsevier.com/nedt

The effect of integrating constructivist and evidence-based practice on baccalaureate nursing student's cognitive load and learning performance in a research course Suh-Ing Hsieh a, Li-Ling Hsu b, Tzu-Hsin Huang c,⁎ a b c

Department of Nursing, Chang Gung University of Science and Technology, No. 236, 10th Floor-3, Fusing 1st Road, Guishan District, Taoyuan City, 33375, Taiwan, ROC Health Allied Education, National Taipei University of Nursing and Health Science, Taiwan, ROC Nursing Department, Taoyuan Chang Gung Memorial Hospital, Chang Gung Medical Foundation, No. 123, Dinghu Road, Guishan District, Taoyuan City, 33372, Taiwan, ROC

a r t i c l e

i n f o

Article history: Received 22 August 2015 Received in revised form 11 March 2016 Accepted 29 March 2016 Keywords: Constructivist Evidence-based practice Cognitive load Learning performance Baccalaureate nursing student Research course

a b s t r a c t Background: Baccalaureate nursing students perceive research as unattractive, doubt the value of nursing research, and do not appreciate the link of research with practice. No studies have examined students' cognitive load during an evidence-based practice research course versus a traditional research course. Objectives: To assess the effect of integrating constructivist theories and evidence-based practice on student cognitive load and learning performance in a research course. Design: A true experimental study. Settings: A Registered Nurse-to-Bachelor of Science in Nursing program. Participants: Six classes of second-year students. Methods: Students were randomly allocated to the control group (two classes) or the experimental group (two classes) using cluster randomization. The control group underwent “traditional research”; the experimental group experienced “integrating evidence-based practice into research.” Instruments for outcome assessment include the Cognitive Load Scale, cognitive test, team critique paper, and qualitative feedback on course satisfaction. The between-subjects effects were compared by Analysis of Covariance. Results: The experimental group had significantly higher mental load (8.74 vs. 7.27, p b .001), mental effort (11.07 vs. 10.07, p = .009), mental efficiency (0.33 vs. − 0.31, p b .001), and research knowledge (70.61 vs. 44.92, p b .001) than the control group. The experimental group had better critique paper scores in introduction (92.80%), literature review (91.70%), and assignment requirement and writing (89.40%). Some experimental learners expressed satisfaction with learning evidence-based practice (17.78%) and critiquing a research article (7.78%). Conclusions: Integrating evidence-based practice into a research course not only improved the research knowledge of baccalaureate nursing students, but also increased their mental load, mental effort, and mental efficiency. Additional studies may track learners' responses to different learning systems using the developed instrument to measure the three types of cognitive load. These findings may help educators design more effective and interesting curricula for integrating research and evidence-based practice into the studies of student nurses. © 2016 Published by Elsevier Ltd.

1. Introduction A research course embedded into the curriculum of the bachelor degrees has long been a criterion of comprehensive education as defined by the National League for Nursing (NLN, 1977). With an increasing emphasis on patient safety, cost effectiveness, and quality of patient care, baccalaureate student nurses need education in evidence⁎ Corresponding author at: No. 123, Dinghu Road, Guishan District, Taoyuan City, 33372, Taiwan, ROC. E-mail addresses: [email protected] (S.-I. Hsieh), [email protected] (L.-L. Hsu), [email protected] (T.-H. Huang).

http://dx.doi.org/10.1016/j.nedt.2016.03.025 0260-6917/© 2016 Published by Elsevier Ltd.

based practice (EBP) and research utilization (American Association of Colleges of Nursing [AACN], 2015; Christie et al., 2012; Institute of Medicine, 2001). EBP is a core knowledge competency defined by the Institute of Medicine as a part of good professional education (Hickey et al., 2010). However, surveys show that American Registered Nurses (RNs) receive little or no education in information use and retrieval; neither do they understand or value research or EBP (Pravikoff et al., 2005); these results suggest that RN education should incorporate research use, EBP, and information literacy, to cultivate the knowledge and skills of EBP and an appreciation for research into new nurses. In addition, the U.K. Nursing and Midwifery Council (2010) requires that all undergraduate programs include education in research methods and the use

2

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

of research evidence. Nurse educators must instill in baccalaureate nursing students an awareness of the research process and an appreciation of the worth of research. EBP pedagogy studies show that nurse educators use diverse approaches to integrate EBP into the nursing curriculum (Moch et al., 2010; Moch and Cronje, 2010), thereby formulating long-lasting engagement to foster the ability of students to understand the research process, think analytically, and promote information literacy skills so as to discover evidence to apply to their own practice. However, integration of EBP into research does not always render a positive learning experience. Some baccalaureate nursing students report frustration, perceive research as unattractive, doubt the value of nursing research, believe teaching strategies are futile, and do not appreciate the relationship of research and practice (Kessler and Alverson, 2014; McCurry and Martins, 2010; Spatz, 2008). Likewise, Taiwanese student nurses perceive stress both before and after a learning research course (Tsai et al., 2014). As nurse educators, we found that students in RN-to-BSN programs felt pressure and that learning research was a burden, because most did not study research in the 5-year associate degree in nursing program and they had difficulty understanding the abstract terms and concepts used in research. Compared to other nursing courses, learners in a research course need to put forth added effort to learn because of their lack of prior knowledge. Most creative, constructivist approaches to teaching research are called for so that students should graduate knowing that they can understand, critique, and use research to enhance their practice. Learning EBP in a research course may affect students' cognitive load. Cognitive load theory (CLT) is the theory that learning involves mental effort or cognitive load. Under this theory, students most efficiently learn when they minimize the short-term memory load while enhancing the memory available for transferring knowledge from short-term to long-term memory (Behmake and Atwood, 2013). CLT builds upon an established model of learner memory that includes the subsystems of sensory, working, and long-term memory (Young et al., 2014). The process of learning research and EBP needs working memory for the learner to be actively engaged in understanding and processing the material. As a result, the learner can transfer the learned information into long-term memory. Irrelevant information increases the mental load, straining working memory and interfering with long-term memory (Vogel‐Walcutt et al., 2011). In addition, when the cognitive load associated with a task exceeds the learner's work memory capacity, learning performance is impaired (Young et al., 2014). Naismith et al. (2015) use cognitive load theory for workshop design and evaluation for 59 clinical educators. Results suggest that instructional design optimized learning by managing intrinsic load, stimulating germane load, and minimizing extraneous load. No studies have assessed the cognitive load of learning research or EBP. In addition, there has been a dearth of research on the use of constructivist learning methods in nursing education (DeCoux Hampton, 2012). Therefore, the aim was to examine the effect of integrating constructivist and evidence-based practice on student cognitive load and learning performance in a research course. 2. Literature Cognitive theories are used to study students' learning processes to understand how information is received, organized, stored, and retrieved by the mind (Ertmer and Newby, 2013). The goals of constructivism is to understand the skills used in critical thinking, collaboration, and personal query, skills crucial to nursing practice and to the application of EBP. Constructivism is based on the idea that newly-acquired knowledge is built upon and within the context of previous learning (Lincoln and Guba, 2000). In the constructivist model, instructors first present basic concepts, and then gradually add more complicated concepts, while learners familiarize themselves with the more basic concepts. Within a constructivist classroom, teachers must develop

the skills students need to participate (Cooper, 2007). Lack of education means nurses lack the capacity to apply EBP; for this reason, AACN does not recommend that all of the five steps of the EBP process (ask, acquire, appraise, apply, and audit) be a standard part of the baccalaureate nursing curriculum. Rolloff (2010) demonstrates that constructivism can offer a framework for incorporating these steps into the entire nursing curriculum, from freshman year through senior year. Teachers employ a variety of research and EBP pedagogy, from incorporating EBP into the whole baccalaureate curriculum, to integrating EBP into research using a variety of teaching strategies (Bloom et al., 2013; Odell and Barta, 2011). These strategies include a collaborative clinical project; cooperative learning groups; new knowledge discussion group; a research utilization project; critiqued research and made posters; journal club; the Web Resource Appraisal Process; the use of an internet tool for accessing and appraising evidence; teaching critical appraisal in classes; the use of the PICO questions, databases, and search strategy; education in locating and utilizing synopses journals; EBP reviews and presentations; use of clinical practice guidelines in teaching and in practice; using systematic reviews; use of EBP literature in teaching; discussion of an evidence-based research paper, a presentation, and participation in a faculty-sponsored research project; and the cookie experiment (Meeker et al., 2008; Thompson, 2006). The PICO questions include (P) patient, population or process of interest, (I) intervention or best practice to be assessed, (C) comparison group or unit, and (O) outcome or effect of interest (Hastings and Fisher, 2014). Innovative approaches to teaching baccalaureate nursing research have better outcomes. For example, McCurry and Martins (2010) developed innovative strategies for teaching undergraduate nursing research and EBP; students' perceived effectiveness was greater with the use of innovative assignments than with traditional assignments (t = 6.93, p b .0001). Students also preferred active learning assignments, reading quizzes, clinical nurse researcher presentations, and collaboration with clinical course assignments. Liou et al. (2013) tested the effects on student research knowledge, classroom engagement, and eight core competencies in nursing of using innovative teaching strategies to teach research. Innovative teaching strategies were related to higher posttest outcomes of classroom engagement and nursing eight core competences, after controlling for pretest scores. To ensure that future nurse educators are prepared to teach in an EBP curriculum, it is vital that they understand how do apply CLT to help nursing students learn EBP in nursing. CLT discriminates between three types of cognitive load: intrinsic, extraneous, and germane. Intrinsic cognitive load is determined by an interaction between the nature of the information being learned and the expertise of the learner and cannot be affected directly by instructional design. Extraneous cognitive load is the load arising from suboptimal instructional methods. Extreme extraneous load can generate split-attention and/or redundancy effect (Josephsen, 2015; Leppink et al., 2014). Germane cognitive load is “the load related to processes that contribute to the construction and automation of schemas” (Paas et al., 2003, p.65). Instructional design seeks to decrease extraneous cognitive load and enhance germane cognitive load (Brunken et al., 2003). Schlairet et al. (2015) explored the impact of high-fidelity simulation on emotion and cognitive load in 40 first-semester BSN nursing students and found that simulation had a high mean cognitive load (6.27 ± 1.48) and non-significantly slight positive correlation with emotion. Fraser et al. (2012) examined the effect of simulation education on 84 medical students and found cognitive load was higher in active participants according to the level of their engagement. Chen and Wu (2015) investigated the influence of three types of video lectures on sustained attention, emotion, cognitive load, and learning performance of 37 undergraduate verbalizers and visualizers using a two factor experimental design. Lecture capture (using a digital video camera to record classroom lectures including a lecturer's voice, image, and instructional aids) and picture-in-picture types of lectures (displaying a lecturer's recorded image and voice, PowerPoint slides, subtitles, and

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

other flash animation features) resulted in higher learning performance and cognitive load than the voice-over type. Visualizers had higher cognitive load than verbalizers with the voice-over type. In summary, CLT has been applied to traditional instruction and technology-enhanced learning strategies. It can help quantify the cognitive load associated with integrating EBP into a research course.

3

3.4. Measurements Instruments included the Background Questionnaire, CLS, Research Cognitive Test, mental efficiency, team paper critique, and qualitative feedback on course satisfaction as shown in Table 1. 3.5. The Background Information

3. Methods 3.1. Research Design This study is a true experimental study. Six classes of students in the two-year Registered Nurse-to-Bachelor of Science in Nursing (RN-toBSN) program, who graduated from 5-year associate degree programs and held a license as an RN or/and Licensed Practical Nurse (LPN), were randomly allocated by class to the control group or the experimental group using SPSS to do the randomization. The control group (n = 97) underwent “traditional lecture-based research course,” whereas the experimental group (n = 90) experienced “EBP-based research course” (Table 1).

3.2. Participants and Setting This study used a convenience sampling from a university in northern Taiwan. The eligible participants were student nurses of the RN-toBSN program enrolled in a two-credit research course in the second year and recruited to join this study after signing an informed consent form. The sample size was estimated using Analysis of Covariance (ANCOVA) with α level of .05, β level of 0.80, a medium effect size of 0.25, a covariate, and a 10% attrition rate using G*Power 3.1 (Faul et al., 2007). The total estimation of sample size was 141 students. Fig. 1 shows the process from verifying sample eligibility to analyzing the data.

3.3. Intervention EBP, cognitive theory, and constructivism were used to design the educational program. EBP was integrated into the research course with PowerPoint slides, the Great Cookie Experiment, exemplar analyses, asking questions and discussion, peer instruction, group discussion, individualized team guidance, research article critique and practice. The standard research program included reading a required textbook, attending didactic lectures with PowerPoint slides, asking questions, and participating in group discussions. The educational program of the experimental group was delivered using the same teaching materials by well-trained faculty, while the control group was taught by senior faculty who had taught the course for many years. Both group had three hours of instruction every week for thirteen weeks, from December 2012 to April 2013.

Table 1 Study design and measurement. Before

Intervention

Just after

R

O1

X1

O2

R

O1

X2

O2

Measures 1. Background information 2. Cognitive load 3. Research Cognitive Test 4. Team critique paper 5. Qualitative feedback on course satisfaction

✓ ✓ ✓ ✓ ✓

Note. R, class-cluster randomization; O1, the first measure; X1, integrating evidence-based practice (EBP) into the research; X2, traditional research; O2, the second measure.

The Background Questionnaire includes general demographic questions such as age, gender, possession of nursing license, interest in nursing, experience with nursing research studies, previous research experience, and previous attendance at any EBP conferences. 3.6. The Cognitive Load Scale (CLS) The CLS was developed to evaluate students' cognitive load after participating in this study, based on relevant literature (Hwang et al., 2013; Sweller et al., 1998). The self-report scale contains two dimensions and four items: the 2-item mental effort dimension (extraneous and germane cognitive loads) and the 2-item mental load dimension (intrinsic cognitive load). Higher total scores represent higher mental effort, mental load, and cognitive load. The CLS scale-content validity index (S-CVI) and item-CVI (I-CVI) were 1.00 with Cronbach's α 0.85. 3.7. The Research Cognitive Test (RCT) The Research Cognitive Test (RCT) includes 20-item multiple choice questions (MCQ) and 3-item essay questions based on the teaching objectives of each unit, to measure students' individualized learning performance. The essay questions constituted 40% of the total score, and the answers were rated by trained research assistants with master's degree level using standardized answers. The total score ranged from 0 to 100 points. Higher scores represented higher cognitive understanding of research and critical appraisal competence of a research article. The S-CVI and I-CVI of the RCT were 0.95 and 0.99 for MCQ and 1.00 for essay questions, whereas the K-R 20 of MCQ and essay questions were 0.71 and 0.79. The inter-rater reliability of essay questions was 0.97. The difficulty level and discriminant index of the MCQ were 0.67 and 0.42 and those of the essay questions were 0.40 and 0.27, respectively. 3.8. Mental Efficiency Mental efficiency (E) was calculated using the formula E = (P − ME) / 2, with self-rated mental effort (ME) and performance (P) z scores. ME assumes equivalent performance and mental effort z scores (P = ME), is positive when performance scores are higher than mental scores, and is negative when performance scores are lower than mental scores. The formula can control for the plausibility that self-rated ME might be a reliable measurement of self-confidence or subject comfort rather than cognitive load (Galy et al., 2012). Thus, this study used RCT z score as the performance z score to calculate mental efficiency for comparison. 3.9. Team Critique Paper The team critique contained six domains of the rubric: introduction (10%), literature review (10%), research methods (40%), research results (15%), discussion and conclusion (10%), and assignment requirement and writing (15%). Each criterion of the six domains counted for 2–15% of four grades (excellent, 90–100; good, 89–89; neutral, 70–79; needs improvement, ≤69). The total score ranged from 0 to 100 points. Higher scores indicated greater appraisal competence. The Cronbach's α was 0.81 and the inter-rater reliability was 0.60 (p = .002).

4

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

Fig. 1. The process of verifying the sample eligible for data analysis.

3.10. Qualitative Feedback on the Course Satisfaction and Suggestions An open questionnaire was used to elicit from students the most satisfactory part of the teaching content, activities, or other areas and the aspects of this course needing improvement.

χ2(1) = 4.70, p = .030). Therefore, this variable was treated as a covariate for comparing mean difference in ANCOVA. 4.2. Cognitive Load

The Background Questionnaire was collected before the course (Table 1). The CLS, qualitative feedback on course satisfaction and suggestions, the RCT, and team critique paper were collected just after the course.

The experimental group had a higher mean cognitive load (19.81 ± 4.46) than the control group (17.34 ± 4.46). The experimental group had significantly higher means of mental load (intrinsic cognitive load) (F(1, 184) = 13.35, p b .001, η2 = .068) and mental effort (extraneous and germane cognitive load) (F(1, 184) = 39.36, p = .009, η2 = .037) and cognitive load (F(1, 184) = 13.02, p b .001, η2 = .066) than the control group after controlling for previous research experience (Table 3).

3.12. Data Analysis

4.3. Learning Performance

SPSS 21.0 Window software package (IBM Corporation, Armonk, NY, USA) was used to check assumptions and run descriptive and inferential analyses after deletion of five outliers using parametric analyses. Unpaired t-test, Pearson χ2 test, and Fisher Exact test were used to compare the background information of groups. The variable of “previous experience with research” differed between groups. ANCOVA was used to examine the mean differences in cognitive load and RCT by group. Reliability analysis was used to analyze the inter-rater and internal consistency reliability of the scales. Qualitative feedback was analyzed using content analysis.

The total RCT score of the experimental group (70.61 ± 8.03) was higher than that of the control group (44.92 ± 10.11). The mean MCQ score was 46.77 ± 6.28 for the experimental group and 31.33 ± 7.40 for the control group; the mean score of essay questions was 23.84 ± 3.68 for the experimental group and 13.59 ± 5.39 for the control group. Mean total score (F(1, 184) = 473.32, p b .001, η2 = .720), mean MCQ score (F(1, 184) = 232.52, p b .001, η2 = .558), and mean essay questions score (F(1, 184) = 387.66, p b .001, η2 = .678) were significantly higher in the experimental group than the control group (Table 3).

3.11. Data Collection

4.4. Mental Efficiency 3.13. Ethical Considerations

4. Results

Average mental efficiency scores were 0.33 ± 0.55 for the experimental group and − 0.31 ± 0.54 for the control group. Thus, performance scores of the experimental group were higher than mental effort scores, while performance scores of the control group were lower than mental effort scores. ANCOVA revealed a significant mean difference in mental efficiency between the two groups after controlling for previous research experience (F(1, 184) = 65.92, p b .001, η2 = .264) (Table 3).

4.1. Background Information

4.5. Team Critique Paper on Published Nursing Research Studies

Nursing students' age ranged 20–23 years with an average of 21.35 ± 0.52 (Table 2). All participants were single females who had graduated from 5-year ADN. Most participants had LPN or RN licenses (92.5%). Few students (8.0%) in either group had ever studied or conducted nursing research, but significantly more students in the experimental group had conducted nursing research (10.0% vs. 21.6%,

Total scores of 12 experimental groups on the critique paper ranged from 80.80 to 94.00 (88.45 ± 14.74). Fig. 2 displays the highest correct rates: introduction (92.80%), literature review (91.70%), and assignment requirement and writing (89.40%); the lowest correct rates were for results (86.33%), methods (87.00%), and discussion and conclusion (88.40%).

This work was carried out in accordance with The Declaration of Helsinki and the study protocol was approved by the Institutional Review Board (102-3873C). Written informed consent was obtained before the study from participants. Two students refused to participate and were assigned to other classes of their choice.

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

5

Table 2 Background information on the nursing students (n = 187) in the experimental and control groups. Variable

Experimental (n = 90)

Control (n = 97)

n (%)

n (%)

21–23 21.40 (0.52)

20–23 21.31 (0.53)

7 (7.8) 83 (92.2)

7 (7.2) 90 (92.8)

3 (3.3) 48 (53.3) 34 (37.8) 5 (5.6) 3 (2–5) 3.46 (0.66)

3 (3.1) 42 (43.3) 42 (43.3) 10 (10.3) 4 (2–5) 3.61 (0.72)

84 (93.3) 6 (6.7)

88 (90.7) 9 (9.3)

81 (90.0) 9 (10.0) 5 (5.6) 1 (1.1) 4 (4.4)

76 (78.4) 21 (21.6) 9 (9.3) 1 (1.0) 12 (12.4)

88 (97.8) 2 (2.2)

92 (94.8) 5 (5.2)

90 (100.0)

97 (100.0)

a

Age Range Mean (SD) Nursing licenseb RN LPN & RN Interest in nursingc Uninterested Neutral Interested Very interested Median (range) Mean (SD)a Previous nursing research studiesb No Yes Previous research experienceb No Yes Research subject or intervieweed Part-time research assistant Part-time workerd Previous attendance at any EBP conferencesc No Yes Previous attendance at literature appraisal conferences No

Statistical test

p value (2-tailed)

t(185) = −1.19

.236

χ2(1) = 0.02

.884

2.68

.456

t(185) = 1.52

.131

χ2(1) = 0.43

.511

χ2(1) = 4.70

.030

NA

.447

NA

NA

Note. SD, standard deviation; ADN, Associated Degree of Nursing; RN, Registered Nurse; LPN, Licensed Practical Nurse; EBP, evidence-based practice; NA, not applicable. a Unpaired t test. b Pearson χ2 test. c Fisher Exact test. d Multiple choice.

4.6. Qualitative Feedback on Course Satisfaction In the experimental group, student feedback indicated that learning EBP made them satisfied with asking and acquiring a PICO question and appraising the level of evidence; the process made them feel happy and effective and think about the patient's problem (n = 16, 40.0%). The second highest satisfaction was related to critiquing a research article (n = 7, 17.5%). Experimental students stated that critiquing a research article made them feel fulfilled and fruitful; helped them understand the research process; helped them blend harmoniously; and helped them clarify unclear parts through group discussion, class discussion, and individualized group discussion with instructors. By contrast, students in the control group (n = 8, 23.5%) reported that critiquing a research article helped them learn a lot and think; know nursing research; raise their ability to have an objective perspective and evaluate; and promoted their ability to critique evidence-based literature, understand group members' thoughts, clarify unclear portions with group discussion, and conduct individualized group discussion with instructors. The learners of experimental group found writing team critique paper rather stressful and recommended cancelation of the final examination (n = 7, 17.5%). They felt it was burdensome to spend time discussing PICO learning sheet (n = 4, 10%). Conversely, the learners of the control group felt difficulty in learning this course (n = 3, 8.82%) and wanted more exemplars for understanding research terms and course contents (n = 3, 8.82%).

5. Discussion The goal of this study was to understand the effect of integrating constructivist learning and EBP on student cognitive load and learning performance in a research course. In terms of cognitive load, results

showed that students in the experimental group had significantly higher mean item, subscale, and whole scale scores than those in the control group. Not surprisingly, students in the experimental group needed to learn relevant concepts of EBP and research, meaning that the learning environment for this group was more intense than that of the control group. Constructivist approaches emphasize the importance of deep learning so as to acquire learning strategies or methods to apply knowledge (Vogel‐Walcutt et al., 2011). Although the experimental group used some strategies (the cookie experiment, peer instruction, concise figures and tables in PowerPoint slides, and exemplar analyses) to optimize germane cognitive load, the poor command of English of the students limited their competence in acquiring and appraising evidence-based literature. Likewise, writing team assignments also required students to clarify the similarities and differences between research and EBP. Thus, students must make more effort and study more on new research terms and new concepts in order to overcome any difficulty, pass the course, and apply the knowledge to future practice. This finding was supported by studies of Schlairet et al. (2015), Fraser et al. (2012, 2014), Hwang et al. (2014), and Chen and Wu (2015) regarding the effects on cognitive load of simulation, webbased problem solving with concept mapping, mobile learning, and video lecture types of online learning media. However, this result was inconsistent with studies of Hwang et al. (2013) and Wu et al. (2012) on the effect of mobile learning on cognitive load. Hwang et al. (2013) found that a group of sixth graders learned with the inquiry-based mobile approach had significantly lower cognitive load than those learned with the traditional style during the in-field learning activity. Wu et al. (2012) found that nursing students learning physical assessment through mobile learning had significantly less cognitive load than those who learned through the traditional style with learning sheets. This difference might be explained by the need to acquire complex

6

Table 3 A comparison of cognitive load and Research Cognitive Test in the control and experimental groups using ANCOVA (n = 187). Levene's test F value

p value (2-tailed)

I felt distracted by the content of the course. Group Previous experience of conducting research I found it burdensome to deal with the content of the course. Group Previous experience of conducting research Mental load Group Previous experience of conducting research I felt that I put in a great effort to understand learning activities in this course. Group Previous experience of conducting research I felt that I put in a great effort to learn and understand course contents. Group Previous experience of conducting research Mental efforts Group Previous experience of conducting research CLS total score Group Previous experience of conducting research MCQ Group Previous experience of conducting research Essay questions Group Previous experience of conducting research Total scores of the NRCT Group Previous experience of conducting research Mental efficiency Group Previous experience of conducting research

F(1, 185) = 1.09

.298

F(1, 185) = 5.16

F(1, 185) = 0.21

F(1, 185) = 0.12

F(1, 185) = 0.05

F(1, 185) = 0.00

F(1, 185) = 0.00

F(1, 185) = 2.02

F(1, 185) = 0.90

F(1, 185) = 1.99

F(1, 185) = 0.20

Experimental (n = 90)

ANCOVA test

Mean (SD)

Mean (SD)

F value

95% CI

Partial η2

Observed power

.020 .855

−0.88 to −0.08 −0.50–0.60

.029 .000

0.64 0.05 0.99 0.14

p value (2-tailed)

R2 (adjusted R2) .029 (.019)

3.22 (1.33)

3.69 (1.42)

F(1, 184) = 5.48 F(1, 184) = 0.03

4.05 (1.63)

5.06 (1.49)

F(1, 184) = 17.62 F(1, 184) = 0.72

b.001 .397

−1.43 to −0.52 −0.89–0.35

.087 .004

7.27 (2.71)

8.74 (2.64)

F(1, 184) = 13.35 F(1, 184) = 0.16

b.001 .689

−2.24 to −0.67 −1.28–0.85

.068 .001

.953 .068

5.05 (1.17)

5.51 (1.20)

F(1, 184) = 5.61 F(1, 184) = 2.75

.019 .099

−0.76 to −0.07 −0.86–0.08

.030 .015

0.654 0.378

5.02 (1.32)

5.56 (1.27)

F(1, 184) = 7.35 F(1, 184) = 0.14

.007 .707

−0.90 to −0.14 −0.62–0.42

.038 .001

.770 .066

10.07 (2.39)

11.07 (2.37)

F(1, 184) = 39.96 F(1, 184) = 1.06

.009 .305

−1.63 to −0.24 −1.44–0.45

.037 .006

.753 .176

17.34 (4.46)

19.81 (4.46)

F(1, 184) = 13.02 F(1, 184) = 0.62

b.001 .432

−3.69 to −1.08 −2.49–1.07

.066 .003

0.95 0.12

31.33 (7.40)

46.77 (6.28)

F(1, 184) = 232.52 F(1, 184) = 0.68

b.001 .410

−5.86 to −4.52 −0.53–1.30

.558 .004

1.00 0.13

13.59 (5.39)

23.84 (3.68)

F(1, 184) = 387.66 F(1, 184) = 0.37

b.001 .547

−16.20 to −13.25 −2.62–1.39

.678 .002

1.00 0.09

44.92 (10.11)

70.61 (8.03)

F(1, 184) = 473.32 F(1, 184) = 0.08

b.001 .079

−33.04 to −27.55 −3.21–4.27

.720 .000

1.00 0.06

0.33 (0.55)

F(1, 184) = 65.92 F(1, 184) = 1.12

b.001 .291

.264 .006

1.00 0.18

.024

.098 (.088)

.645

.072 (.062)

.731

.051 (.041)

.819

.042 (.031)

.990

.048 (.037)

.960

.075 (.065)

.157

.561 (.556)

.345

.686 (.683)

.160

.724 (.721)

.658

.264 (.256) −0.31 (0.54)

Note. ANCOVA, Analysis of Covariance; SD, standard deviation; CI, confidence interval; CLS, Cognitive Load Scale; MCQ, Multiple Choice Questions; RCT, Research Cognitive Test.

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

Variable/covariates

Control (n = 97)

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

7

Fig. 2. The correct rates of each section and total score for critiquing research articles, in the experimental group.

and abstract concepts for students with little or no past experience. However, mental efficiency was positive in the experimental group and negative in the control group, with significantly different mean scores. The higher mental load (intrinsic cognitive load), mental effort (extraneous and germane cognitive load), and mental efficiency of the experimental group students indicated that they had a more complicated learning content and more challenging tasks in their learning environment. Conversely, the control group students in a conventional research program had a less complex environment and fewer challenges in their learning materials and tasks. On the RCT, the experimental group had significant higher means of scores on MCQ and essay questions. This result was similar to those of Liou et al. (2013). The implication is that the higher nursing research knowledge, mental load, mental effort, and mental efficiency of the experimental group was caused by the complexity and challenges of learning the content, but students made the effort to conquer and learn. Students in the experimental group had higher correct rates on the sections of introduction, literature review, and assignment requirement and writing, and lower correct rates on the sections of results, methods, and discussion and conclusion. According to Bloom's taxonomy, as revised by Anderson et al. (2001), the six major levels of learning are, in the order of complexity: remembering, understanding, applying, analyzing, evaluating, and creating. The student provided with good results and methods early in the assignment is prepared to apply a higherlevel analysis to discussions and conclusions integrated into the research course. Students need to learn higher order thinking, such as making decisions systematically, rather than relying primarily on rote memorization. In terms of course satisfaction, students in the experimental group were satisfied with learning EBP (40.0%) and critiquing the research article (17.5%) through learning by doing, collaborative learning, and individualized group discussion with instructors. These results are similar to those of McCurry and Martins (2010), Balakas and Sparks (2010), and Smith-Stoner and Molle (2010). Conversely, some students in the control group reported satisfaction with learning to critique the research article through learning by doing and group discussion (23.5%).

with this research design. Thus, we could not standardize the true- or quasi-experimental research articles and grading rubric for critique papers with both cohorts. The results of the paper critique apply only to the experimental group. Third, although inter-rater reliability of the team critique paper is low, it has a significant correlation. This might be due to the small number of twelve-team based papers. Fourth, there are low response rate in the qualitative feedback on course satisfaction on the experimental group (44.44%) and the control group (35.05%). This might not provide a whole profile about learner satisfaction on the research course. 6. Conclusion Integrating EBP into a research program not only improves the research knowledge of RN-to-BSN students but also increases their mental load, mental effort, and mental efficiency. Ongoing research and applications of the theory continue in the nursing curriculum. 6.1. Implications Cognitive load and performance in nursing research and EBP require further studies to quantify learners' responses in different learning systems (2-year RN-to-BSN and 4-year BSN program) using the developed instrument to measure three types of cognitive load. Future studies need to use standardized research papers for the team critique paper for both groups. These findings may help educators design more effective and interesting curricula for nursing research and EBP, to foster student competence in applying EBP to patient care. Contributions SIH and LLH were responsible for research conception and design. SIH delivered the intervention, collected the data, and conducted data analysis. SIH and LLH drafted and revised the manuscript. TSH has given opinions and revised the manuscript. All authors have read and approved the final draft.

5.1. Limitations

Funding

Although these findings are helpful to educators who plan to integrate EBP into a research course, there are several limitations. First, because this study was conducted in an RN-to-BSN program at a university, it was not generalizable to other types of nursing education. Second, instructors of the control group were reluctant to cooperate

This study was sponsored by a CDRPF Grant (CDRPF1B00111B0013) from Linkou Chang Gung Memorial Hospital of Chang Gung Medical Foundation in Taiwan. The funder played no role in study design; in the collection, analysis and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

8

S.-I. Hsieh et al. / Nurse Education Today 42 (2016) 1–8

Conflicts of interest None. Acknowledgments We appreciate all the baccalaureate nursing students who participated in this study and the administrators who provided assistance at Chang Gung University of Science and Technology. References American Association of College of Nursing, 2015. Hallmarks of the professional nursing practice environment (Retrieved July 25, 2015, from http://www. aacn.nche.edu/ publications/white-papers/hallmarks-practice-environment). Anderson, L.W., Krathwohl, D.R., Bloom, B.S., 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. Allyn & Bacon, New York, NY. Balakas, K., Sparks, L., 2010. Teaching research and evidence-based practice using a service-learning approach. J. Nurs. Educ. 49 (12), 691–695. http://dx.doi.org/10. 3928/01484834-20100831-07. Behmake, D.A., Atwood, C.H., 2013. Implementation and assessment of cognitive load theory (CLT) based questions in an electronic homework and testing system. Chem. Educ. Res. Pract. 14 (3), 247–256 http://101039/C3RP20153H. Bloom, K.C., Olinzock, B.J., Radjenovic, D., Trice, L.B., 2013. Leveling EBP content for undergraduate nursing students. J. Prof. Nurs. 29 (4), 217–224. http://dx.doi.org/10.1016/j. profnurs.2012.05.015. Brunken, R., Plass, J.L., Leutner, D., 2003. Direct measurement of cognitive load in multimedia learning. Educ. Psychol. 38 (1), 53–61. Chen, C.M., Wu, C.H., 2015. Effects of different video lecture types on sustained attention, emotion, cognitive load, and learning performance. Comput. Educ. 80 (2015), 108–121. http://dx.doi.org/10.1016/j.compedu.2014.08.015. Christie, J., Hamill, C., Power, J., 2012. How can we maximize nursing students' learning about research evidence and utilization in undergraduate, preregistration programmes? A discussion paper. J. Adv. Nurs. 68 (12), 2789–2801. http://dx.doi. org/10.1111/j.1365-2648.2012.05994.x. Cooper, R., 2007. An investigation into constructivism within an outcomes based curriculum. Issues in Educ. Res. 17 (1), 15–39 Retrieved August 10, 2015 from http://www. iier.org.au/iier17/cooper.html. DeCoux Hampton, M., 2012. Constructivism applied to psychiatric-mental health nursing: an alternative to supplement traditional clinical education. Int. J. Ment. Health Nurs. 21 (1), 60–68. http://dx.doi.org/10.1111/j.1447-0349.2011.00755.x. Ertmer, P.A., Newby, T.J., 2013. Behaviorism, cognitivism, constructivism: comparing critical features from an instructional design perspective. Perform. Improv. Q. 26 (2), 43–71. http://dx.doi.org/10.1002/piq.21143. Faul, F., Erdfelder, E., Lang, A.G., Buchner, A., 2007. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39 (2), 175–191. http://dx.doi.org/10.3758/BF03193146. Fraser, K., Ma, I., Teteris, E., Baxter, H., Wright, B., McLaughlin, K., 2012. Emotion, cognitive load and learning outcomes during simulation training. Med. Educ. 46 (11), 1055–1062. http://dx.doi.org/10.1111/j.1365-2923.2012.04355.x. Fraser, K., Huffman, J., Ma, I., Sobczak, M., McIlwrick, J., Wright, B., McLaughlin, K., 2014. The emotional and cognitive impact of unexpected simulated patient death: a randomized controlled trial. Chest 145 (5), 958–963. http://dx.doi.org/10.1378/chest. 13-0987. Galy, E., Cariou, M., Melan, C., 2012. What is the relationship between mental workload factors and cognitive load types? Int. J. Psychophysiol. 83 (3), 269–275. http://dx. doi.org/10.1016/j.ijpsycho.2011.09.023. Hastings, C., Fisher, C.A., 2014. Searching for proof: creating and using an actionable PICO question. Nurs. Manag. 45 (8), 9–12. http://dx.doi.org/10.1097/01. Hickey, M.T., Forbes, M., Greenfield, S., 2010. Integrating the Institute of Medicine competencies in a baccalaureate curricular revision: process and strategies. J. Prof. Nurs. 26 (4), 214–222. http://dx.doi.org/10.1016/j.profnurs.2010.03.001. Hwang, G.J., Wu, P.H., Zhuang, Y.Y., Huang, Y.M., 2013. Effects of the inquiry-based mobile learning model on the cognitive load and learning achievement of students. Interact. Learn. Environ. 21 (4), 338–354. http://dx.doi.org/10.1080/10494820.2011.575789. Hwang, G.J., Kuo, F.R., Chen, N.S., Ho, H.J., 2014. Effects of an integrated concept mapping and web-based problem-solving approach on students' learning achievements, perceptions and cognitive loads. Comput. Educ. 71, 77–86. http://dx.doi.org/10.1016/j. compedu.2013.09.013. Institute of Medicine (US), 2001. Committee on quality of health care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academy Press, Washington, DC.

Josephsen, J., 2015. Cognitive load theory and nursing simulation: an integrative review. Clin. Simul. Nurs. 11 (5), 259–267. http://dx.doi.org/10.1016/j.ecns.2015.02.004. Kessler, T.A., Alverson, E.M., 2014. Mentoring undergraduate nursing students in research. Nurs. Educ. Perspect. 35 (4), 262–264. http://dx.doi.org/10.5480/11-555.1. Leppink, J., Paas, F., van Gog, T., van der Vleuten, C.P.M., van Merrienboer, J.J.G., 2014. Effects of pairs of problems and examples on task performance and different types of cognitive load. Learn. Instr. 30, 32–42. http://dx.doi.org/10.1016/j.learninstruc. 2013.12.001. Lincoln, Y.S., Guba, E.G., 2000. Paradigmatic controversies, contradictions, and emerging confluences. In: Denzin, N.K., Lincoln, Y.S. (Eds.), The SAGE Handbook of Qualitative Research, second ed. Sage, Thousand Oaks, CA, pp. 136–188. Liou, S.R., Chang, C.Y., Tsai, H.M., Chang, C.H., 2013. Innovative strategies for teaching nursing research in Taiwan. Nurs. Res. 62 (5), 335–343. http://dx.doi.org/10.1097/ NNR.0b013e31829fd827. McCurry, M.K., Martins, D.C., 2010. Teaching undergraduate nursing research: a comparison of traditional and innovative approaches for success with millennial learners. J. Nurs. Educ. 49 (5), 276–279. Meeker, M.A., Jones, J.M., Flanagan, N.A., 2008. Teaching undergraduate nursing research from an evidence-based practice perspective. J. Nurs. Educ. 47 (8), 376–379. http:// dx.doi.org/10.3928/01484834-20080801-06. Moch, S.D., Cronje, R.J., 2010. Part II. Empowering grassroots evidence-based practice: a curricular model to foster undergraduate student-enable practice change. J. Prof. Nurs. 26 (1), 14–22. http://dx.doi.org/10.1016/j.profnurs.2009.03.003. Moch, S.D., Cronje, R.J., Branson, J., 2010. Part I. Undergraduate nursing evidence-based practice education: envisioning the role of students. J. Prof. Nurs. 26 (1), 5–13. http://dx.doi.org/10.1016/j.profnurs.2009.01.015. Naismith, L.M., Haji, F.A., Sibbald, M., 2015. Practising what we preach: using cognitive load theory for workshop design and evaluation. Perspect. Med. Educ. 4, 344–348. http://dx.doi.org/10.1007/s40037-015-0221-9. National League for Nursing, 1977. Criteria for the Appraisal of Baccalaureate and Higher Degree Programs in Nursing. Author, New York, NY. Nursing and Midwifery Council, 2010. New Standards to Support Learning and Assessment in Practice: NMC Standards for Mentors, Practice Teachers and Teachers. Author, London. Odell, E., Barta, K., 2011. Teaching evidence-based practice: the Bachelor of Science in nursing essentials at work at the bedside. J. Prof. Nurs. 27 (6), 370–377. http://dx. doi.org/10.1016/j.profnurs.2011.04.007. Paas, F., Tuovinen, J.E., Tabbers, H., van Gerven, P.W.M., 2003. Cognitive load measurement as a means to advance cognitive load theory. Educ. Psychol. 38 (1), 63–71. http://dx.doi.org/10.1207/S15326985EP3801_8. Pravikoff, D.S., Tanner, A.B., Pierce, S.T., 2005. Readiness of U.S. nurses for evidence-based practice: many don't understand or value research and have little or no training to help them find evidence on which to base their practice. Am. J. Nurs. 105 (9), 40–51. http://dx.doi.org/10.1097/00000446-200509000-00025. Rolloff, M., 2010. A constructivist model for teaching evidence-based practice. Nurs. Educ. Perspect. 31 (5), 290–293. http://dx.doi.org/10.1043/1536-5026-31.5.290. Schlairet, M.C., Schlairet, T.J., Sauls, D.H., Bellflowers, L., 2015. Cognitive load, emotion, and performance in high-fidelity simulation among beginning nursing students: a pilot study. J. Nurs. Educ. 54 (3), S5–S11. http://dx.doi.org/10.3928/01484834-20150218-10. Smith-Stoner, M., Molle, M.E., 2010. Collaborative action research: implementation of cooperative learning. J. Nurs. Educ. 49 (6), 312–318. http://dx.doi.org/10.3928/ 01484834-20100224-06. Spatz, D., 2008. Involving undergraduate students in nursing research: why it is a win– win. Dean's Notes 30 (2), 1–2 Retrieved August 10, 2015, from http://www.ajj. com/sites/default/files/services/publishing/deansnotes/nov08.pdf. Sweller, J., van Merrienboer, J.J.G., Paas, F.G.W.C., 1998. Cognitive architecture and instructional design. Educ. Psychol. Rev. 10 (3), 251–296. http://dx.doi.org/10.1023/A: 1022193728205. Thompson, C.J., 2006. Fostering skills for evidence-based practice: the student journal club. Nurse Educ. Pract. 6 (2), 69–77. http://dx.doi.org/10.1016/j.nepr.2005.08.004. Tsai, H.M., Cheng, C.Y., Chang, C.H., Liou, S.R., 2014. Preparing the future nurses for nursing research: a creative teaching strategy for RN-to-BSN students. Int. J. Nurs. Pract. 20 (1), 25–31. http://dx.doi.org/10.1111/ijn.12119. Vogel‐Walcutt, J.J., Gebrim, J.B., Bowers, C., Carper, T.M., Nicholson, D., 2011. Cognitive load theory vs. constructivist approaches: which best leads to efficient, deep learning? J. Comput. Assist. Learn. 27 (2), 133–145. http://dx.doi.org/10.1111/j.13652729.2010.00381.x. Wu, P.H., Hwang, G.J., Su, L.H., Huang, Y.M., 2012. A context-aware mobile learning system for supporting cognitive apprenticeships in nursing skills training. J. Educ. Technol. Soc. 15 (1), 223–236http://www.jstor.org/stable/jeductechsoci.15.1.223. Young, J.Q., Van Merrienboer, J., Durning, S., Cate, O.T., 2014. Cognitive load theory: implications for medical education: AMEE Guide No. 86. Med. Teach. 36 (5), 371–384. http://dx.doi.org/10.3109/0142159X.2014.889290.