The Effects of Biology Lab Delivery on Academic Achievement in Biology in a Sample of Non-Majors College

Undergraduate StudentsPaper presented at the annual meeting of the Southwest Educational Research Association, San Antonio, TX February, 16, 2017

Jaime Ann McQueen (Doctoral Candidate)

Dr. Lauren Cifuentes- Dissertation Chair Dr. Tonya Jeffery- Committee Member

Statement of the Problem

Physical Lab issues:• Content often difficult for students to visualize • Non-majors students struggle • Require extensive time and space commitments

Virtual labs may help to:• Provide an additional level of support to students • Ease demand for STEM physical lab sections

Statement of the problem

Virtual labs:• Viable economic alternatives to costly laboratory equipment and

chemicals • Conserve laboratory resources • Save valuable laboratory space

Virtual lab issues:•Do not adequately represent dissection of physical specimens •May not give students the necessary foundation of laboratory skills

•Lack effectiveness of modeling physical laboratories

Context

Physical Labs:• Offer limited provision of learner control as they are constrained by very specific

instructions, time and scheduling concerns, and limited opportunities for repetition (Brinson, 2015).

• Instructor presence, where learners are able to communicate, ask questions, and receive guidance from instructors during a course or lab has been shown to enhance student learning and understanding of course and laboratory content (De Jong, Linn, & Zacharia, 2013; Picciano, 2002; Stuckey-Mickell & Stuckey-Danner, 2007).

Virtual Labs:• Students are actively in control of interaction with simulated lab equipment and

experiments, pacing, repetition, and their own learning (Pyatt & Sims, 2012).• Communication between instructors and students is critical to students’ success in online

learning environments, immediacy may be lacking in distance based learning (Crippen et al., 2013; De Jong et al., 2013; Dunlap, Verma, & Johnson, 2016; Jaggars, Edgecombe, & Stacey, 2013; Picciano, 2002).

Theoretical Framework

Instructor Presence and Learner control

• Instructor presence: includes specific levels of guidance provided by instructors which promote successful student learning in STEM subjects (Ahmed & Hasegawa, 2014; Chen et al., 2016; Pedersen & Irby, 2014; Smith, 2015; Zacharia et al., 2015).

• Learner control: learners take responsibility for the pace, repetition, and sequence of content in learning environments (Dede, 2009; Hanafin, 1984; Simsek, 2012).

Literature Review

• Studies on physical vs. virtual labs in STEM (Akpan, 2001; Zacharia

et al., 2015). • Virtual labs provide guidance to students (Parker & Loudon, 2012;

Swan & O’Donnell, 2009)

• Effectiveness of virtual labs (Carnevale, 2003; Flowers, 2011). • Recent studies on virtual labs (Brinson, 2015; Hallyburton & Lunsford,

2013 Ma & Nickerson, 2006; Zacharia et al., 2015)

Literature Review

Previous Research • Instructor presence (Ahmed & Hasegawa, 2014; Chen et al., 2016; Pedersen & Irby, 2014; Smith, 2015; Zacharia et al., 2015)

Gaps in Current Research

Effects and impacts of instructor presence learner control in virtual lab environments

• Learner control (Hasler, Kersten, & Sweller, 2007; Zacharia et al., 2015)

• Instructor presence (Ahmed & Hasegawa, 2014; Chen et al., 2016; Pedersen & Irby, 2014; Smith, 2015; Zacharia et al., 2015).

Purpose of study

Measure comparative effects of four levels of biology lab delivery and qualitatively explore how students describe their experiences of instructor presence and learner control of pace

Compare effects and impacts of instructor presence and learner control in physical and virtual labs

Research Questions & HypothesesThe quantitative and qualitative research questions that guided the study are as follows:

Quantitative1. What are the comparative effects of four levels of biology lab delivery on non-majors college biology

students’ test scores immediately following completion of a lab, and after a one week delay? The four levels compared are-

a. a physical based lab with instructor presence (PL), b. a virtual lab with no instructor presence (VL), c. a virtual lab with instructor presence (VLIP) , and d. a virtual lab with instructor presence and direction for learner control of pace and repetition beyond

lab time (VLIPLC).Qualitative2. How do non-majors college biology students describe their experiences of instructor presence and

learner control of pace and repetition in each of the four treatments?

Three alternate hypotheses will be tested: 1) main effect intervention. 2) main effect time. 3) intervention by time interaction effect.

 

Operational DefinitionsMode of biology lab delivery• Instructor presence in physical based labs (PL)-the instructor being physically present and available to answer questions

in the lab environment for a duration of 50 minutes. The instructor is available via phone, e-mail, and Black Board before

and after class as well.

• VL virtual lab with no instructor presence- a lab completed solely in an online environment (Brinson, 2015; Ma &

Nickerson, 2006).

• Instructor presence in virtual based labs (VLIP)-the instructor being virtually present and available to answer questions

through phone, e-mail, and Black Board.

• Learner control in physical based labs-the ability for students to control the pacing, repetition, time spent learning, and

access to available guidance when they need it.

• Learner control in virtual based labs (VLIPLC)-the ability for students to control the pacing, repetition, and access

available guidance online when they need it.

30 item Test- one of three content knowledge tests consisting of 30 multiple choice items selected from biology test banks

published by Openstax .

Academic achievement –student scores measured by two 30 item multiple choice biology content knowledge post-tests

(immediate, one-week delayed).

Students’ experience of instructor presence and learner control –how students describe the availability of the instructor and

their control of pacing, repetition, time spent, and access to available guidance in the PL, VL, VLIP, and VLIPLC levels of

biology lab delivery.

PL Group VL Group VLIP Group VLIPLC Group

Instructor Presence Instructor is available in person to answer questions and to help with lab

No IPInstructor is virtually available as needed and encourages student contact for help with lab

Instructor is virtually available as needed and encourages student contact for help with lab

Learner Control Student follows lab manual for 50 mins.

Student follows lab manual at their own pace

Student follows lab manual at their own pace

Student follows lab manual at their own pace and is encouraged to repeat the processes

DirectedLC

Operational Definition Table

Method

Participants: Non-probability sampling was used to select student participants enrolled in four sections of a college level undergraduate introductory biology course (BIOL 1308) at a south Texas University during the fall 2016 semester. Each of the four sections was randomly assigned to a lab delivery mode treatment.• PL- (n=21) • VL- (n=25)• VLIP- (n=22)• VLIPLC-(n=24)

Quantitative VariablesIndependentMode of Biology Lab Delivery• PL• VL• VLIP• VLIPLC

Dependent• Performance on two Post Tests (immediate, one-week delayed)

Method

Qualitative Approach

Focus group and Interview• Three (30min) focus groups, one each for: PL (n=5), VL (n=4), and

VLIPLC (n=5)• One (30min) interview, for VLIP (n=1)• Sample questions:

“How did the lab help you to learn biology content?” “How many times did you repeat the lab and how?” “Did you seek or receive help from your instructor while completing the virtual lab, if so, how?”

Research Design

• Sequential explanatory mixed methods (Creswell, 2014; Creswell & Plano Clark, 2006; Creswell et al., 2003)

• Quasi-experimental study, lacks the random sampling of a true experiment (Shadish, Cook, & Campbell, 2002).

Research Design

Pre-Test Treatment Immediate Recall Post-Test

Delayed Recall Post-Test

PL Group y X1 y y

VL Group y X2 y y

VLIP Group y X3 y y

VLIPLC Group y X4 y y

• 4 X 3 repeated measures split plot design

Materials

QuantitativeAll four groups completed the same pre-test, lab objectives, and post-tests.• All training and guidance will be delivered prior to beginning labs

VL groups: Computer based introductory tutorial provided by Sapling Learning that acquainted students with the virtual lab interface.PL Group: Students received guidance and directions from the course instructor as part of normal course pre-lab activity.

• The same lab content will be delivered to the PL and VLs groups– VL Groups: Meiosis & Mitosis (Sapling Learning, General Biology, 2016). – PL Group: Exercises 6.1 The Cell Cycle & 6.2 Meiosis (Pendarvis & Crawley, 2016)

• Content validity of the Sapling virtual lab and the physical based lab has been established by previous research studies. These materials are previously published and widely distributed.

• Subject matter expert reviewed both labs to further ensure content validity.

Quantitative Instrumentation

Quantitative• The researcher designed three equivalent, matched, test forms on the topic of

meiosis and mitosis to measure students’ academic achievement. – 30 item multiple-choice pre-test administered prior to lab delivery. – 30 item multiple-choice immediate recall post-test given immediately following delivery of labs.– 30 item multiple-choice delayed recall post-test given one week following lab completion.

Questions were selected from previously published test banks from Openstax Biology and Concepts of Biology, published by Rice University.

• Reliability (Cronbach’s α):– Summer II Pilot: Pre-Test (.71), Immediate Post-Test (.81), One-week Delayed Post-Test (.84)– Study: Pre-Test (.62), Immediate Post-Test (.76), One-week Delayed Post-Test (.81)

• Matching and equivalence of each test item across the pre-test and post-tests was ensured through correlation of unique question ID numbers and difficulty scales provided as part of the test banks.

PL Group VL Group VLIP Group VLIPLC Group

Weeks 1-4 of semester

Consent of the study y y y y

Week 5 of the semester

Participate in the pretest y y y y

Week 6 of the semester

1.Receive lab tutorial y y y y

2.Complete lab treatment PL VL VLIP VLIPLC

3.Immediate Post-Test y y y y

Week 7 of the semester

Delayed Post-Test y y y y

Week 8 of the semesterFocus Group y y y y

Data Collection & Procedure

Quantitative Data Analysis

Quantitative• Pre-experimental equivalence was assumed, a 4x3 repeated

measures ANOVA was conducted (Huck, 2000; Urdan, 2010).

• IBM (SPSS) v. 23.

• The mean difference effect sizes were computed to examine practical significance of the findings.

Qualitative Data Analysis

Qualitative• Focus group data was audio recorded using the voice memo feature of an

iPhone 6. • Audio data was transcribed verbatim into Microsoft word and sorted into

codes, categories, and themes using MAXQDA 11. • Researcher took analytic memos, as suggested by Saldana (2009). • First cycle coding: structural coding, Second cycle coding: magnitude coding

(Saldana, 2009).• Qualitative findings were integrated with the quantitative results of the study

to describe students experiences of the affordances of IP and LC in biology labs

Methodological Framework• Interpretivism (Crotty, 1998)

Quantitative ResultsQuantitative• The time effect was statistically significant F(2,176) =148.65, p < 0.01, all groups learned

significantly from the pre-test to the immediate post-test, scores remained constant between the immediate post-test and one-week delayed post-test.

• The mode of delivery effect was not statistically significant F(3,88) = 0.38, p = 0.76, all students performed equivalently well, regardless of lab delivery mode.

• Mean difference effect sizes were computed to examine practical significance of the findings.– Pre-Test to Immediate post-test effect size range: 0.99-2.00– Immediate post test to One-week delayed post-test effect size range: -0.25-0.44– Pre-Test to One-week delayed post-test effect size range: 1.23-1.71

*Note: 0.20 = small effect, 0.50 = medium effect, and > 0.80 = large effect (Cohen, 1988)

• Small sample sizes (low power) was acknowledged as interaction effect and interaction x time effect were not statistically significant. Output analysis revealed sample sizes of (n=30) per group would have yielded a statistically significant interaction effect

Qualitative Results

Qualitative• An analysis of the data from the interview and the three focus groups resulted in three themes: instructor presence, learner control, and unique laboratory experiences

• eight categories : instructor-student communication; instructor-guidance; repetition;

pacing; time spent learning; access to available guidance as needed; students' insight

into learning; and students' suggestions to improve labs.

Discussion

Quantitative• Time effect: The improvement in scores from the pre-test to immediate post-test

indicates students in all groups learned significantly. The lack of statistically significant change in scores between the immediate post-test and one-week delayed post-test indicates students retained knowledge.

• Interaction effect: The equivalent performance among students in all lab delivery modes indicates that virtual labs can produce learning outcomes equivalent to physical labs (cite).

• Meaningful effect sizes: Indicate that lack of a statistically significant interaction effect is due to the small sample sizes of the groups (low power).

Instructor presence and learner control• Students in PL, VLIP, and VLIPLC group made use of instructor presence during lab

time, but not in the week following.• Students in VL, VLIP, and VLIPLC groups made use of learner control during lab time,

but not in the week following.

Discussion QualitativePL Group• Appreciated having a physically available instructor• Felt constrained by lack of microscopes and lab equipment• Wanted more time to review lab content

VL Groups• Enjoyed being able to go at their own pace, repeat the lab, and look at cell animations.• Appreciated when an instructor was present, but didn’t feel it was necessary to learn.• Enjoyed not having to “mess with complicated lab equipment”• Expressed some confusion related to the hints and feedback provided by the virtual lab.

Instructor presence and learner control• Students expressed they did not use instructor presence after the lab due to the rapid pacing of

the semester “we’re moving on to something different next week”• Students expressed they did not use learner control and repeat the virtual lab, because they

“had a course biology test for a grade” that week.

Discussion

The Best Finding of All…

• Students in all lab delivery modes felt their lab was beneficial to their learning!• Despite the ‘glitches’ of physical and virtual labs, ultimately, students can be

positive of their laboratory learning experiences, thanks to helpful instructors.• As instructional designers, researchers, and curriculum publishers, we should

continue to support our students during their labs. Additionally, we should continue to research best practices in laboratory teaching and find new ways to deliver supportive labs to our students.

Significance of the Study

Findings from this study will inform science educators regarding the effects of instructor presence which is afforded in physical labs and learner control which is afforded in virtual labs.

Virtual Labs can: • Expand science education options for college students. • help online learners, non-science majors students, students with disabilities.

This research will help inform the fields of higher education, curriculum and instruction, and educational technology. • Virtual lab research is timely and relevant (Darrah et al., 2014; Johnson,

2002; Miller, 2008).

I intend to share my study and findings with institutions of higher learning, curriculum publishers, and all other parties interested in the utility of virtual laboratories.

ReferencesAkpan, J. P. (2001). Issues associated with inserting computer simulations into biology instruction: A review of the literature. Electronic Journal of Science Education, 5(3).

Bell, J. (1999). The biology labs on-line project: Producing educational simulations that promote active learning. Interactive multimedia electronic journal of computer-

enhanced learning, 1(2).

Bhargava, P. Antonakakis, J., Cunningham, C. & Zehnder, A.T. (2006). Web-based virtual torsion laboratory. Computer Applications in Engineering Education, 14(1), 1-8.

Brinson, J. R. (2015). Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: A review of the empirical research.

Computers & Education, 38(3), 218-237. doi:10.1016/j.compedu.2015.07.003

Chen, J. A., Tutwiler, M. S., Metcalf, S. J., Kamarainen, A., Grotzer, T., & Dede, C. (2016). A multi-user virtual environment to support students' self-efficacy and interest in

science: A latent growth model analysis. Learning and Instruction, 41, 11-22.

Crotty, M. (1998). The foundations of social research: Meaning and perspective in the research process. London, UK: Sage.

Flowers, L. O. (2011). Investigating the effectiveness of virtual laboratories in an undergraduate biology course. The Journal of Human Resource and Adult Learning, 7(2),

110-116.

Ma, J., & Nickerson, J. V. (2006). Hands-on, simulated, and remote laboratories: a comparative literature review. ACM Computing Surveys, 3(1), 1-24.

Swan, A. E., & O’Donnell, A. M. (2009). The contribution of a virtual biology laboratory to college students’ learning. Innovations in Education and Teaching International,

46(4), 405-419.

Zacharia, Z. C., Manoli, C., Xenofontos, N., de Jong, T., Pedaste, M., van Riesen, S. A., & ... Tsourlidaki, E. (2015). Identifying potential types of guidance for supporting

student inquiry when using virtual and remote labs in science: A literature review. Educational Technology Research and Development, 63(2), 257-302.

Questions ???

I appreciate your feedback and questions.

Thank you

Contact me via e-mailjmcqueen@islander.tamucc.edu