Introduction
Students in an ungraduated biology lab course are tasked with developing their scientific communication skills by writing a lab report on the experiments conducted during class. The lab report grade is worth 10% of their final grade making it a summative assignment. Historically, students have struggled to apply the scientific method to their writing resulting in low lab report grades. Scaffolding has been shown to be an effective method for improving student abstract writing. (Christian and Kearns, 2018) Consequently, I decided to investigate whether scaffolding by allowing students practicing scientific writing through a formative and summative draft submission results in increased lab report grades. I hypothesized that students who practice scientific writing skills through scaffolded assessments earn passing lab report grades. This hypothesis was tested by allowing students to submit a formative draft of their lab report as practice prior to submitting a summative final report. Implementation of formative and summative assessment is linked to improved learning outcomes for students (Lynn et al., 2016). Additionally, I applied Bloom’s taxonomy to a rubric as demonstrated effected by Bissell and Lemons to assess each draft (Bissell and Lemons, 2006).
Methods
Students submitted formative drafts, of each section (Introduction, Methods, Results, and Discussion) of the lab report. These drafts together were worth 0.06% of their final grade. A rubric was designed using Blooms’s taxonomy to assess how effectively the lab reports communicated the students’ scientific findings. I provided detailed written feedback, which aligned with the rubric regarding how well the hypothesis was supported by observations in the primary literature and used to design an experiment and generate rationally predicted outcomes. I also provided feedback on how clearly the tables and figures conveyed the data presented and how reasonably the conclusions explained the observed results with support from the primary literature. Students were assessed based on the criteria outlined in the rubric. After receiving feedback on the drafts, students were given the opportunity to make corrections before submitting a summative final lab report, which was worth 10% of their final grade. Student grades were measured after each submission. Rubrics for the draft and final report were identical. Data was collected for 58 students from three separate sections of an undergraduate biology lab course over the course of three semesters (1 section per semester).
Results
After practicing their writing skills and incorporating my feedback into their lab reports, student grades increased from 58.11% to 73.94% (Fig. 1A). Scaffolding coupled with instructor feedback more greatly impacted student grades in the lower quartile and the minimum grade (Fig. 1A). These differences in scores from draft 1 to draft 2 can be attributed to significant improvement in four components, one from each section of the lab report (i.e. Introduction, Methods, Results, Discussion) (Fig. 1B).
Figure 1 Lab report grades increase after scaffolding. Students submitted a lab report draft (Draft 1), following which I provided feedback based on a rubric and an opportunity for students to revise the draft. Revised student lab reports (Draft 2) were then graded using an identical rubric. (A) Data depicts lab report grade distribution across drafts. Data was collected from three course sections of 60 total students over the course of three semesters. (B) Data shown depicts average student grades per component that was significantly increased in draft 2 relative to draft 1. I4 is Introduction component 4, M3 is Methods component 3, R3 is Results component 3, and D1 is Discussion component 1. Data was collected from three course sections of 60 total students over the course of three semesters.
The component score breakdown revealed that lab report scores changed most dramatically in the Methods 3 (M3) component, which assessed students’ ability to describe and operationally define variables and controls (Table 1). Scores for this component increased on average from 29.36% to 79.26% (Fig. 1B). Students’ ability to justify a hypothesis with primary literature, summarize their results in written text, use their results to form a discussion also increased significantly following scaffolding and instructor feedback (Fig. 1B).
Table 1 – Components for which a significant increase in average component grade was observed
Lab report grades improved by an average of 15 percentage points (Fig. 2A). While the biggest increase in student grade was observed for those students who scored poorly on the first draft (Fig. 2B).
Figure 2 Lab report grades increase after instructor feedback and student revision. Lab report grades increase after instructor feedback and student revision. Students submitted a lab report draft (Draft 1), following which I provided feedback based on a rubric and an opportunity for students to revise the draft. Revised student lab reports (Draft 2) were then graded using an identical rubric. (A) Shown is the percentage point by which individual student grades increased. (B) Data shows individual student lab report grades for Draft 1 relative to Draft 2. Data was collected from three course sections of 60 total students over the course of three semesters.
On average, lab report scores improved by ~30%, while some scores improved by as much as 300% or as little as 0% (Fig. 3A). The greatest percent improvement was seen in students who scored the lowest on the first draft (Fig. 3B).
Figure 3 Lab report grades reflect high percent improvement after scaffolding. Students submitted a lab report draft (Draft 1), following which I provided feedback based on a rubric and an opportunity for students to revise the draft. Revised student lab reports (Draft 2) were then graded using an identical rubric. (A) Depicted is percent improvement of Draft 2 relative to Draft 1. (B) Data shows percent improvement of Draft 2 relative to Draft 1 plotted against Draft 1 percentage score. Data was collected from three course sections of 60 total students over the course of three semesters.
Discussion
Following assessment by rubric and instructor feedback, lab report grades increased significantly from a failing to a passing grade from the formative draft to the summative report. This data suggests that scaffolding and instructor feedback can help improve students’ scientific writing skills evidenced by better assessment outcomes across lab report iterations. While draft scores significantly improved on average due to scaffolding and instructor feedback, those first drafts which earned a poor grade (i.e. 60 or below) benefitted most compared to those first drafts which earned an acceptable (i.e. 70 to 80) or exceptional score (i.e. 90 or above). This was expected as lower scores have greater opportunity for improvement. Some scores exhibited a negative percent improvement potentially due to a lack of instructor feedback incorporation, lack of submission, or lack of buy-in towards formative assessments (Brazeal and Couch, 2017). Draft score improvement in component I4 suggests that students improved their ability to clearly state a testable hypothesis and support this hypothesis with primary literature. Additionally, improved component M3 scores reflect an increased ability of students to operationally define variables and controls in sufficient and pertinent detail. Students demonstrated an increased ability to summarize the results in written text (R3) and discuss the implications of those results (D1). These findings support the hypothesis that students who practice scientific writing skills through scaffolded assignments earn passing grades. Practice was facilitated by the scaffolding of formative and summative assessments as well as the application of Bloom’s taxonomy to an assessment rubric. Together, these pedagogical practices improved students’ scientific writing skills. However, these teaching strategies were not effective for all students, as some student lab reports exhibited no or negative percent improvement potentially due to lack of buy-in towards scaffolding of summative and formative assessments as mentioned previously. Future studies should address student factors (i.e. beliefs, biases, and behaviors) that have been shown to impact buy-in prior to implementing scaffolded formative and summative assessments (Brazeal and Couch, 2017). It is expected that fewer students would show limited score improvement if they “bought into” formative assessments as a means of improving summative grades.
Rubric
| Possible Points | Points Earned | Comments | |
| Introduction | |||
| The introduction contains an experimental question. Including clear statements of the following: | |||
| I1 a.) Background and significance of the problem, specifically, how does this research connect to a real-world issue? | 1 | ||
| I2 b.) Statement of the question driving the research. | 1 | ||
| I3 The hypothesis is clearly stated and can be easily identified. | 1 | ||
| I4 The hypothesis is justified with the support of in-text citations (APA format). | 2 | ||
| I5 The background information illustrates the need for this experiment without overwhelming the reader with irrelevant detail. | 3 | ||
| I6 The introduction is written in a broad-to-specific manner. | 1 | ||
| The introduction describes in general terms how the hypothesis was tested. | 1 | ||
| Methods | |||
| M1 Sections are written in paragraph form (no numbered lists or bullet points of steps followed). | 1 | ||
| M2 Important techniques are described with appropriate level of detail. Standard tasks not requiring explanation are not included. (e.g., measuring volumes of liquids for concentration and weighing solids, etc.) Is the study repeatable from the information given? | 3 | ||
| M3 Variables are described and operationally defined. Include what controls were used and what variables were manipulated/measured; use scientific terminology to describe methods; | 2 | ||
| M4 This section describes only methods, not background or results (description of methods is a concise narrative of pertinent information). | 2 | ||
| Results | |||
| R1 Results section includes a written text summary of all of the important data collected. | 1 | ||
| R2 Written text summary refers to figures and data, does not interpretate the results, and only states the findings. | 1 | ||
| R3 Written text summarizes the data to an appropriate level of detail (important trends and major results are reported, not every single number collected). | 2 | ||
| R4 Appropriate choice of charts or tables for the type of data collected illustrating important trends in the data. | 2 | ||
| R5 Figures have descriptive titles and captions. | 1 | ||
| R6 Figures contain the following: legend, axis, and scale. Categories are clear and accurate. | 2 | ||
| Discussion | |||
| D1 Results are summarized and include a statement about the results of controls used. Focus on discussing results, not methods. | 2 | ||
| D2 The section compares expected vs. observed results, which are based on the literature [in-text citations (APA format)]. | 3 | ||
| D3 Discuss possible causes for observations, especially any problems, inconsistencies, or biases. Analysis of limitations of the experiment. | 3 | ||
| D4 Conclusion is related back to the original experimental question and hypothesis and follows logically from the data. | 2 | ||
| D5 Discuss future directions/ideas based on these results of this experiment (What should be done differently?). | 2 | ||
| Total Points | 50 | 0 |
References
Bissell, A.N., and Lemons, P.P. (2006). A New Method for Assessing Critical Thinking in the Classroom. BioScience 56, 66-72.
Brazeal, K.R., and Couch, B.A. (2017). Student Buy-In Toward Formative Assessments: The Influence of Student Factors and Importance for Course Success. Journal of Microbiology & Biology Education 18, 10.1128/jmbe.v1118i1121.1235.
Christian, N., and Kearns, K.D. (2018). Using Scaffolding and Deliberate Practice to Improve Abstract Writing in an Introductory Biology Laboratory Course. J Microbiol Biol Educ 19.
Lynn, C.R., Katerina, S., Tutrang, N., Diane, K.O., Dowd, Thurston, D., and Mark, W. (2016). Evaluating Promising Practices in Undergraduate STEM Lecture Courses. RSF: The Russell Sage Foundation Journal of the Social Sciences 2, 212.