Standard 8: Connecting course statistics

Providing students with opportunities to assess their learning is a key strategy in online education. Not only does it promote a personalized learning environment, but it can also be useful for enhancing course design and delivery. From the student’s perspective, they are given the opportunity to test prior knowledge on topics to be presented in a particular module of instruction. This allows them to identify areas where they need to focus their efforts and areas where they have prior knowledge. For the instructor, class data on student performance can help them adjust their instruction and delivery based on the overall performance of all students. This allows them to focus on areas where students are weaker while minimizing emphases in areas of strength.

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”
• Formative Assessment
• iCollege Now – Creating a Quiz
• iCollege Now – Grading a Quiz
• Creating an Interactive Video Quiz in Kaltura
• iCollege Now – Grading with QuickEval

Examples

I have examples of this from my CHEM 1212 Master course build.

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #4 – “To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned” (p. 5).
• Principle #5  – “Goal-directed practice, coupled with targeted feedback enhances the quality of students’ learning” (p. 5).
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).
• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience

https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/

• Insight #4 – Retrieval practice strengthens memory and helps students flexibly apply what they learn. Retrieval activities, like self-testing and low-stakes quizzing, that ask students to practice remembering the information they’ve been taught by retrieving it from their long-term memory actually change the nature of memory by strengthening the path to memory and enriching the memory itself. In this way, retrieval practice leads to stronger and more enduring learning.
• Insight #4a – Encourage students to use self-tests to assess what knowledge is not easily retrieved and to flag for more retrieval practice. Flashcards can be a useful self-testing tool.
• Insight #4c – Make quizzes low-stakes, predictable (not pop!), simple, and quick. Having students generate questions for quizzes can be doubly beneficial!

One of the best strategies for developing a student-centered or student-focused classroom is to provide students with a voice and to listen to that voice. In online courses, this can be accomplished through surveys integrated into each module of instruction. Students are asked to complete these surveys as part of their module learning activities. Surveys can be structured in such a way as to be anonymous (which promotes engagement) and provide valuable data to the instructor, which can help inform course design and facilitation revisions. 

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”
• Formative Assessment
• iCollege Now – Creating a Quiz
• iCollege Now – Grading a Quiz
• Creating an Interactive Video Quiz in Kaltura
• iCollege Now – Grading with QuickEval

Examples

All of the BBA courses have surveys built into them that could be used as examples of this.

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #4 – “To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned” (p. 5).
• Principle #5  – “Goal-directed practice, coupled with targeted feedback enhances the quality of students’ learning” (p. 5).
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).
• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience

https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/

• Insight #4 – Retrieval practice strengthens memory and helps students flexibly apply what they learn. Retrieval activities, like self-testing and low-stakes quizzing, that ask students to practice remembering the information they’ve been taught by retrieving it from their long-term memory actually change the nature of memory by strengthening the path to memory and enriching the memory itself. In this way, retrieval practice leads to stronger and more enduring learning.
• Insight #4a – Encourage students to use self-tests to assess what knowledge is not easily retrieved and to flag for more retrieval practice. Flashcards can be a useful self-testing tool.
• Insight #4c – Make quizzes low-stakes, predictable (not pop!), simple, and quick. Having students generate questions for quizzes can be doubly beneficial!

Student-instructor interaction is one of the best ways to minimize transactional distance in online courses (Stavredes, 2011). Through the use of automated communication channels such as intelligent agents, faculty can customize messages that are then delivered to students based on those students meeting a set of predetermined conditions. What is so nice about these automated messages is that they can be encouraging as well as critical and can provide students with just-in-time remediation techniques, all while streamlining the communication process for faculty.

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”

Examples

There are plenty of IA examples in the BBA program and in my BUSA 1105 course.

Research

• Cleveland-Innes, M., & Campbell, P. (2012). Emotional presence, learning, and the online environment. The International Review of Research in Open and Distributed Learning, 13(4), 269-292.
• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/
• Insight #1 – Learning is a process that involves effort, mistakes, reflection, and refinement of strategies. Learning is a continual process that leads to the development of new knowledge as well as changes in existing knowledge. Helping students understand that many aspects of learning, including strategy and effort, are under their control fosters students’ beliefs in their own agency to learn. When students see failure as an opportunity to find out what they do not know (and adjust their learning strategies accordingly), rather than as an indication of self-worth, they are more likely to persevere.
• Insight #1a – Provide feedback that focuses on the process and helps students see both the productive effort and the effective strategies they used. Encourage students to employ a growth mindset by reminding them of their progress while supporting them to work through challenges.
• Insight #1b – Foster an environment where mistakes and failure are fodder for reflection and positive discussion, so students feel supported and safe to learn from mistakes. Use discussion questions to reframe failure as fodder: Have you ever felt proud of making a mistake? Have you ever discovered something new after making a mistake?
• Insight #1c – Frame assessments as opportunities for students to show themselves what they know and can do at that given moment versus a diagnostic that labels them.
• Insight #3b – Use language like “I believe you can do this” to communicate high expectations and encourage students to persist through challenges. This language also reassures them that this is a desirable difficulty and that real learning is happening!
• Insight #9 – Students’ physical well-being, including nutrition, sleep, and exercise, impacts learning. Learning will be impaired if students’ basic physiological needs are not met. Students with good overall physical well-being have better cognitive skills than when they are in poor condition. Aerobic exercise can transiently improve the brain’s plasticity and can increase hippocampal volume (a key part of the brain involved in learning new information). Sleep is critical for solidifying learning from the day and is one of the most important (and easiest) ways to strengthen learning. Basic nutrition is also important for brain health.
• Insight #10 – The entire environment, from space to temperature to lighting, can affect learning. Elements of the physical environment can play a role in determining whether the classroom will be conducive for focus and learning. Exposure to sunlight, as well as views of nature from the classroom, has been shown to boost student achievement, well-being, and behavior.

Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. Jossey-Bass.

When building a course gradebook, it is important to ensure that the items in the gradebook match the assessments and assignments in the course. Additionally, it may be beneficial to categorize assessment types based on how they are categorized in the course. Alternatively, the gradebook could be structured chronologically so that students can see grade items sequentially in the course. By structuring your gradebook in this fashion, you will more easily be able to see student performance data and accurately review student progress. 

Resources

• Quality Matters 3.2 – “The course grading policy is stated clearly at the beginning of the course.”
• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”

Examples

ENGL 1102 Master has a well-structured gradebook.

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).

Unlike the face-to-face classroom, the online classroom requires a significant amount of planning and scheduling. Because of their asynchronous nature, effective online courses have complete calendars with all due dates and upcoming items for the entire semester. Even when assessments are located in time-restricted modules, the assessments themselves should be listed in the course schedule so that students can plan their study efforts and prioritize their time accordingly.

Resources

• Quality Matters 3.4 – “ The assessments used are sequenced, varied, and suited to the level of the course”
• Quality Matters 5.4 – “The requirements for learner interaction are clearly stated.”

Examples

The online BBA courses should have complete course schedules built into them.

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).

Providing students with opportunities to assess their learning is a key strategy in online education. Not only does it promote a personalized learning environment, but it can also be useful for enhancing course design and delivery. From the student’s perspective, they are given the opportunity to test prior knowledge on topics to be presented in a particular module of instruction. This allows them to identify areas where they need to focus their efforts and areas where they have prior knowledge. For the instructor, class data on student performance can help them adjust their instruction and delivery based on the overall performance of all students. This allows them to focus on areas where students are weaker while minimizing emphases in areas of strength.

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”
• Formative Assessment
• iCollege Now – Creating a Quiz
• iCollege Now – Grading a Quiz
• Creating an Interactive Video Quiz in Kaltura
• iCollege Now – Grading with QuickEval

Examples

I have examples of this from my CHEM 1212 Master course build.

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #4 – “To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned” (p. 5).
• Principle #5  – “Goal-directed practice, coupled with targeted feedback enhances the quality of students’ learning” (p. 5).
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).
• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience

https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/

• Insight #4 – Retrieval practice strengthens memory and helps students flexibly apply what they learn. Retrieval activities, like self-testing and low-stakes quizzing, that ask students to practice remembering the information they’ve been taught by retrieving it from their long-term memory actually change the nature of memory by strengthening the path to memory and enriching the memory itself. In this way, retrieval practice leads to stronger and more enduring learning.
• Insight #4a – Encourage students to use self-tests to assess what knowledge is not easily retrieved and to flag for more retrieval practice. Flashcards can be a useful self-testing tool.
• Insight #4c – Make quizzes low-stakes, predictable (not pop!), simple, and quick. Having students generate questions for quizzes can be doubly beneficial!

When considering assessments and assessment strategies for a course, as an instructor you need to answer the question, “why am I having students complete this assessment?” Typically, you want students to be able to do something in a course. That something is usually outlined in the Course Goals and Student Learning Outcomes. Essentially, you want your students to demonstrate mastery of the concepts stated through those Goals and Outcomes. It is somewhat unreasonable to think that every student can demonstrate that mastery in the same way on the same assessment at the same time. Allowing students an opportunity to iterate on their work is what develops mastery.

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”
• Formative Assessment
• iCollege Now – Creating a Quiz
• iCollege Now – Grading a Quiz
• Creating an Interactive Video Quiz in Kaltura
• iCollege Now – Grading with QuickEval

Research

• Ambrose, A. A., Bridges, M. W., DiPietro, M., Lovett, M. C., Norman, M. K. (2010). How learning works: Seven research-based principles for smart teaching. Jossey-Bass.
• Principle #4 – “To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned” (p. 5).
• Principle #5  – “Goal-directed practice, coupled with targeted feedback enhances the quality of students’ learning” (p. 5).
• Principle #7 – “To become self-directed learners, students must learn to monitor and adjust their approaches to learning” (p. 6).
• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/
• Insight #1 – Learning is a process that involves effort, mistakes, reflection, and refinement of strategies. Learning is a continual process that leads to the development of new knowledge as well as changes in existing knowledge. Helping students understand that many aspects of learning, including strategy and effort, are under their control fosters students’ beliefs in their own agency to learn. When students see failure as an opportunity to find out what they do not know (and adjust their learning strategies accordingly), rather than as an indication of self-worth, they are more likely to persevere.
• Insight #1b – Foster an environment where mistakes and failure are fodder for reflection and positive discussion, so students feel supported and safe to learn from mistakes. Use discussion questions to reframe failure as fodder: Have you ever felt proud of making a mistake? Have you ever discovered something new after making a mistake?
• Insight #6b – Invite students to generate purposeful, self-focused goals such as, “gain skills I can use in a job to help others,” and learn the material to “become an educated citizen that can contribute to society.” Doing this encourages students to develop an internal drive for learning and find meaning in mundane schoolwork.

Life happens. It happens to us as faculty and it happens to our students. As faculty, we should be there to motivate our students, not demotivate them. Given that so many students are extrinsically motivated through grades and class performance, we should provide them with opportunities to earn those grades. One way to do this is through a student-focused approach where due dates and late work is accepted and feedback is provided. So long as other course policies (such as student communication) are adhered to, then students should be afforded every possible opportunity to demonstrate their mastery of the course content and be provided with valuable feedback, even if it comes with penalty.

Resources

• Complete College Georgia > Mindset Supportive Practices
• Developing a grown Mindset with Carol Dweck

Research

• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/
• Insight #5c – Encourage your students to break up their studying of a topic into multiple days or weeks, studying the information more than once and allowing a good amount of time in between study sessions.
• Insight #9 – Students’ physical well-being, including nutrition, sleep, and exercise, impacts learning. Learning will be impaired if students’ basic physiological needs are not met. Students with good overall physical well-being have better cognitive skills than when they are in poor condition. Aerobic exercise can transiently improve the brain’s plasticity and can increase hippocampal volume (a key part of the brain involved in learning new information). Sleep is critical for solidifying learning from the day and is one of the most important (and easiest) ways to strengthen learning. Basic nutrition is also important for brain health.
• Insight #10 – The entire environment, from space to temperature to lighting, can affect learning. Elements of the physical environment can play a role in determining whether the classroom will be conducive for focus and learning. Exposure to sunlight, as well as views of nature from the classroom, has been shown to boost student achievement, well-being, and behavior.

In face-to-face courses assessment feedback is easily provided through face-to-face assessment review. In asynchronous online courses this process is more challenging. Oftentimes, faculty rely on the LMS or other technology to provide feedback for them. However, the simplified “Sorry, wrong” or “Correct. Good job!” is not substantial enough to be effective in the online environment. Instead, faculty should build substantive feedback into assessment tools that can provide that feedback in a timely manner. All students need timely feedback to remediate and close gaps in the learning process. This is even more true for those online, asynchronous students that typically endure several assessments before they receive feedback, by which time the knowledge gap is too wide to bridge.

Resources

• Quality Matters 3.5 – “The course provides learners with multiple opportunities to track their learning progress with timely feedback.”
• OSCQR #45 – “Course includes frequent and appropriate methods to assess learners’ mastery of content.”
• OSCQR #47 – “Learners have opportunities to review their performance and assess their own learning throughout the course (pre-tests, automated self-tests, reflective assignments, etc.).”
• Formative Assessment
• iCollege Now – Creating a Quiz
• iCollege Now – Grading a Quiz
• Creating an Interactive Video Quiz in Kaltura
• iCollege Now – Grading with QuickEval

 Examples

Examples are built into the quiz features in the BBA courses.

Research

• Learning Sciences Insights from Digital Promise and the Institute for Applied Neuroscience https://researchmap.digitalpromise.org/topics/introduction-learning-sciences/
• Insight #1 – Learning is a process that involves effort, mistakes, reflection, and refinement of strategies. Learning is a continual process that leads to the development of new knowledge as well as changes in existing knowledge. Helping students understand that many aspects of learning, including strategy and effort, are under their control fosters students’ beliefs in their own agency to learn. When students see failure as an opportunity to find out what they do not know (and adjust their learning strategies accordingly), rather than as an indication of self-worth, they are more likely to persevere.
• Insight #1a – Provide feedback that focuses on the process and helps students see both the productive effort and the effective strategies they used. Encourage students to employ a growth mindset by reminding them of their progress while supporting them to work through challenges.
• Insight #1b – Foster an environment where mistakes and failure are fodder for reflection and positive discussion, so students feel supported and safe to learn from mistakes. Use discussion questions to reframe failure as fodder: Have you ever felt proud of making a mistake? Have you ever discovered something new after making a mistake?
• Insight #1c – Frame assessments as opportunities for students to show themselves what they know and can do at that given moment versus a diagnostic that labels them.
• Insight #3b – Use language like “I believe you can do this” to communicate high expectations and encourage students to persist through challenges. This language also reassures them that this is a desirable difficulty and that real learning is happening!
• Insight #9 – Students’ physical well-being, including nutrition, sleep, and exercise, impacts learning. Learning will be impaired if students’ basic physiological needs are not met. Students with good overall physical well-being have better cognitive skills than when they are in poor condition. Aerobic exercise can transiently improve the brain’s plasticity and can increase hippocampal volume (a key part of the brain involved in learning new information). Sleep is critical for solidifying learning from the day and is one of the most important (and easiest) ways to strengthen learning. Basic nutrition is also important for brain health.
• Insight #10 – The entire environment, from space to temperature to lighting, can affect learning. Elements of the physical environment can play a role in determining whether the classroom will be conducive for focus and learning. Exposure to sunlight, as well as views of nature from the classroom, has been shown to boost student achievement, well-being, and behavior.

Stavredes, T. (2011). Effective online teaching: Foundations and strategies for student success. Jossey-Bass.