Blended learning has demonstrated higher student success rates across many disciplines. This study uses learning analytics captured from pre-recorded lecture videos to investigate the impact of student engagement in an advanced undergraduate blended engineering course. Results revealed positive correlations between lecture video viewership and student grades in the blended course.

INTRODUCTION: Educators in various disciplines, including engineering, have embraced active learning strategies such as flipped classroom instruction, problem-based learning, and small-group collaboration. Active learning is said to improve students’ comprehension while directing instructors away from the “sage on the stage” or instructor-centered approach and toward a student-centered approach (Freeman et al. 2014). Flipped classroom refers to the teaching strategy where students study the concepts by themselves prior to coming to class and the class time is organized around problem-based learning. The flipped classroom is often utilized with blended learning. Blended learning combines face-to-face and online instruction while reducing in-class lecture time (C. Dziuban et al. 2011). Research comparing fully-online, face-to-face, and blended instruction reveals that the blended approach can result in greater student success rates and enhanced student satisfaction when compared to traditional face-to-face instruction ( Dziuban et al. 2018; Dziuban et al. 2011). Specifically, in the STEM fields, studies have shown that blended courses outperform traditional face-to-face courses. Both the blended and flipped approaches often use video lectures as a key instructional tool. Scagnoli et al. define a video lecture as “a video recording of a lecture, conference or presentation by a professor to introduce key concepts and additional information or examples to enhance students’ learning” (Scagnoli, Choo, and Tian 2019). In online or blended classrooms, instructor-generated videos are believed to increase students’ engagement due to at least two reasons. First, the instructor’s videos increase the instructor’s presence. Research has shown that the presence of instructors in online courses increases students’ attention, motivation, and cognition (Baker 2010). Second, video is a superior content delivery tool that facilitates information processing because it includes multimedia (audio and visuals), instead of text alone (Mayer 2009). Another key benefit of video lectures is that they enable self-paced learning as students can control how, how often, when, and where they watch the video. In this study, the authors use learning analytics from the Panopto platform to evaluate the impact of a blended course on student success in an upper-level high-enrollment engineering course. The authors redesigned the upper-level engineering course into the blended format to create a novel instructional format for this department. Learning analytics refers to the data obtained from learning management systems, including video or other software platforms that students interact with while navigating the online course content. These platforms offer detailed data on student engagement and performance as they progress through their coursework, as opposed to students’ perceptions of their course interactions as provided through self-report data. Previous studies in the literature show that learning analytics allows us to observe students’ engagement with online courses and can provide another indicator of their success. These studies have mostly focused on cumulative data with regards to video views, video watch times, or course grades (Garrick 2018; Brozina et al. 2019; Yoon, Lee, and Jo 2021). Utilizing the learning analytics available from the Panopto video-hosting platform in this study, the authors investigate individualized student video viewership across student demographics and correlate it with their course performance. Students’ perceptions and satisfaction of the blended course are gauged through IRB-approved mid-semester and end-of-the-semester surveys. Student success in the blended course is measured by comparing course DFW rates with the face-to-face course counterpart. METHODS: The effect of the blended course on student learning was analyzed using surveys, course grades, and video viewing behavior of students. The key factors in the evaluation were student perception, student engagement, student satisfaction, and student performance. Three Institutional Review Board (IRB) approved surveys were given to the students to assess student attitudes toward the lecture videos and different aspects of the blended course. The surveys were kept anonymous and were administered through Instructure Canvas. Two of these surveys were given out mid-semester after each mid-term exam, followed by a final end-of-semester survey. The survey questions were a combination of Likert-scale-based multiple-choice questions and open-ended questions. The final survey questions were grouped into three categories of demographic and major-related questions, lecture video-specific questions, and blended modality-specific questions. Student engagement data for the blended class was collected using a two-fold approach. Students’ video analytics data was obtained from the Panopto hosting platform. One of the primary reasons for choosing the Panopto platform for hosting lecture videos was its deep integration capability with Canvas, which allowed the researchers to track individual student-video interaction. Data such as the number of video views, total minutes viewed, number of downloads, and device usage were available on Panopto and could be cross-referenced with the Canvas gradebook for the course. Additionally, student engagement was gauged from student self-reported data on lecture video-specific survey questions in the mid-semester and end-of-semester surveys. Student performance in the blended class was monitored through course grades. To analyze the effect of student engagement with the videos on their course performance, video analytics data was downloaded from Panopto and merged with the course gradebook data from Canvas. Student performance in the three major assessments (two mid-terms and the final) was used in the analysis. These assessments were administered through a proctored testing facility maintained by the College of Engineering and Computer Science at the university. The researchers also evaluated the effect of the blended modality on student performance by comparing the cumulative class performance in the blended class with that of the regular face-to-face class in the previous semesters, taught by the same instructor. To maintain uniformity in data collection, only the sections taught by the instructor were chosen for data comparison. Additionally, student withdrawal rates and student failure rates were compared across multiple semesters using student information system (SIS) data maintained by the Institutional Knowledge Management (IKM) unit. SUMMARY OF RESULTS Video analytics data revealed significant differences in video views and downloads amongst students of different ethnicities and status. Additionally, first generation students had significantly higher number of video views and downloads than non-first-generation students. While correlating video analytics data with course performance, the authors observed a strong correlation between video views and exam scores, specifically in the high performing student group. Video analytics data also suggested that the motivation level of high performing students as gauged through views and downloads remained high throughout the semester as opposed to low performing students. Students in the blended course had a positive perception of content learning through lecture videos with greater than 70% of the students reporting increased confidence in getting good grades, improved understanding of key course concepts, and enhanced self-regulated learning capabilities. A minor dissatisfaction amongst the students was the length and the number of videos in the blended course. In terms of student performance, the results of this study were consistent with the literature, with higher success rates and lower failure rates in the blended course as compared to its face-to-face counterpart. FUTURE WORK Future work will focus on tracking student performances in this course and other high enrollment blended courses in engineering through multiple semesters to measure students’ success in their career path. The authors will also continue studying the impact of lecture videos on course performance across different student demographics in other engineering courses. Overall, the study results suggest that blended learning can have positive outcomes on student learning and success in high enrollment engineering courses and lecture video analytics can serve as an effective instructional resource in both monitoring student engagement and predicting student success in blended courses. REFERENCES Freeman, et al, 2014. “Active Learning Increases Student Performance in Science, Engineering, and Mathematics.” Proceedings of the National Academy of Sciences 111 (23): 8410–15. https://doi.org/10.1073/pnas.1319030111. Dziuban, C., Joel Hartman, Thomas B. Cavanagh, and P. Moskal. 2011. “Blended Courses as Drivers of Institutional Transformation.” In . https://doi.org/10.4018/978-1-60960-479-0.CH002. Dziuban, Charles, Charles R. Graham, Patsy D. Moskal, Anders Norberg, and Nicole Sicilia. 2018. “Blended Learning: The New Normal and Emerging Technologies.” International Journal of Educational Technology in Higher Education 15 (1): 3. https://doi.org/10.1186/s41239-017-0087-5. Dziuban, Charles, Joel Hartman, Thomas B. Cavanagh, and Patsy D. Moskal. 2011. “Blended Courses as Drivers of Institutional Transformation.” Chapter. Blended Learning across Disciplines: Models for Implementation. 2011. https://doi.org/10.4018/978-1-60960-479-0.ch002. Scagnoli, Norma I., Jinhee Choo, and Jing Tian. 2019. “Students’ Insights on the Use of Video Lectures in Online Classes.” British Journal of Educational Technology 50 (1): 399–414. https://doi.org/10.1111/bjet.12572. Baker, Credence. 2010. “The Impact of Instructor Immediacy and Presence for Online Student Affective Learning, Cognition, and Motivation.” Journal of Educators Online 7 (January). https://doi.org/10.9743/JEO.2010.1.2. Mayer, Richard E. 2009. Multimedia Learning. 2nd ed. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511811678. Garrick, Rob. 2018. “Flipped Classroom Video Analytics.” In . https://peer.asee.org/flipped-classroom-video-analytics. Brozina, Cory, David B. Knight, Timothy Kinoshita, and Aditya Johri. 2019. “Engaged to Succeed: Understanding First-Year Engineering Students’ Course Engagement and Performance Through Analytics.” IEEE Access 7: 163686–99. Yoon, Meehyun, Jungeun Lee, and Il-Hyun Jo. 2021. “Video Learning Analytics: Investigating Behavioral Patterns and Learner Clusters in Video-Based Online Learning.” The Internet and Higher Education 50 (June): 100806. https://doi.org/10.1016/j.iheduc.2021.100806.