This session will chronicle the three-decade evolution of digital learning research at the University of Central Florida. The presenters will describe their approach to quality research, what went well, what did not go well, and how systematic teaching and learning research became an integral part of the UCF culture.

In this 30th anniversary of the Online Learning Consortium, this session will chronicle the similar 30-year journey of the Research Initiative for Teaching Effectiveness (RITE) at the University of Central Florida. RITE began as the research extension of UCF’s online and blended learning initiative in the mid-1990s and it is now integrated into the university culture. We will discuss how it dealt with the evolution of online learning, how it developed customized outcome measures, and how it became the primary support for the scholarship of teaching and learning, seeding research projects across the university. RITE has had considerable success at UCF and across the country and, of course, made numerous blunders (we term those opportunities!) that we will share with others who want to research online learning. RITE’s journey has been remarkable and evolutionary, initiating national trends and responding to the influences and pressures of the outside world. There have been important phases including: Phase 1: In the Beginning Digital learning at UCF began with an off-hand comment at a meeting. Max King, then president of Brevard Community College (BCC) chided John Hitt, UCF President, for not having an educational outreach program. BCC had developed a 2-year online program that provided better access for nontraditional students. John came back from the meeting in 1994 and told us that we were to design a similar program at UCF. Out of that initial planning meeting, the support units for online learning were born, which included a strong focus in faculty development and programmatic planning and a focus on research. One of those units--the Distributed Learning Impact Evaluation--was assigned the responsibility of assessing the initiative’s impact and quality. We were in new and exciting territory and when it became apparent that this movement would be a foundation of UCF, that evaluation became RITE. Phase 2: Trouble in River City Those early years of online learning brought an onslaught of criticisms about the quality of online learning course modalities with epithets like diploma mills, eroding quality, and marginalizing faculty. The first challenge faced by RITE was to compare the impact of course modalities on learning outcomes for students--a daunting task. We will explain how we did it, how it worked, and how our metric design became a national model. Further, we will explain the model’s shortcomings which surfaced over the long haul. Phase 3: Student Satisfaction This phase came from a growing voice of dissatisfaction expressed by students about the quality, engagement, and effectiveness of their online courses. Most were one-off student comments to the provost, questions from parents to the President, or editorial opinions in the student newspaper. “Do they cheat online” became a common concern with many expecting the end of the traditional classroom. Partially in response to these comments, and because our research indicated that many of our online courses were being completed by students in on-campus labs, the university developed multiple course modalities. The most prominent modality was what UCF called mixed-mode. This blended modality offered better student access and classroom efficiencies with reduced seat time while maintaining some face-to-face contact. At UCF this became “the best of both worlds” and our evaluation results today still indicate that these classes offer increased student success over face-to-face alone. We continue the study of student satisfaction and focus on the impact of student voice. Phase 4: The Growth of Blended Learning Since this is an OLC session the participants know that this conference is a direct historical result of support by the Alfred P. Sloan Foundation and in particular OLC founding Board members, Ralph Gomory and Frank Mayadas. Frank was the grounded leader and had a habit of throwing out terms like “asynchronous learning networks” and “localness” and telling us to put definitional wheels on them. Grant money came from Sloan which at the time was not interested in anything but online learning. However, Tony Picciano and Chuck Dziuban began lobbying Frank about the possibilities of blended learning and he finally agreed to a small seminar on the topic led by Mary Niemiec at the University of Illinois, Chicago which continued as a blended learning workshop for years. This started the blended learning movement in the country and over time resulted in RITE, the City University of New York, and Brigham Young University publishing five books on blended learning research. In this session we will explain how RITE evaluates blending learning at UCF and the outcomes we have found. Phase 5: The Scholarship of Teaching and Learning (SoTL)--Bringing Faculty into the Research Community Early on, it became clear that faculty across the university were interested in assessing the impact of their teaching in the digital learning environment and creating a research culture more relevant to their teaching, rather than at the university level. Creative faculty research topics included examining students’ knowledge acquisition, student engagement, the impact of various educational technologies on student success, and dozens of other areas. Supporting this initiative fell to RITE who makes a pitch to faculty. If faculty have a SoTL research project related to improving their teaching we will help: 1. Define your research problem and design an effective study. 2. Develop or find any needed data collection protocols. 3. Find research literature on what others have found related to your research. 4. Maneuver IRB successfully. 5. Collect study data. 6. Analyze the data. 7. Create presentation slides and/or publication manuscripts. This was one of the most successful campus-wide initiatives and has resulted in numerous faculty publications and presentations across many disciplines. It has benefitted faculty tenure and promotion, research awards, and created a culture of grounded research at UCF. Phase 6: Dissemination of our research--The publications and presentations From day one, RITE made every effort to share what we have learned, the mistakes we have made, and our future directions. Through the longevity of our research we have had numerous publications and presentations. Here are prototypical examples with citations illustrating the impact of our work: Blended Learning the New Normal (2018)-1,605 Blended Learning: ECAR Bulletin (2004)--1,421 Blended Learning: A Dangerous Idea? (2013)--930 A Time-Based Model of Blended Learning (2011)--687 Higher Education, Blended Learning, and the Generations: Knowledge is Power No More (2005)—460 Phase 7: Adaptive Learning In 2014, UCF began exploring personalized adaptive learning (PAL) to improve student success, with faculty selecting Realizeit as the university’s enterprise platform. PAL provides individualized content based on students’ performance, allowing students who need more practice to get it while those who excel can move through content more efficiently. This radically impacted the evaluation of student learning outcomes by providing valuable learning analytics data as students progressed through the system. We developed a working research partnership developed with Colm Howlin, chief data scientist for Realizeit and Connie Johnson, Provost at Colorado Technical University. That partnership became one of the most successful adaptive learning collaborative research initiatives and benefited faculty development and students at both universities. Phase 8: The Pandemic COVID caused a complete shift in RITE’s operation because we had to scramble to keep the university open. We had to turn our attention to evaluating the impact of completely remote teaching and learning, and the impact of those instructional changes on all aspects of the educational process. This involved a reassessment of what data and information would be most meaningful in this new normal. The evolution continues. Phase 9: Generative Artificial Intelligence (AI). Today, Large Language Models (LLMs) or Generative AI has swept over higher education. It has shaken the foundation of most aspects of university functioning. Floridi (2023) indicates “Neural networks that were once only theoretically interesting have become common tools in machine learning. The number of digital devices interacting with each is already several times greater than the human population.” RITE is now confronting the impact of this latest phenomenon. About the Session Chuck Dziuban and Patsy Moskal share their research trajectory at UCF providing a model of supporting the teaching and learning culture while discussing the future of digital learning research. We hope to open a dialogue with participants that helps seed similar initiatives at other universities as we share our lessons learned from the past three decades. References Dziuban, C., Hartman, J. & Moskal, P. (March 30, 2004). Blended Learning. ECAR Research Bulletin. Dziuban, C.D., Moskal, P.D. & Hartman, J. (2005). Higher education, blended learning, and the generations: Knowledge is power: No more. In J. Bourne & J. C. Moore (Eds.), Elements of Quality Online Education: Engaging Communities. Needham, MA: Sloan Center for Online Education. Dziuban, C., Graham, C.R., Moskal, P.D., Norberg, A., and Sicilia, N. (2018). Blended learning: the new normal and emerging technologies. International Journal of Educational Technology in Higher Education. Floridi, L. (2023). The Ethics of Artificial Intelligence: Principles, Challenges, & Opportunities. Oxford University Press. Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? Internet and Higher Education, 18, 15-23. Norberg, A., Dziuban, C. Moskal, P. (2011). A time-based blended learning model. On the Horizon, 19(3), 207-216.