This project aims to apply machine learning models that can fuse multimodal features extracted from audio, video, and transcripts for reliable engagement prediction in video-based online learning scenarios. Furthermore, based on the engagement predictions, we aim to conduct a systematic and comprehensive investigation into the engagement patterns in online learning through a data-driven manner. Specifically, we are interested in questions such as how the engagement of the whole class evolves through time, how students interact (e.g., ask/answer questions) with each other or teachers, what are the indicators/behaviors for low/high engagement, and what are the contexts for such moments.

Students will be given access to real-world labeled video data in an online learning scenario provided by an international communication training company. Students need to experiment with existing machine learning models for multimodal engagement prediction.

They are encouraged to build simple demos to investigate the engagement patterns reflected by the model predictions.