Evaluating Quality over Quantity: A Systematic Review of Discussion Forum Metrics in Online Learning
DOI:
https://doi.org/10.37934/picl.3.1.2141Keywords:
Online learning, online discussion, discussion forum, forum evaluationAbstract
Discussion forums have become an integral component of online learning environments, providing a platform for interaction and exchange of knowledge among learners. However, the number of posts is an inadequate indicator of engagement or educational outcomes, as it often fails to reflect the true depth and relevance of learners’ contributions. Despite their widespread use, the typical evaluation of these forums relies heavily on such quantitative indicators, thereby failing to capture meaningful engagement. Therefore, this systematic literature review examines methods used to evaluate participation in discussion forums over the last 5 years. Utilizing Boolean search techniques, relevant studies from 2020 to 2025 were extracted from the Scopus and Web of Science databases. The review identifies four key analytical approaches: Social Dimension, Cognitive Dimension, Framework-Guided Analysis, and AI-Assisted Analysis. Studies focusing on the social dimension employed Social Network Analysis to map interaction patterns and influential participants. Cognitive dimension studies primarily used content and sequential analyses to assess the depth of critical thinking and knowledge construction. Framework-guided analyses employed established theoretical models such as the Community of Inquiry (CoI) to systematically evaluate learning processes. Finally, AI-assisted analyses leveraged machine learning and deep learning to automate content classification, sentiment analysis, and detection of engagement patterns. Overall, the review suggests that combining traditional analytical methods with AI-driven insights can enhance the depth and accuracy of analysing student learning in online forums. The findings emphasize the need for a shift towards quality-based metrics that prioritize meaningful interactions, thereby guiding educators toward more effective evaluation and fostering deeper cognitive and social engagement in online learning environments.
Downloads
References
[1] KILINC, Hakan and Hakan Altinpulluk. "Discussion Forums as a Learning Material in Higher Education Institutions." In International Journal of Higher Education Pedagogies 2, no. 1 (2021): 1-9. https://doi.org/10.33422/ijhep.v2i1.25
[2] Du, Zhao, Fang Wang, Shan Wang, and Xiao Xiao. "Enhancing Learner Participation in online discussion forums in Massive Open Online Courses: The role of mandatory participation." Frontiers in psychology 13 (2022): 819640. https://doi.org/10.3389/fpsyg.2022.819640
[3] Kumi, Richard. "Discussion forums and student engagement: A social network analysis perspective." Decision Sciences Journal of Innovative Education 21, no. 4 (2023): 197-207. https://doi.org/10.1111/dsji.12299
[4] Farrow, Elaine, Johanna Moore, and Dragan Gašević. "Markers of cognitive quality in student contributions to online course discussion forums." Journal of Learning Analytics 9, no. 2 (2022): 38-65. https://doi.org/10.18608/jla.2022.7250
[5] Kew, Si Na, and Zaidatun Tasir. "Analysing Students' Cognitive Engagement in E-Learning Discussion Forums through Content Analysis." Knowledge Management & E-Learning 13, no. 1 (2021): 39-57. https://doi.org/10.34105/j.kmel.2021.13.003
[6] Liberati, Alessandro, Douglas G. Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C. Gøtzsche, John PA Ioannidis, Mike Clarke, Philip J. Devereaux, Jos Kleijnen, and David Moher. "The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration." Bmj 339 (2009). https://doi.org/10.1371/journal.pmed.1000097
[7] Saqr, Mohammed, Olga Viberg, and Henriikka Vartiainen. "Capturing the participation and social dimensions of computer-supported collaborative learning through social network analysis: which method and measures matter?." International Journal of Computer-Supported Collaborative Learning 15, no. 2 (2020): 227-248. https://doi.org/10.1007/s11412-020-09322-6
[8] Yen, Cherng-Jyh, Chih-Hsiung Tu, Moussa Tankari, Emrah Emre Ozkeskin, Hoda Harati, and Alexandra Miller. "A Predictive Study of Students Social Presence and Their Interconnectivities in the Social Network Interaction of Online Discussion Board." Journal of Applied Learning & Teaching 5, no. 1 (2022). https://doi.org/10.37074/jalt.2022.5.1.13
[9] Tsoni, Rozita, Christos Τ. Panagiotakopoulos, and Vassilios S. Verykios. "Revealing latent traits in the social behavior of distance learning students." Education and Information Technologies 27, no. 3 (2022): 3529-3565. https://doi.org/10.1007/s10639-021-10742-6
[10] Vasodavan, Vinothini, Dorothy DeWitt, Norlidah Alias, and Mariani Md Noh. "E-moderation skills in discussion forums: Patterns of online interactions for knowledge construction." Pertanika Journal of Social Sciences and Humanities 28, no. 4 (2020): 3025-3045. https://doi.org/10.47836/pjssh.28.4.29
[11] Rothstein, Ruth, Yonghee Lee, EDWARD J. Berger, Jeffrey Rhoads, and Jennifer Deboer. "Collaborative engagement and help-seeking behaviors in engineering asynchronous online discussions." International Journal of Engineering Education 39, no. 1 (2023): 189-207.
[12] Chen, Bodong, and Oleksandra Poquet. "Uncovering socio-temporal dynamics in online discussions: An event-based approach." Australasian Journal of Educational Technology 39, no. 6 (2023): 1-16. https://doi.org/10.14742/ajet.8618
[13] Essa, Fatima Vally, Grant Andrews, Belinda Mendelowitz, Yvonne Reed, and Ilse Fouche. "Humanising Online Pedagogy through Asynchronous Discussion Forums: An Analysis of Student Dialogic Interactions at a South African University." Online Learning 27, no. 4 (2023): 508-529. https://doi.org/10.24059/olj.v27i4.3652
[14] Tsoni, Rozita, Evgenia Paxinou, Aris Gkoulalas-Divanis, Dimitrios Karapiperis, Dimitrios Kalles, and Vassilios S. Verykios. "Exploiting Properties of Student Networks to Enhance Learning in Distance Education." Information 15, no. 4 (2024): 234. https://doi.org/10.3390/info15040234
[15] Xing, Wanli, Hai Li, Taehyun Kim, Wangda Zhu, and Yukyeong Song. "Investigating the behaviors of core and periphery students in an asynchronous online discussion community using network analysis and topic modeling." Education and Information Technologies (2024): 1-28. https://doi.org/10.1007/s10639-024-13038-7
[16] Galikyan, Irena, Wilfried Admiraal, and Liesbeth Kester. "MOOC discussion forums: The interplay of the cognitive and the social." Computers & Education 165 (2021): 104133. https://doi.org/10.1016/j.compedu.2021.104133
[17] Alwafi, Enas Mohammad. "Designing an Online Discussion Strategy with Learning Analytics Feedback on the Level of Cognitive Presence and Student Interaction in an Online Learning Community." Online Learning 26, no. 1 (2022): 80-92. https://doi.org/10.24059/olj.v26i1.3065
[18] Lee, Jeonghyun, Farahnaz Soleimani, John Hosmer IV, Meryem Yilmaz Soylu, Roy Finkelberg, and Saurabh Chatterjee. "Predicting Cognitive Presence in At-Scale Online Learning: MOOC and For-Credit Online Course Environments." Online Learning 26, no. 1 (2022): 58-79. https://doi.org/10.24059/olj.v26i1.3060
[19] Husni, Nur Amin, Nurul Jumaat, and Zaidatun Tasir. "Investigating student’s cognitive engagement, motivation and cognitive retention in learning management system." International Journal of Emerging Technologies in Learning (iJET) 17, no. 9 (2022): 184-200. https://doi.org/10.3991/ijet.v17i09.29727
[20] Lijun, Cheng, and Masami Yoshida. "Sequential Analysis of Online Scholarly Discussions of University Students using a Cognitive Subset of Critical Thinking." Journal of Educational and Social Research 13 (2023). https://doi.org/10.36941/jesr-2023-0053
[21] Prestridge, Sarah, and Deniese Cox. "Play like a team in teams: A typology of online cognitive-social learning engagement." Active Learning in Higher Education 24, no. 1 (2023): 3-20. https://doi.org/10.1177/1469787421990986
[22] O'Riordan, Tim, David E. Millard, and John Schulz. "Is critical thinking happening? Testing content analysis schemes applied to MOOC discussion forums." Computer Applications in Engineering Education 29, no. 4 (2021): 690-709. https://doi.org/10.1002/cae.22314
[23] Chi, Michelene TH, and Ruth Wylie. "The ICAP framework: Linking cognitive engagement to active learning outcomes." Educational psychologist 49, no. 4 (2014): 219-243. https://doi.org/10.1080/00461520.2014.965823
[24] Gunawardena, Charlotte N., Constance A. Lowe, and Terry Anderson. "Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing." Journal of educational computing research 17, no. 4 (1997): 397-431. https://doi.org/10.2190/7MQV-X9UJ-C7Q3-NRAG
[25] Garrison, D. Randy, Terry Anderson, and Walter Archer. "Critical inquiry in a text-based environment: Computer conferencing in higher education." The internet and higher education 2, no. 2-3 (1999): 87-105. https://doi.org/10.1016/S1096-7516(00)00016-6
[26] Garrison, D. Randy, Terry Anderson, and Walter Archer. "Critical thinking, cognitive presence, and computer conferencing in distance education." American Journal of distance education 15, no. 1 (2001): 7-23. https://doi.org/10.1080/08923640109527071
[27] Raković, Mladen, Zahia Marzouk, Amna Liaqat, Philip H. Winne, and John C. Nesbit. "Fine grained analysis of students’ online discussion posts." Computers & Education 157 (2020): 103982. https://doi.org/10.1016/j.compedu.2020.103982
[28] Sezgin, Sezan. "Cognitive relations in online learning: Change of cognitive presence and participation in online discussions based on cognitive style." Participatory Educational Research 8, no. 1 (2021): 344-361. https://doi.org/10.17275/per.21.20.8.1
[29] Gillingham, Kelly, Kyle Eggleton, and Felicity Goodyear-Smith. "Is reflective learning visible in online discussion forums for medical students on general practice placements? A qualitative study." Teaching and learning in medicine 32, no. 4 (2020): 434-441. https://doi.org/10.1080/10401334.2020.1730184
[30] Kim, Min Kyu, Yinying Wang, and Tuba Ketenci. "Who are online learning leaders? Piloting a leader identification method (LIM)." Computers in Human Behavior 105 (2020): 106205. https://doi.org/10.1016/j.chb.2019.106205
[31] Kim, Min Kyu, In Heok Lee, and Yinying Wang. "How students emerge as learning leaders in small group online discussions." Journal of Computer Assisted Learning 36, no. 5 (2020): 610-624. https://doi.org/10.1111/JCAL.12431/v1/review2
[32] Eryilmaz, Evren, Brian Thoms, and Zafor Ahmed. "Cluster analysis in online learning communities: A text mining approach." Communications of the Association for Information Systems 51, no. 1 (2022): 41. https://doi.org/10.17705/1CAIS.05132
[33] Floriasti, Tri Wahyuni. "Analysis of Online Discussion Forum in Constructing English Language Skills through Collaborative Learning of Preservice Teachers in Indonesia." Mextesol Journal 47, no. 1 (2023): n1. https://doi.org/10.61871/mj.v47n1-4
[34] Ba, Shen, Xiao Hu, David Stein, and Qingtang Liu. "Anatomizing online collaborative inquiry using directional epistemic network analysis and trajectory tracking." British Journal of Educational Technology 55, no. 5 (2024): 2173-2191. https://doi.org/10.1111/bjet.13441
[35] deNoyelles, Aimee, Janet Zydney, and Jacqueline Roberts. "Photo-Based Protocols to Support Community of Inquiry in Online Discussions." TechTrends 68, no. 2 (2024): 307-316. https://doi.org/10.1007/s11528-023-00929-8
[36] Bunsu, Christnalter, and Noor Dayana Abd Halim. "A review of trends and applications of learning analytics in higher education in the post-pandemic era." Innovative Teaching and Learning Journal 7, no. 2 (2023): 19-24. https://doi.org/10.11113/itlj.v7.131
[37] Onan, Aytuğ, and Mansur Alp Toçoğlu. "Weighted word embeddings and clustering‐based identification of question topics in MOOC discussion forum posts." Computer Applications in Engineering Education 29, no. 4 (2021): 675-689. https://doi.org/10.1002/cae.22252
[38] Zou, Wenting, Xiao Hu, Zilong Pan, Chenglu Li, Ying Cai, and Min Liu. "Exploring the relationship between social presence and learners’ prestige in MOOC discussion forums using automated content analysis and social network analysis." Computers in Human Behavior 115 (2021): 106582. https://doi.org/10.1016/j.chb.2020.106582
[39] Hu, Yuanyuan, Rafael Ferreira Mello, and Dragan Gašević. "Automatic analysis of cognitive presence in online discussions: An approach using deep learning and explainable artificial intelligence." Computers and Education: Artificial Intelligence 2 (2021): 100037. https://doi.org/10.1016/j.caeai.2021.100037
[40] Chanaa, Abdessamad. "Sentiment analysis on massive open online courses (MOOCs): multi-factor analysis, and machine learning approach." International Journal of Information and Communication Technology Education (IJICTE) 18, no. 1 (2022): 1-22. https://doi.org/10.4018/IJICTE.310004
[41] Yee, Michael, Anindya Roy, Meghan Perdue, Consuelo Cuevas, Keegan Quigley, Ana Bell, Ahaan Rungta, and Shigeru Miyagawa. "AI-assisted analysis of content, structure, and sentiment in MOOC discussion forums." In Frontiers in Education, vol. 8, p. 1250846. Frontiers Media SA, 2023. https://doi.org/10.3389/feduc.2023.1250846
[42] Alsuhaimi, Rahaf, and Omaima Almatrafi. "Identifying Learners’ Confusion in a MOOC Forum Across Domains Using Explainable Deep Transfer Learning." Information 15, no. 11 (2024): 681. https://doi.org/10.3390/info15110681
[43] Žitnik, Slavko, and Glenn Gordon Smith. "Automated analysis of postings in fourth grade online discussions to help teachers keep students on-track." Interactive Learning Environments 32, no. 8 (2024): 4587-4612. https://doi.org/10.1080/10494820.2023.2204327
[44] Wei, Xin, Yajun Chen, Jianhua Shen, and Liang Zhou. "Fail or pass? Investigating learning experiences and interactive roles in MOOC discussion board." Computers & Education 217 (2024): 105073. https://doi.org/10.1016/j.compedu.2024.105073
[45] Castellanos-Reyes, Daniela, Larisa Olesova, and Ayesha Sadaf. "Transforming online learning research: Leveraging GPT large language models for automated content analysis of cognitive presence." The Internet and Higher Education (2025): 101001. https://doi.org/10.1016/j.iheduc.2025.101001
[46] Jain, Keerthi, and J. Naga Venkata Raghuram. "Unlocking potential: The impact of AI on education technology." Multidisciplinary Reviews 7, no. 3 (2024): 2024049-2024049. https://doi.org/10.31893/multirev.2024049
