Usability in learning management systems: A mixed-method analysis

##plugins.themes.academic_pro.article.sidebar##

Published Jun 24, 2025
https://doi.org/10.56158/jpte.2025.120.4.01
Download
FULL ARTICLE - PDF
Statistic
Vol. 4 No. 01 (2025): June

##plugins.themes.academic_pro.article.main##

Pembe Pelin Koca
Department of Computer Education and Instructional Technology, Graduate School of Natural and Applied Sciences, Amasya University, Amasya, Turkey
https://orcid.org/0000-0002-0058-9578
Hakan Özcan
Department of Computer Technology, Design Vocational School, Amasya University, Amasya, Turkey
https://orcid.org/0000-0002-4352-8716

Abstract





Usability analysis in software has a critical role in creating useful products for users and obtaining the necessary feedback. Learning Management Systems (LMSs) are among the most frequently used software in distance education. The usability of these systems is crucial for both student success and the quality of the service offered. This study examines LMS usability through faculty perceptions at a state university, employing a mixed-methods design. Quantitative data were collected using the System Usability Scale (SUS) from 109 faculty members, while qualitative insights were gathered from semi-structured interviews with nine participants. Statistical and thematic content analyses were employed to interpret and compare results. The quantitative analysis yielded an average SUS score of 63.85 ± 16, indicating moderate usability concerns. Based on findings from both the SUS responses and interview data, several recommendations were proposed, such as enhancing system infrastructure, simplifying the interface, improving instructional guidance, strengthening interaction features, optimizing file management, refining notification systems, addressing character encoding issues, and streamlining listing and reporting functions. The results underscore the importance of a user-centered development approach, incorporating participatory design principles. Future research should track how faculty adapt to LMS updates over time. We hope these findings will guide future usability studies.





##plugins.themes.academic_pro.article.details##

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

How to Cite
Koca, P. P., & Özcan, H. (2025). Usability in learning management systems: A mixed-method analysis. International Journal of Pioneering Technology and Engineering, 4(01), 17–25. https://doi.org/10.56158/jpte.2025.120.4.01

References

  1. Şahi̇n Kölemen, C. (2023). The effect of distance education on lifelong learning. İnsan ve Toplum Bilimleri Araştırmaları Dergisi, 12(3), 1396–1411.
  2. Özbay, Ö. (2015). Dünyada ve Türkiye’de uzaktan eğitimin güncel durumu. The Journal of International Educational Sciences, 2(5), 376–376.
  3. Cavus, N. (2015). Distance Learning and Learning Management Systems. Procedia - Social and Behavioral Sciences, 191, 872–877.
  4. Pokhrel, S., & Chhetri, R. (2021). A literature review on Impact of COVID-19 pandemic on teaching and learning. Higher Education for the Future, 8(1), 133–141.
  5. Turnbull, D., Chugh, R., & Luck, J. (2020). Learning management systems, an overview. In A. Tatnall (Ed.), Encyclopedia of Education and Information Technologies (pp. 1052–1058). Cham: Springer International Publishing.
  6. Dobre, I. (2015). Learning Management Systems for Higher Education - An Overview of Available Options for Higher Education Organizations. Procedia - Social and Behavioral Sciences, 180, 313–320.
  7. Turnbull, D., Chugh, R., & Luck, J. (2022). An overview of the common elements of learning management system policies in higher education institutions. TechTrends, 66(5), 855–867.
  8. Bradley, V. M. (2020). Learning Management System (LMS) Use with Online Instruction. International Journal of Technology in Education, 4(1), 68.
  9. Dias, S. B., & Diniz, J. A. (2014). Towards an Enhanced Learning Management System for Blended Learning in Higher Education Incorporating Distinct Learners’ Profiles. Journal of Educational Technology & Society, 17(1), 307–319. Retrieved from https://www.jstor.org/stable/jeductechsoci.17.1.307
  10. Tempelaar, D., Rienties, B., & Nguyen, Q. (2021). Dispositional Learning Analytics for Supporting Individualized Learning Feedback. Frontiers in Education, 6, 703773.
  11. Alghabban, W. G., & Hendley, R. (2022). Perceived level of usability as an evaluation metric in adaptive e-learning: a case study with dyslexic children. SN Computer Science, 3(3), 238.
  12. Alonso-Ríos, D., Vázquez-García, A., Mosqueira-Rey, E., & Moret-Bonillo, V. (2009). Usability: A Critical Analysis and a Taxonomy. International Journal of Human-Computer Interaction, 26(1), 53–74.
  13. Nielsen, J. (1992). The usability engineering life cycle. Computer, 25(3), 12–22.
  14. Stockless, A. (2018). Acceptance of learning management system: The case of secondary school teachers. Education and Information Technologies, 23(3), 1101–1121.
  15. Butler, K. A. (1996). Usability engineering turns 10. Interactions, 3(1), 58–75.
  16. Kruachottikul, P., Dumrongvute, P., Tea-makorn, P., Kittikowit, S., & Amrapala, A. (2023). New product development process and case studies for deep-tech academic research to commercialization. Journal of Innovation and Entrepreneurship, 12(1), 48.
  17. Van Nuland, S. E., Eagleson, R., & Rogers, K. A. (2017). Educational software usability: artifact or design? Anatomical Sciences Education, 10(2), 190–199.
  18. Özcan, H., & Emiroğlu, B. G. (2020). Bulut tabanlı öğrenme yönetim sistemi seçiminde bulanık çok kriterli karar analizi yaklaşımı. Bilişim Teknolojileri Dergisi, 97–111.
  19. Alshammari, M., Anane, R., & Hendley, R. J. (2015). Design and Usability Evaluation of Adaptive e-learning Systems Based on Learner Knowledge and Learning Style. In J. Abascal, S. Barbosa, M. Fetter, T. Gross, P. Palanque, & M. Winckler (Eds.), Human-Computer Interaction – INTERACT 2015 (Vol. 9297, pp. 584–591). Cham: Springer International Publishing.
  20. Gunesekera, A. I., Bao, Y., & Kibelloh, M. (2019). The role of usability on e-learning user interactions and satisfaction: a literature review. Journal of Systems and Information Technology, 21(3), 368–394.
  21. Chu, A., Biancarelli, D., Drainoni, M.-L., Liu, J., Schneider, J., Sullivan, R., & Sheng, A. (2019). Usability of Learning Moment: Features of an E-learning Tool That Maximize Adoption by Students. Western Journal of Emergency Medicine, 21(1), 78–84.
  22. Mtebe, J. S., & Kissaka, M. M. (2015). Heuristics for evaluating usability of Learning Management Systems in Africa. In 2015 IST-Africa Conference (pp. 1–13). Presented at the 2015 IST-Africa Conference, Lilongwe, Malawi: IEEE.
  23. Palve, S. S., & Palve, S. B. (2023). Attitude and perceptions of the faculty toward use of LMS in a tertiary medical college: An interventional study. Journal of Education and Health Promotion, 12(1).
  24. De Kock, E., Van Biljon, J., & Botha, A. (2016). User Experience of Academic Staff in the Use of a Learning Management System Tool. In Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists (pp. 1–10). Presented at the SAICSIT ’16: Annual Conference of the South African Institute of Computer Scientists and Information Technologists, Johannesburg South Africa: ACM.
  25. Bousbahi, F., & Alrazgan, M. S. (2015). Investigating IT Faculty Resistance to Learning Management System Adoption Using Latent Variables in an Acceptance Technology Model. The Scientific World Journal, 2015(1), 375651.
  26. Lehong, S., Van Biljon, J., & Sanders, I. (2024). Usability Requirements for Learning Management Systems in Open Distance Electronic Learning Environments: Considering Lecturers’ Views on Students’ Needs. International Journal of Human–Computer Interaction, 40(3), 567–583.
  27. Brooke, J. (1996). SUS: A “Quick and Dirty” Usability Scale. In P. W. Jordan, B. Thomas, I. L. McClelland, & B. Weerdmeester (Eds.), Usability Evaluation In Industry (1st ed., pp. 207–212). CRC Press.
  28. Demirkol, D., & Şeneler, Ç. (2018). A Turkish translation of the system usability scale: the SUS-TR. Uşak Üniversitesi Sosyal Bilimler Dergisi, 11(3), 237–253. Retrieved from https://dergipark.org.tr/tr/pub/usaksosbil/issue/40289/480920
  29. Farrokhi, F., & Mahmoudi-Hamidabad, A. (2012). Rethinking Convenience Sampling: Defining Quality Criteria. Theory and Practice in Language Studies, 2(4), 784–792.
  30. Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An Empirical Evaluation of the System Usability Scale. International Journal of Human-Computer Interaction, 24(6), 574–594.
  31. Lewis, J. R., & Sauro, J. (2018). Item benchmarks for the system usability scale. Journal of Usability studies, 13(3), 158–167. Retrieved from http://uxpajournal.org/wp-content/uploads/sites/7/pdf/JUS_Lewis_May2018.pdf
  32. Alturki, U. T., Aldraiweesh, A., & Kinshuck. (2016). Evaluating the usability and accessibility of LMS “Blackboard” at King Saud University. Contemporary Issues in Education Research, 9(1), 33–44. Retrieved from https://eric.ed.gov/?id=EJ1087602
  33. Binyamin, S., Rutter, M., & Smith, S. (2016). The utilization of system usability scale in learning management systems: A case study of Jeddah Community College (pp. 5314–5323). Presented at the International Technology, Education and Development Conference, Seville, Spain.
  34. Kurata, Y. (2018). Usability evaluation of an educational management system affecting the user – satisfaction among tertiary students in an undergraduate engineering program in the Philippines (Vol. 8). Presented at the 8th Annual International Conference on Industrial Engineering and Operations Management, IEOM Society.
  35. Medina-Flores, R., & Morales-Gamboa, R. (2015). Usability evaluation by experts of a learning management system. IEEE Revista Iberoamericana de Tecnologias del Aprendizaje, 10(4), 197–203. Presented at the IEEE Revista Iberoamericana de Tecnologias del Aprendizaje.
  36. Ateş, V., & Güyer, T. (2016). Bir öğrenme yönetim sisteminin öğretim elemanları tarafından değerlendirilmesi: Gazi Üniversitesi örneği. Bilişim Teknolojileri Dergisi, 9(1), 1–12.
  37. Özonur, M., Kamışlı, H., Yanpar Yelken, T., & Sancar Tokmak, H. (2019). Uzaktan eğitim öğrencilerinin Enocta öğrenme yönetim sistemi hakkında görüşlerinin incelenmesi. Mehmet Akif Ersoy Üniversitesi Eğitim Fakültesi Dergisi, (50), 283–302.
  38. Turan, O. S., & Canal, M. R. (2012). Öğrenme yönetim sistemi kullanılabilirlik incelemesi; Gazi İngilizce Dil Okulu örneği. Bilişim Teknolojileri Dergisi, 4(3), 47–52. Retrieved from https://dergipark.org.tr/tr/pub/gazibtd/issue/6623/87940
  39. Tanışık, S. (2024). Üniversite web sitelerinin kullanılabilirlik sorunları üzerine bir araştırma: Başkent üniversitesi örneği. Bilim Eğitim Sanat ve Teknoloji Dergisi, 8(1), 22–43.
  40. Lavidas, K., Komis, V., & Achriani, A. (2022). Explaining faculty members’ behavioral intention to use learning management systems. Journal of Computers in Education, 9(4), 707–725.
  41. Lim, Y. M., Ayesh, A., & Chee, K. N. (2013). Socio-Demographic Differences in the Perceptions of Learning Management System (Lms) Design. International Journal of Software Engineering & Applications, 4(5), 15–35.
  42. Kous, K., Pušnik, M., Heričko, M., & Polančič, G. (2020). Usability evaluation of a library website with different end user groups. Journal of Librarianship and Information Science, 52(1), 75–90.
  43. Tinmaz, H., & Lee, J. H. (2020). An analysis of users’ preferences on learning management systems: a case on German versus Spanish students. Smart Learning Environments, 7(1), 30.