Assessing the Usability of a Comprehensive Research Management System: A Mixed-Use Study in Iran
Subject Areas : Expert Systems
Mehdi Beyramijam
1
,
Mohammad Sadegh Sohrabi
2
*
,
Mohammad Babamiri
3
,
Arezo Farhadi
4
1 - Dept. of Health in Emergency and Disaster, School of Public Health, University of Medical Sciences, Hamadan, Iran.
2 - Occupational Health and Safety Research Center, Institute of Health Sciences and Technology, Hamadan University of Medical Sciences, Hamadan, Iran
3 - Department of Ergonomics, School of Public Health, Research Center for Health Sciences, Institute of Health Sciences and Technology, Hamadan University of Medical Sciences, Hamadan, Iran.
4 - Ph.D. student, Department of Medical Library and Information Science, School of Health Management and Information Sciences, Iran University of Medical Sciences, Tehran, Iran.
Keywords: Research Management System, User-Computer Interface, Usability, convergent mixed methods study, Ergonomics,
Abstract :
Background and Objective: Usability is a key dimension of the quality of research systems. This study aimed to explain usability and examine users' experiences and challenges in interacting with the "Pajouhshiyar" research affairs management system at several medical universities in Iran.
Methods: A convergent parallel mixed-methods design was employed. Quantitatively, 169 users (76 men, 93 women) were assessed using a questionnaire for evaluating the usability of web-based software; independent t-tests examined differences by gender and by role. Qualitatively, 16 semi-structured interviews were conducted via purposive sampling and analyzed using content analysis.
Results: The mean “Relative Usability” score was 33.19 % (low to moderate). The lowest subscale was “Ease of Use” (23.82 %), and the highest was “Content/Output Quality” (42.40 %). In between-group comparisons, women scored higher than men across most domains (no significant differences for “System Terminology and Information” and “System Capabilities”). Likewise, officers/managers outscored researchers on most scales (no significant differences for “System Terminology and Information,” “Personalization,” and “Error Management”). The qualitative analysis yielded five main categories: complex pathways/menus and similarity of options; system errors and saving/storage problems; insufficient training and support; strengths in documentation and national integration; and improvement suggestions including process simplification, redesign of search/reporting, personalized dashboards, and attention to a mobile version.
Conclusions: Despite strengths in documentation and integration, deficits in ease of use and learnability hinder optimal efficiency and satisfaction. User-centered workflow redesign, improved error handling and feedback, structured training, and continuous user experience monitoring could enhance system adoption, user satisfaction, and productivity.
[1] ISO 9241-11:2018. Ergonomics of human-system interaction – Part 11: Usability: Definitions and concepts, International Organization for Standardization, 2018.
[2] P. W. Jordan, B. Thomas, I. L. McClelland, and B. Weerdmeester, Usability Evaluation In Industry. Taylor & Francis, 1996.
[3] M. M. Yusof, J. Kuljis, A. Papazafeiropoulou, and L. K. Stergioulas, "An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit)," (in eng), Int J Med Inform, vol. 77, no. 6, pp. 386-98, Jun 2008, doi: 10.1016/j.ijmedinf.2007.08.011.
[4] P. J, I. Pendidikan, Lisnawita, and C. Loneli, "Exploring Research and Service Information System Usability by Heuristic Evaluation as a Compelement of System Usability Scale," presented at the Jurnal Penelitian Pendidikan IPA, 2023. [Online]. Available: https://jppipa.unram.ac.id/index.php/jppipa/article/download/5571/4298.
[5] P. Edilson, M. Érica, V. José, M. Cristiano, and T. Daniela, "Avaliação da Usabilidade do Sistema de Informação e Gestão de Projetos (SIGProj) e da Aceitação de uma Proposta na Extensão Universitária," Anais do Computer on the Beach, vol. 14, pp. 272-279, 2023 2023, doi: https://doi.org/10.14210/cotb.v14.p272-279.
[6] S. C et al., "Empowering Researchers to Query Medical Data and Biospecimens by Ensuring Appropriate Usability of a Feasibility Tool: Evaluation Study," JMIR Human Factors, vol. 19, no. 10, p. e43782, 2022 2022, doi: 10.2196/43782.
[7] A. M. S. Mohd, S. Ali, N. S. A. Mohd, and H. Hasrul, "The Importance of Usability Features in Enhancing Online Communication Satisfaction," Malaysian Journal of Communication, vol. 32, no. 1, pp. 1-15, 2016 2016, doi: 10.17576/jkmjc-2016-3201-01.
[8] S. Etemadinezhad, E. Amouzadeh, and J. Charati, "Investigating the Usability of the Integrated Research Automation System (SYAT): A Heuristic Evaluation," Iranian Journal of Ergonomics, vol. 10, no. 1, pp. 46-55, 2023 2023, doi: 10.18502/iehfs.v10i1.14411.
[9] F. Bahador, F. Salehi, E. Esmaeeli, A. Akbari, and A. Sabahi, "Evaluation the Usability of Hamava System Using Heuristic Evaluation," Frontiers in Health Informatics, vol. 9, no. 12, pp. 11045–11052, 2023 2023, doi: 10.30699/fhi.v12i0.462.
[10] G.-P. Patricio, F.-P. Félix, and U.-U. Pilar, "Hybrid Procedure for Measuring Usability of Management Systems in Higher Education," Iberian Conference on Information Systems and Technologies, 2019 2019, doi: 10.23919/CISTI.2019.8760905.
[11] M. D. E. Fernanda, E. Leandro, and P. B. Denise, "Usability of the “Systematic Review Support” computer system: a methodological study," Revista Latino-Americana de Enfermagem, 2024 2024, doi: 10.1590/1518-8345.7081.4374.
[12] J. Nikokaran, S. A. Fatemi Aghda, H. Dehghan, M. Mirfakhraei, M. M. Poornematy, and M. Mirfakhraei, "Usability Evaluation of the Ministry of Health Exam System: Viewpoint the Users," (in eng), Journal of Health and Biomedical Informatics, Original Article vol. 11, no. 3, pp. 176-185, 2024, doi: 10.34172/jhbmi.2024.23.
[13] R. Kazemi, Z. Rasouli kahaki, and A. S. Jafari Roodbandi, "The relationship between the usability of research centers system and mental workload caused by its interaction with users of shiraz university of medical sciences," (in eng), Iran Occupational Health Journal, Research vol. 18, no. 1, pp. 274-284, 2021, doi: 10.52547/ioh.18.1.274.
[14] J. Faradmal and J. Keshvari Kamran, "The validity and reliability of an usability assessment tool for a web-based software," (in eng), Iranian Journal of Ergonomics, Research vol. 2, no. 3, pp. 57-69, 2014. [Online]. Available: http://journal.iehfs.ir/article-1-107-fa.html.
[15] S. Etemadinezhad, E. Amouzadeh, and J. Yazdani Charati, "Investigating the Usability of the Integrated Research Automation System (SYAT): A Heuristic Evaluation," Iranian Journal of Ergonomics, vol. 10, no. 1, pp. 46-55, 2022, doi: 10.18502/iehfs.v10i1.14411.
[16] A. Bangor, P. T. Kortum, and J. T. Miller, "An Empirical Evaluation of the System Usability Scale," International Journal of Human–Computer Interaction, vol. 24, no. 6, pp. 574-594, 2008/07/29 2008, doi: 10.1080/10447310802205776.
[17] I. Dianat, Z. Ghanbari, and M. AsghariJafarabadi, "Psychometric properties of the persian language version of the system usability scale," (in eng), Health Promot Perspect, vol. 4, no. 1, pp. 82-9, 2014, doi: 10.5681/hpp.2014.011.
[18] U. H. Graneheim and B. Lundman, "Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness," (in eng), Nurse Educ Today, vol. 24, no. 2, pp. 105-12, Feb 2004, doi: 10.1016/j.nedt.2003.10.001.
[19] M. Beyramijam, M. S. Sohrabi, M. Babamiri, and A. Farhadi, "Usability of Research and Educational Systems: A Policy Brief," (in eng), Iranian Journal of Ergonomics, Brief Report/Brief Policy vol. 13, no. 4, 2026.