Usability Evaluation of SIMAK at Udayana University Using Outlier Identification with Webuse and Heuristic Methods

Deppy librata; Putu Desiana  Wulaning Ayu; Roy Rudolf  Huizen

doi:10.15408/jti.v19i1.48807

Authors

Deppy librata Department of Magister Information Systems, Institut Teknologi dan Bisnis STIKOM Bali, Indonesia
Putu Desiana Wulaning Ayu Department of Magister Information Systems, Institut Teknologi dan Bisnis STIKOM Bali, Indonesia
Roy Rudolf Huizen Department of Magister Information Systems, Institut Teknologi dan Bisnis STIKOM Bali, Indonesia

DOI:

https://doi.org/10.15408/jti.v19i1.48807

Keywords:

Usability, Evaluation, SIMAK, Webuse, Heuristic Evaluation

Abstract

Universities require reliable and user-friendly academic information systems to support teaching, learning, and administrative processes. However, the usability of such systems often encounters obstacles that affect user satisfaction and operational efficiency. This study evaluates the usability of the Sistem Informasi Manajemen Akademik (SIMAK) at Udayana University using a combined Webuse questionnaire and Heuristic Evaluation approach. A total of 100 student responses were collected, and outlier identification using standard deviation analysis removed 34 inconsistent responses. This step was essential for preventing inflated Webuse scores and ensuring that the final dataset (n = 66) more accurately reflected typical user experiences.The Webuse results classify all usability dimensions in the “Excellent” category Content Organization & Readability (0.86), Navigation & Links (0.83), User Interface Design (0.84), and Performance & Effectiveness (0.81). However, the heuristic evaluation conducted by four expert evaluators identified 19 moderate to high-severity issues, revealing critical weaknesses in system responsiveness, interface consistency, and error prevention. These contrasting outcomes highlight that high perceived satisfaction does not necessarily align with expert-validated usability standards.The main contribution of this study lies in integrating outlier detection to refine questionnaire-based usability data, resulting in more valid interpretations. The findings offer practical recommendations for improving SIMAK’s performance, interface clarity, and error-handling mechanisms, while providing methodological insights for future usability evaluations.

References

[1] D. Khair, “Peran Lembaga Pendidikan dalam Masyarakat di era Modrn,” Ilm. Keagamaan, Pendidik. Kemasyarakatan, vol. 12, no. 2, pp. 24–36, 2021.

[2] F. Agustina, R. Annisa, N. Ansori, W. Findiastuti, and I. D. Aisyah, “Usability Evaluation of Academic Information System Using the WEBUSE Method: A Study on University of Trunojoyo Madura Web Portal,” Proceeding - IEEE 8th Inf. Technol. Int. Semin. ITIS 2022, no. October 2022, pp. 184–189, 2022, doi: 10.1109/ITIS57155.2022.10009969.

[3] and A. M. . N. A. N. . Rachman, E. N. F. . Dewi, Reynaldi Akbar Maulana, “USABILITY EVALUATION SIMAK SILIWANGI UNIVERSITY USING HEURISTIC EVALUATION AND WEBUSE APPROACH,” J. Tek. Inform., vol. 3, no. 4, pp. 983–991, 2022, doi: https://doi.org/10.20884/1.jutif.2022.3.4.218.

[4] A. A. N. H. Susila and D. M. Sri Arsa, “Analisis System Usability Scale (SUS) dan Perancangan Sistem Self Service Pemesanan Menu di Restoran Berbasis Web,” Maj. Ilm. UNIKOM, vol. 21, no. 1, pp. 3–8, 2023, doi: 10.34010/miu.v21i1.10683.

[5] T. Junita Maulani, Suprapto, and A. Reza Perdanakusuma, “Evaluasi User Experience Menggunakan Metode Usability Testing dan User Experience Questionnaire (UEQ) (Studi Kasus: Website Superprof.co.id dan Zonaprivat.com),” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 6, pp. 2639–2645, 2021, [Online]. Available: http://j-ptiik.ub.ac.id

[6] M. I. Maksum, B. T. Hanggara, and A. Rachmadi, “Evaluasi Usability dengan Metode Heuristic Walkthrough (Studi Kasus: Website Beasiswa XYZ),” vol. 5, no. 2, pp. 549–557, 2021, [Online]. Available: http://j-ptiik.ub.ac.id

[7] M. Afriansyah, A. Sunyoto, and A. Nasiri, “Evaluasi Kualitas Website Pemerintah Daerah Dengan Menggunakan WebQual 4.0 Pada Instansi Pemerintahan ABC,” J. Teknol. Inf. dan Komun., vol. 8, no. 2, 2020, doi: 10.30646/tikomsin.v8i2.468.

[8] A. S. Fachruzi, “Analisis Penerapan Sistem Informasi Aplikasi Pegadaian Digital Service Dalam Memudahkan Nasabah Bertransaksi (Studi Kasus Pegadaian Unit Betun),” HOAQ (High Educ. Organ. Arch. Qual. J. Teknol. Inf., vol. 12, no. 2, pp. 52–62, 2023, doi: 10.52972/hoaq.vol12no2.p52-62.

[9] M. Penha, W. Franklin, M. Correia, F. Ferreira, and C. Campos, “Heuristic Evaluation of Usability-a Case study with the Learning Management Systems (LMS) of IFPE,” Int. J. Humanit. Soc. Sci., vol. 4, no. 6, pp. 295–303, 2014, [Online]. Available: http://teleduc.nied.unicamp.br/

[10] N. B. N. Rozali and M. Y. B. Said, “Usability testing on government agencies web portal: A study on Ministry of Education Malaysia (MOE) web portal,” 2015 9th Malaysian Softw. Eng. Conf. MySEC 2015, pp. 37–42, 2016, doi: 10.1109/MySEC.2015.7475192.

[11] T. K. Chiew and S. S. Salim, “Webuse: Website usability evaluation tool,” Malaysian J. Comput. Sci., vol. 16, no. 1, pp. 47–57, 2003.

[12] L. M. Ginting, G. Sianturi, and C. V. Panjaitan, “Perbandingan Metode Evaluasi Usability Antara Heuristic Evaluation dan Cognitive Walkthrough,” J. Manaj. Inform., vol. 11, no. 2, pp. 146–157, 2021, doi: 10.34010/jamika.v11i2.5480.

[13] M. G. L. Putra, N. R. Sabilla, and S. R. Natasia, “Evaluasi Usability Website Berita Online Prokal.Co Menggunakan Evaluasi Heuristic dan Webuse,” J. Teknol. Inf. dan Ilmu Komput., vol. 7, no. 5, pp. 911–922, 2020, doi: 10.25126/jtiik.2020753707.

[14] S. Danil and M. S. Hasibuan, “Evaluasi Sistem ATR/BPN Berbasis Webuse dan Heuristic Evaluation,” J. Digit. Lit. Volunt., vol. 2, no. 1, pp. 1–10, 2023, doi: 10.57119/litdig.v2i1.41.

[15] Y. A. Gerhana, M. N. Nudyawati, D. R. Ramdania, A. Wahana, and N. Lukman, “Heuristic and Webuse Method to Evaluate UI/UX of Faculty Website,” 2022 10th Int. Conf. Cyber IT Serv. Manag. CITSM 2022, pp. 1–5, 2022, doi: 10.1109/CITSM56380.2022.9935889.

[16] R. Listiya, “Analisis Kinerja Web Disdukcapil Dengan Menggunakan Heuristic Evaluation Dan Webuse Web Performance Analysis with Heuristic Evaluation and Webuse,” J. Digit. Lit. Volunt., vol. 2, no. 1, pp. 11–22, 2023, doi: 10.57119/litdig.v2i1.42.

[17] N. W. J. K. Dewi, I. M. Candiasa, and G. Indrawan, “Evaluasi Website Sistem Informasi Perencanaan Studi Stmik Stikom Indonesia Ditinjau Dari Pengguna Mahasiswa Menggunakan Teknik Webqual 4.0, Firstclick, Dan Heuristik,” J. Nas. Pendidik. Tek. Inform. JANAPATI, vol. 9, no. 2, pp. 266–280, 2020, [Online]. Available: https://ejournal.undiksha.ac.id/index.php/janapati/article/view/23212

[18] N. Tian, C. H. Lai, K. Pericleous, J. Sun, and W. Xu, “Contraction-expansion coefficient learning in quantum-behaved particle swarm optimization,” Proc. - 2011 10th Int. Symp. Distrib. Comput. Appl. to Business, Eng. Sci. DCABES 2011, vol. 35, no. 3, pp. 303–308, 2011, doi: 10.1109/DCABES.2011.32.

[19] R. Rahmi, I. M. A. Pradnyana, and M. W. A. Kesiman, “Usability Testing Berbasis ISO 9241-11 Pada Aplikasi Salak Bali (Studi Kasus : Polres Buleleng),” Kumpul. Artik. Mhs. Pendidik. Tek. Inform., vol. 8, no. 3, pp. 510–521, 2019.

[20] C. Erika, E., Yunior, K., Devita, F., Tamara, I., & Herryanto, “Pengaruh Disiplin Kerja, Kompensasi dan Kepemimpinan terhadap Kinerja Karyawan dalam PT. Sabas Indonesia,” Jesya (Jurnal Ekon. Ekon. Syariah), vol. 4, no. 2, pp. 905–914, 2021, doi: https://doi.org/10.36778/jesya.v4i2.465.

[21] T. Wahyuningrum, Mengukur Usability Perangkat Lunak, Pertama. Sleman: DEEPUBLISH, 2021. [Online]. Available: https://play.google.com/books?id=Pzk9EAAAQBAJ

[22] M. M. Sanaky, “Analisis Faktor-Faktor Keterlambatan Pada Proyek Pembangunan Gedung Asrama Man 1 Tulehu Maluku Tengah,” J. Simetrik, vol. 11, no. 1, pp. 432–439, 2021, doi: 10.31959/js.v11i1.615.

[23] I. Machali, Metode Penelitian Kuantitatif “Panduan Praktis Merencanakan, Melaksanakan dan Analisis dalam Penelitian Kuantitatif,” 3rd ed. Yogyakarta: Fakultas Ilmu Tarbiyah dan Keguruan Universitas Islam Negeri (UIN) Sunan Kalijaga Yogyakarta, 2021.

[24] P. R. Sihombing, S. Suryadiningrat, D. A. Sunarjo, and Y. P. A. C. Yuda, “Identifikasi Data Outlier (Pencilan) dan Kenormalan Data Pada Data Univariat serta Alternatif Penyelesaiannya,” J. Ekon. Dan Stat. Indones., vol. 2, no. 3, pp. 307–316, 2023, doi: 10.11594/jesi.02.03.07.

[25] Sugiyono, Metode Penelitian Kuantitatif, Kualitatif dan R&D. Bandung: Alfabeta, 2011.

[26] O. Ismael, “The Standard Deviation Score: a novel similarity metric for data analysis,” J. Big Data, vol. 12, no. 1, 2025, doi: 10.1186/s40537-025-01091-z.

[27] G. Syamsuar, “MODUL WORKSHOP STATISTIKA (EKM235) Analisis Data Non-Parametrik,” Sekol. Tinggi Ilmu Ekon. Indones. Jakarta, no. 1, 2020.

[28] A. Field, “Discovering statistics using IBM SPSS statistics,” SAGE Publ., 2013.