Diagnostic instruments for numeracy skills in chemistry learning: a development study Oriondo Antonio
DOI:
https://doi.org/10.15408/es.v17i2.49862Keywords:
Numeracy skills, diagnostic instrument, chemistry learning, validity, reliabilityAbstract
This study aims to develop a valid and reliable diagnostic instrument to measure students' numeracy skills in the context of chemistry learning. The instrument development followed a systematic procedure based on the Oriondo & Antonio model, which includes the stages of planning, construction, validation, and revision. The instrument was designed based on numeracy indicators integrated with chemistry content, particularly on topics requiring quantitative understanding, such as reaction rates. Content validity was reviewed by chemistry and education experts, while empirical reliability testing was conducted using Rasch modelling. The analysis results showed that the instrument had an Aiken validity of 0.80 (valid category), item reliability of 0.88 (very good category), person reliability of 0.80 (good category), and Cronbach's alpha value of 0.75 (good category). These findings indicate that the developed instrument meets the criteria for validity and reliability and is capable of specifically identifying students' misconceptions and numeracy gaps. Therefore, this instrument has the potential to be used as an initial diagnostic tool in designing remedial or enrichment learning strategies in chemistry classes
References
Abdurrahman, I. S., & Mahmudah, F. N. (2023). Development of a Digital-Preneurship Measurement Instrument: Alignment Approach Through Project-Based Learning. International Journal of Educational Methodology, 9(1), 283–295. https://doi.org/10.12973/IJEM.9.1.283
Achmad Rante Suparman, Eli Rohaeti, & Sri Wening. (2024). Student Misconception In Chemistry: A Systematic Literature Review. Pegem Journal of Education and Instruction, 14(2). https://doi.org/10.47750/pegegog.14.02.28
Ahmad, Alias, Hamat, & Mohamed. (2024). RELIABILITY ANALYSIS: APPLICATION OF CRONBACH’S ALPHA IN RESEARCH INSTRUMENTS. Pioneering the Future: Delving Into E‐Learning’s Landscape, 114–119. https://appspenang.uitm.edu.my/sigcs/2024-2/Articles/20244_ReliabilityAnalysis-ApplicationOfCronbachsAlphaInResearchInstruments.pdf
Aiken, L. R. (1980). Content validity and reliability of single items or questionnaires. Educational and Psychological Measurement, 40(4), 955–959. https://doi.org/10.1177/001316448004000419
Aldossary, A., Campos-Gonzalez-Angulo, J. A., Pablo-García, S., Leong, S. X., Rajaonson, E. M., Thiede, L., Tom, G., Wang, A., Avagliano, D., & Aspuru-Guzik, A. (2024). In Silico Chemical Experiments in the Age of AI: From Quantum Chemistry to Machine Learning and Back. In Advanced Materials (Vol. 36, Issue 30). John Wiley and Sons Inc. https://doi.org/10.1002/adma.202402369
Ananiadou, K., Claro, M., & Magdalean Claro, oecdorg. (2009). 21st Century Skills and Competences for New Millennium Learners in OECD Countries. OECD Education Working Papers, 2009(41), 33. https://doi.org/10.1787/218525261154
Andersen, E. B. (1973). A goodness of fit test for the rasch model. Psychometrika, 38(1). https://doi.org/10.1007/BF02291180
Ariani, Y., Suparman, Helsa, Y., Zainil, M., & Rahmatina. (2024). ICT-Based or-Assisted Mathematics Learning and Numerical Literacy: A Systematic Review and Meta-Analysis. International Journal of Information and Education Technology, 14(3). https://doi.org/10.18178/ijiet.2024.14.3.2060
Bravenec, A. D., & Ward, K. D. (2023). Interactive Python Notebooks for Physical Chemistry. Journal of Chemical Education, 100(2). https://doi.org/10.1021/acs.jchemed.2c00665
Chin, H., & Chew, C. M. (2023). Cognitive diagnostic assessment with ordered multiple-choice items for word problems involving ‘Time.’ Current Psychology, 42(20). https://doi.org/10.1007/s12144-022-02965-8
D’Alessio, G., Parente, A., Stagni, A., & Cuoci, A. (2020). Adaptive chemistry via pre-partitioning of composition space and mechanism reduction. Combustion and Flame, 211. https://doi.org/10.1016/j.combustflame.2019.09.010
Easa, E., & Blonder, R. (2022). Development and validation of customized pedagogical kits for high-school chemistry teaching and learning: the redox reaction example. Chemistry Teacher International, 4(1). https://doi.org/10.1515/cti-2021-0022
Education Assessment Centre. (2023). National AKM Report. Ministry of Education, Culture, Research, and Technology. https://anbk.kemdikbud.go.id/anbk2023/
Fauzi, A. A., Susongko, P., & Hayati, M. N. (2022). Tes Kemampuan Berpikir Kritis pada Pembelajaran IPA di SMP Berbasis Model Rasch. PSEJ (Pancasakti Science Education Journal), 7(1). https://doi.org/10.24905/psej.v7i1.146
Hakkarainen, A., Cordier, R., Parsons, L., Yoon, S., Laine, A., Aunio, P., & Speyer, R. (2023). A systematic review of functional numeracy measures for 9–12 -year-olds: Validity and reliability evidence. International Journal of Educational Research, 119. https://doi.org/10.1016/j.ijer.2023.102172
Huang, L., Shu, X., Ge, N., Gao, L., Xu, P., Zhang, Y., Chen, Y., Yue, J., & Wu, C. (2023). The accuracy of screening instruments for sarcopenia: a diagnostic systematic review and meta-analysis. In Age and Ageing (Vol. 52, Issue 8). https://doi.org/10.1093/ageing/afad152
Linacre, J., & Wright, B. D. (1994). Dichotomous Mean Square Chi-square fit statistics. Rasch Measurement Transactions1, 8(2).
Merino-Soto, C. (2023). Aiken’s V Coefficient: Differences in Content Validity Judgments. MHSalud, 20(1). https://doi.org/10.15359/mhs.20-1.3
Morel, F., & Morgan, J. (1972). A Numerical Method for Computing Equilibria in Aqueous Chemical Systems. Environmental Science and Technology, 6(1). https://doi.org/10.1021/es60060a006
Moruk, S., & Sulisworo, D. (2024). Literature Review on Longitudinal Study of Improving Numerical Literacy at Elementary Education. Buletin Edukasi Indonesia, 3(03). https://doi.org/10.56741/bei.v3i03.757
Mullis, Martin, & Davier, V. (2021). TIMSS 2023 Assessment Framework. In TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College.
Nguyen, H. T., Domingo, P., Vervisch, L., & Nguyen, P. D. (2021). Machine learning for integrating combustion chemistry in numerical simulations. Energy and AI, 5. https://doi.org/10.1016/j.egyai.2021.100082
Oriondo, L. Loyola., & Antonio, E. M. D.-. (1984). Evaluating educational outcomes : tests, measurement and evaluation. Rex Book Store.
Patac, L. P., Adriano, Jr. P., & Bactil, C. M. (2021). Factor analytic method in developing scoring rubric for word problems. Asia Research Network Journal of Education, 1(3).
Pentapati, K. C., Chenna, D., Kumar, V. S., & Kumar, N. (2025). Reliability generalization meta-analysis of Cronbach’s alpha of the oral impacts on daily performance (OIDP) questionnaire. In BMC Oral Health (Vol. 25, Issue 1). https://doi.org/10.1186/s12903-025-05496-3
Pradana, P. W., Febriani, F., Ibnusaputra, M., & Jumadi, J. (2023). Development of Physics Test Instrument to Measure Verbal Representation of High School Student on Optical Instrument Topic. Jurnal Penelitian Pendidikan IPA, 9(10). https://doi.org/10.29303/jppipa.v9i10.3775
Ramadhan, W., Malahati, F., Romadhon, K., & Ramadhan, S. (2023). Analisis Butir Soal Tipe Multiple Choice Questions pada Penilaian Harian Sekolah Dasar. Tarbiyah Wa Ta’lim: Jurnal Penelitian Pendidikan Dan Pembelajaran, 10(2). https://doi.org/10.21093/twt.v10i2.6155
Rini Rahma Safitri, Gita Asyari, Dara Avira, & Abdul Fattah Nasution. (2024). Rekonstruksi Minat Belajar Peserta Didik Abad 21 Melalui Model Sistem Dinamis. Student Scientific Creativity Journal, 3(1), 133–143. https://doi.org/10.55606/sscj-amik.v3i1.4785
Sayre, J., Nabua, E., Salic-Hairulla, M., Alcopra, A., & Fernandez, M. J. (2025). Assessing General Chemistry Learning Gaps: A Needs Assessment of Competency Mastery among Grade 11 Learners. International Journal of Research and Innovation in Social Science, IX(IV). https://doi.org/10.47772/ijriss.2025.90400472
Surhasimi, & Arikunto. (2016). Prosedur Penelitian : Suatu Pendekatan Praktik. Rineka Cipta, 2006(2006).
Üce, M., & Ceyhan, İ. (2019). Misconception in Chemistry Education and Practices to Eliminate Them: Literature Analysis. Journal of Education and Training Studies, 7(3). https://doi.org/10.11114/jets.v7i3.3990
Wright, B. D. (1977). SOLVING MEASUREMENT PROBLEMS WITH THE RASCH MODEL. Journal of Educational Measurement, 14(2). https://doi.org/10.1111/j.1745-3984.1977.tb00031.x
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Suwahono Suwahono, Fina Sa’adah, Yulianingsih Yulianingsih
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
