The Impact of Z-Score Transformation Scaling on the Validity, Reliability, and Measurement Error of Instrument SATS-36

Sumin Sumin

The Impact of Z-Score Transformation Scaling on the Validity, Reliability, and Measurement Error of Instrument SATS-36

Sumin Sumin

Abstract

The Likert scale is a psychometric scale commonly used for response measurement. This measurement scale includes techniques for designing and administering surveys as well as coding and analyzing data. However, Likert scaling has various limitations that can affect the resulting data. This study aims to reprove the number of dimensions of the SATS-36 instrument, prove the validity, and estimate the reliability of the statistical attitude instrument (SATS-36) on students at religious universities in Indonesia using Z-Score Transformation Scaling. The latent constructs of cognitive competence, value, difficulty, effect, and effort were constructed using a Likert scale according to the pattern of statements on each item. This study uses confirmatory research with a quantitative approach. For students at religious universities in Indonesia, 243 respondents were selected using a stratified one-stage cluster random sampling technique. Proof of validity and estimation of reliability was done using confirmatory factor analysis. The results of this study show that the rescaling method can improve the validity of the factors but cannot increase Cronbach's coefficient of internal consistency and cannot reduce the standard error of measurement for each item. This research implies that it is not enough to rescale or transform the data to improve the validity and reliability of a measuring instrument. However, it is necessary to calibrate the statement sentence or item question so that the item measures its construct. Further research also needs to test the effectiveness of rescaling in addition to the Z-Score in improving the validity and reliability of measuring instruments.

Keywords

reliability; rescaling; SATS-36; validity; Z-Score

References

Azwar, S. (2016). Dasar-dasar psikometrika (II). Pustaka Pelajar.

Bose, D. (2017). Development and validation of an instrument to measure attitude of undergraduate students towards Statistics.

Brace, I. (2018). Questionnaire design: How to plan, structure and write survey material for effective market research. Kogan Page Publishers.

Brown, T. A. (2015). Confirmatory factor analysis for applied research. Guilford publications.

Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating objective measuring instruments. Psychological Assessment, 31(12), 1412–1427. https://doi.org/10.1037/pas0000626

Cox, J., & Cox, K. B. (2008). Your opinion, please!: How to build the best questionnaires in the field of education. Corwin Press.

Crouch, C. H., Wisittanawat, P., Cai, M., & Renninger, K. A. (2018). Life science students’ attitudes, interest, and performance in introductory physics for life sciences: An exploratory study. Physical Review Physics Education Research, 14(1), 10111.

Didow Jr, N. M., Keller, K. L., Barksdale Jr, H. C., & Franke, G. R. (1985). Improving measure quality by alternating least squares optimal scaling. Journal of Marketing Research, 22(1), 30–40.

Dwyer, E. E. (1993). Attitude scale construction: a review of the literature.

Gillespie, B. J., Mulder, C. H., & Eggleston, C. M. (2021). Measuring migration motives with open‐ended survey data: Methodological and conceptual issues. Population, Space and Place, 27(6), e2448.

Gure, G. S. (2015). Different scale construction approaches used to attitude measurement in social science research. International Journal of Research in Economics and Social Sciences, 5(1), 26–44.

Harrington, D. (2009). Confirmatory factor analysis. Oxford university press.

Hommik, C., & Luik, P. (2017). Adapting the survey of attitudes towards statistics (SATS-36) for Estonian secondary school students. Statistics Education Research Journal, 16(1), 228–239.

Iwaniec, J. (2019). Questionnaires: Implications for effective implementation. In The Routledge handbook of research methods in applied linguistics (pp. 324–335). Routledge.

Jaeger, R. G., & Halliday, T. R. (1998). On confirmatory versus exploratory research. Herpetologica, S64–S66.

Kandasamy, I., Kandasamy, W. B., Obbineni, J. M., & Smarandache, F. (2020). Indeterminate Likert scale: feedback based on neutrosophy, its distance measures and clustering algorithm. Soft Computing, 24(10), 7459–7468.

Khavenson, T., Orel, E., & Tryakshina, M. (2012). Adaptation of survey of attitudes towards statistics (SATS 36) for Russian sample. Procedia-Social and Behavioral Sciences, 46, 2126–2129.

Kinyua, K., & Okunya, L. O. (2014). Validity and reliability of teacher-made tests: case study of year 11 physics in Nyahururu District of Kenya. African Educational Research Journal, 2(2), 61–71.

Lakshmi, S., & Mohideen, M. A. (2013). Issues in reliabilityand validity of research. International Journal of Management Research and Reviews, 3(4), 2752.

Lovelace, M., & Brickman, P. (2013). Best practices for measuring students’ attitudes toward learning science. CBE—Life Sciences Education, 12(4), 606–617.

Malik, M. A. A., Mustapha, M. F., Sobri, N. M., Abd Razak, N. F., Zaidi, M. N. M., Shukri, A. A., & Sham, M. A. L. Z. (2021). Optimal reliability and validity of measurement model in confirmatory factor analysis: different likert point scale experiment. Journal of Contemporary Issues and Thought, 11, 105–112.

Maloshonok, N., & Terentev, E. (2017). The mismatch between student educational expectations and realities: prevalence, causes, and consequences. European Journal of Higher Education, 7. https://doi.org/10.1080/21568235.2017.1348238

Mehra, S. (2017). Scaling techniques of attitude measurement. International Journal of Advanced Education and Research, 2(2), 41–50.

Molugaram, K., Rao, G. S., Shah, A., & Davergave, N. (2017). Statistical techniques for transportation engineering. Butterworth-Heinemann.

Naji Qasem, M. A., & Ahmad Gul, S. B. (2014). Effect of items direction (positive or negative) on the factorial construction and criterion related validity in likert scale. Khazar Journal of Humanities and Social Sciences, 17(3), 77–85. https://doi.org/10.5782/2223-2621.2014.17.3.77

Onwuegbuzie, A. J., & Wilson, V. A. (2003). Statistics anxiety: nature, etiology, antecedents, effects, and treatments--a comprehensive review of the literature. Teaching in Higher Education, 8(2), 195–209.

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: A review of the literature and its implications. International Journal of Science Education, 25(9), 1049–1079.

Patten, M. (2016). Questionnaire research: A practical guide. Routledge.

Pelch, M. A., & McConnell, D. A. (2017). How does adding an emphasis on socioscientific issues influence student attitudes about science, its relevance, and their interpretations of sustainability? Journal of Geoscience Education, 65(2), 203–214.

Pervez, A. K. M. K., Maniruzzaman, M., Shah, A. A., Nabi, N., & Ado, A. M. (2020). The meagerness of simple likert scale in assessing risk: how appropriate the fuzzy likert is? NUST Journal of Social Sciences and Humanities, 6(2), 138–150.

Ramirez, C., Schau, C., & Emmioğlu, E. (2012). The importance of attitudes in statistics education. Statistics Education Research Journal, 11(2), 57–71.

Saidi, S. S., & Siew, N. M. (2019). Investigating the validity and reliability of survey attitude towards statistics instrument among rural secondary school students. International Journal of Educational Methodology, 5(4), 651–661.

Schau, C. (2003). Students’ attitudes: The “other” important outcome in statistics education. Proceedings of the Joint Statistical Meetings, 3673–3681.

Schau, C., Millar, M., & Petocz, P. (2012). Research on attitudes towards statistics. Statistics Education Research Journal, 11(2), 2–5.

Simms, L. J., Zelazny, K., Williams, T. F., & Bernstein, L. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological Assessment, 31(4), 557–566. https://doi.org/10.1037/pas0000648

Taherdoost, H. (2019). What is the best response scale for survey and questionnaire design; review of different lengths of rating scale/attitude scale/Likert scale. Hamed Taherdoost, 1–10.

Thanasegaran, G. (2009). Reliability and validity issues in research. Integration & Dissemination, 4.

Walpole, R. E., Myers, R. H., Myers, S. L., & Ye, K. (1993). Probability and statistics for engineers and scientists (Vol. 5). Macmillan New York.

Wang, T., & Berlin, D. (2010). Construction and validation of an instrument to measure Taiwanese elementary students’ attitudes toward their science class. International Journal of Science Education, 32(18), 2413–2428.

Full Text: PDF

DOI: 10.15408/jp3i.v11i2.26591