Development and Psychometric Validation of the Belonging-Supportive Deep Learning Environments (BSDLE) Instrument Using CFA and Multidimensional IRT

Oktavia Oktavia; Wardani Rahayu; Iva Sarifah

doi:10.15408/jp3i.v15i2.49207

Authors

Oktavia Oktavia Educational Research and Evaluation, State University of Jakarta
Wardani Rahayu Educational Research and Evaluation, State University of Jakarta
Iva Sarifah Educational Research and Evaluation, State University of Jakarta

DOI:

https://doi.org/10.15408/jp3i.v15i2.49207

Keywords:

belongingness, mirt, cfa

Abstract

This study aimed to develop and validate a multidimensional instrument for assessing Belonging-Supportive Deep Learning Environments (BSDLE) in higher education. A quantitative, cross-sectional instrument development design was employed. Data were collected from 825 university students from five Indonesian universities implementing Flipped Learning. Participants completed a 47-item Likert-scale questionnaire measuring six dimensions: student cohesiveness, involvement, investigation, task orientation, cooperation, and faculty–staff relations. Data analysis was conducted in several stages. Preliminary data screening indicated no missing data and adequate variability. Confirmatory Factor Analysis (CFA) using a multidimensional correlated model demonstrated acceptable model fit (χ² = 2966.51, df = 999, p < .001; RMSEA = 0.049; CFI = 0.97; NFI = 0.95; RFI = 0.95), with most standardized factor loadings exceeding 0.50. Further psychometric evaluation was conducted using Multidimensional Item Response Theory (MIRT) with the Graded Response Model (GRM). The results indicated that most items demonstrated moderate to high discrimination parameters (approximately 0.85–2.10) and ordered threshold parameters, suggesting appropriate functioning of the response categories. Marginal reliability estimates (0.78–0.89) were generally higher than Cronbach’s alpha estimates (0.63–0.80), indicating improved measurement precision under the IRT framework. The Test Information Function (TIF) analysis revealed that the Investigation (IN) and Involvement (IV) dimensions provided higher measurement precision across the ability
range (θ = -1 to +2), whereas the student cohesiveness (SC), cooperation (CO), and faculty–staff relations (FR) dimensions showed relatively lower information values and require further refinement. The findings provide preliminary evidence supporting the construct validity and measurement reliability of the BSDLE instrument as a multidimensional measure of supportive learning environments in higher education. Further refinement, cross-validation, and additional psychometric evaluation are recommended before broader diagnostic app

References

Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day.

International Society for Technology in Education.

Bond, M. (2020). Facilitating student engagement through educational technology: Towards a conceptual framework. Journal of Interactive Media in Education, 2020(1), Article 11.

https://doi.org/10.5334/jime.528

Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer.

https://doi.org/10.1007/978-94-007-6857-4

Cai, L. (2021). Multidimensional item response theory. In P. Irwing, T. Booth, & D. J. Hughes (Eds.),

The Wiley handbook of psychometric testing: A multidisciplinary reference on survey, scale and test development

(pp. 1–35). Wiley. https://doi.org/10.1002/9781119716723.ch6

Chionh, Y. H., & Fraser, B. J. (2009). Classroom environment, achievement, attitudes and self-esteem

in geography and mathematics in Singapore. International Research in Geographical and Environmental

Education, 18(1), 29–44. https://doi.org/10.1080/10382040802591530

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Lawrence Erlbaum

Associates.

Finch, W. H., & French, B. F. (2022). Educational and psychological measurement (2nd ed.). Routledge.

Fraser, B. J. (2012). Classroom learning environments: Retrospect, context and prospect. In B. J. Fraser,

K. Tobin, & C. J. McRobbie (Eds.), Second international handbook of science education (pp. 1191–1239).

Springer. https://doi.org/10.1007/978-1-4020-9041-7_79

Fraser, B. J., Fisher, D. L., & McRobbie, C. J. (1996). Development, validation, and use of personal and

class forms of a new classroom environment instrument. Paper presented at the Annual Meeting of the

American Educational Research Association, New York, NY.

Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept,

state of the evidence. Review of Educational Research, 74(1), 59–109.

https://doi.org/10.3102/00346543074001059

Freeman, T. M., Anderman, L. H., & Jensen, J. M. (2007). Sense of belonging in college freshmen at the

classroom and campus levels. The Journal of Experimental Education, 75(3), 203–220.

https://doi.org/10.3200/JEXE.75.3.203-220

Goodenow, C. (1993). The psychological sense of school membership among adolescents: Scale

development and educational correlates. Psychology in the Schools, 30(1), 79–90.

https://doi.org/10.1002/1520-6807(199301)30:1%3C79::AID-PITS2310300113%3E3.0.CO;2-X

Gopalan, M., & Brady, S. T. (2020). College students’ sense of belonging: A national perspective.

Educational Researcher, 49(2), 134–137. https://doi.org/10.3102/0013189X19897622

Green, S. B., Yang, Y., & McNeish, D. (2021). Coefficient alpha and omega reliability coefficients: A

theoretical and empirical comparison. Psychological Methods, 26(6), 714–732.

https://doi.org/10.1037/met0000343

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis (8th ed.).

Cengage.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Kluwer

Academic Publishers.

Lo, C. K., & Hew, K. F. (2020). A comparison of flipped learning with traditional learning in higher

education: Students’ perspectives. Educational Research Review, 30, Article 100314.

https://doi.org/10.1016/j.edurev.2020.100314

Moore, G. C., & Benbasat, I. (1991). Development of an instrument to measure the perceptions of

adopting an information technology innovation. Information Systems Research, 2(3), 192–222.

https://doi.org/10.1287/isre.2.3.192

Pedler, M., Willis, R., & Nieuwoudt, J. (2022). A sense of belonging at university: Student retention,

motivation and enjoyment. Journal of Further and Higher Education, 46(3), 397–408.

https://doi.org/10.1080/0309877X.2021.1955844