Psychometric Properties of the Resistance to Framing Scale: Evidence from Indonesia

Rahmat Hidayat; Ajeng Putri Pertiwi

Psychometric Properties of the Resistance to Framing Scale: Evidence from Indonesia

Rahmat Hidayat, Ajeng Putri Pertiwi

Abstract

This paper reports the validation of the Indonesian version of the resistance to framing effects scale, which is part of the Adult Decision-Making Competence Index (A-DMC). Framing refers to an individual’s tendency to be influenced by how information is structured. The resistance to framing effects scale is a measurement tool designed to assess an individual’s susceptibility to framing. This scale comprises two dimensions: attribute framing and risky-choice framing. A total of 217 participants (60 men and 157 women) completed the 14-item scale. Item Response Theory and the Multidimensional Graded Response Model (MGRM) were used to evaluate the psychometric properties of the scale. The MGRM analysis results indicated that the data fit the model, as evidenced by global fit statistics. Additionally, all items showed a good fit with the MGRM model. The reliability of this scale was 0.697 for the attribute dimension and 0.722 for the risky-choice dimension. However, we found that one item had low discrimination (ATT5 with a = 0.638), while the remining 13 items had optimal discrimination. Based on these results, we conclude that the 14-item Indonesian version of the resistance to framing effects scale is a reliable measurement tool that can be used for future research in behavioral economics or economic psychology in Indonesia. There are limitations of this study: the relatively small sample size and the lack of convergent validity testing with other instruments. However, the strong psychometric properties observed in this study suggest that this instrument is suitable for use in future research and may also be applied for practical measurement purposes.

Keywords

construct validity; framing effect; item response theory; resistance to framing

References

Aczel, B., Szollosi, A., & Bago, B. (2018). The effect of transparency on framing effects in within-subject designs. Journal of Behavioral Decision Making, 31(1), 25–39. https://doi.org/10.1002/bdm.2036

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.

Bavolar, J. (2013). Validation of the adult decision-making competence in Slovak students. Judgment and Decision Making, 8(3), 386–392. https://doi.org/10.1017/S1930297500006057

Bell, R., & Lumsden, J. (1980). Test length and validity. Applied Psychological Measurement, 4(2), 165–170. https://doi.org/10.1177/014662168000400203

Berthet, V. (2021). The measurement of individual differences in cognitive biases: A review and improvement. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.630177

Berthet, V., Autissier, D., & de Gardelle, V. (2022). Individual differences in decision-making: A test of a one-factor model of rationality. Personality and Individual Differences, 189, 111485. https://doi.org/10.1016/j.paid.2021.111485

Bock, R. D., & Gibbons, R. D. (2021). Item response theory. John Wiley & Sons.

Bruine de Bruin, W., Parker, A. M., & Fischhoff, B. (2007). Individual differences in adult decision-making competence. Journal of Personality and Social Psychology, 92(5), 938–956. https://doi.org/10.1037/0022-3514.92.5.938

Chalmers, R. P. (2012). mirt: A multidimensional item response theory package for the R environment. Journal of Statistical Software, 48(6). https://doi.org/10.18637/jss.v048.i06

de Ayala, R. J. (1994). The influence of multidimensionality on the graded response model. Applied Psychological Measurement, 18(2), 155–170. https://doi.org/10.1177/014662169401800205

De Martino, B. (2012). The effect of context on choice and value. In R. Dolan & T. Sharot (Eds.), Neuroscience of Preference and Choice (pp. 93–119). Elsevier. https://doi.org/10.1016/B978-0-12-381431-9.00005-X

Diederich, A., Wyszynski, M., & Ritov, I. (2018). Moderators of framing effects in variations of the Asian disease problem: Time constraint, need, and disease type. Judgment and Decision Making, 13(6), 529–546. https://doi.org/10.1017/s1930297500006574

Fitzpatrick, A. R., & Yen, W. M. (2001). The effects of test length and sample size on the reliability and equating of tests composed of constructed-response items. Applied Measurement in Education, 14(1), 31–57. https://doi.org/10.1207/S15324818AME1401_04

Gabor, A. M., & Knezović, A. (2016). Decision making styles of investors in capital market. Proceedings of the Faculty of Economics & Business in Zagreb/Zbornik Ekonomskog Fakulteta u Zagrebu, 14(1).

Geiger, S. J., Vintr, J., & Rachev, N. R. (2022a). A reassessment of the resistance to framing scale. Behavior Research Methods, 55(5), 2320–2332. https://doi.org/10.3758/s13428-022-01876-7

Geiger, S. J., Vintr, J., & Rachev, N. R. (2022b). A reassessment of the resistance to framing scale. Behavior Research Methods, 55(5), 2320–2332. https://doi.org/10.3758/s13428-022-01876-7

Giuliani, F., Cannito, L., Gigliotti, G., Rosa, A., Pietroni, D., & Palumbo, R. (2023). The joint effect of framing and defaults on choice behavior. Psychological Research, 87(4), 1114–1128. https://doi.org/10.1007/s00426-022-01726-3

Geiger, S. J., Vintr, J., & Rachev, N. R. (2022). A reassessment of the resistance to framing scale. Behavior Research Methods, 55(5), 2320–2332. https://doi.org/10.3758/s13428-022-01876-7

Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assesing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347–360. https://doi.org/10.1111/j.1745-3984.1984.tb01039.x

Hidayat, R. (2016). Rasionalitas: Overview terhadap pemikiran dalam 50 tahun terakhir. Buletin Psikologi, 24(2), 101–122. https://doi.org/10.22146/buletinpsikologi.26772

Hidayat, R., & Putra, M. D. K. (2022). Pilot testing and preliminary psychometric validation of situational judgment test of resistance to sunk cost. Jurnal Psikologi, 49(3), 211–228. https://doi.org/10.22146/jpsi.73737

Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47(2), 263–292. https://www.jstor.org/stable/1914185

Kang, T., & Chen, T. T. (2008). Performance of the generalized S-χ2 item fit index for polytomous IRT models. Journal of Educational Measurement, 45(4), 391–406. https://doi.org/10.1111/j.1745-3984.2008.00071.x

Katerina, G. (2018). Development of a local version of the scale resistance to the framing effect on the basis of the comprehensive assesment of rational thinking [Thesis, HSE University]. https://www.hse.ru/en/edu/vkr/219398134

Kühberger, A. (2023). A systematic review of risky-choice framing effects. EXCLI Journal, 22, 1012–1031. https://doi.org/10.17179/excli2023-6169

Kühberger, A., & Gradl, P. (2013). Choice, rating, and ranking: Framing effects with different response modes. Journal of Behavioral Decision Making, 26(2), 109–117. https://doi.org/10.1002/bdm.764

LeBoeuf, R. A., & Shafir, E. (2003). Deep thoughts and shallow frames: On the susceptibility to framing effects. Journal of Behavioral Decision Making, 16(2), 77–92. https://doi.org/10.1002/bdm.433

Levin, I. P., & Gaeth, G. J. (1988). How consumers are affected by the framing of attribute information before and after consuming the product. Journal of Consumer Research, 15(3), 374–378. https://www.jstor.org/stable/2489471

Levin, I. P., Schneider, S. L., & Gaeth, G. J. (1998). All frames are not created equal: A typology and critical analysis of framing effects. Organizational Behavior and Human Decision Processes, 76(2), 149–188. https://doi.org/10.1006/obhd.1998.2804

Liang, S., & Zou, Y. (2018). Validation of adult decision-making competence in Chinese college students. Judgment and Decision Making, 13(4), 393–400. https://doi.org/10.1017/S1930297500009268

Lozano, L. M., García-Cueto, E., & Muñiz, J. (2008). Effect of the number of response categories on the reliability and validity of rating scales. Methodology, 4(2), 73–79. https://doi.org/10.1027/1614-2241.4.2.73

Mandel, D. R., & Kapler, I. V. (2018). Cognitive style and frame susceptibility in decision-making. Frontiers in Psychology, 9(AUG). https://doi.org/10.3389/fpsyg.2018.01461

Masters, G. N. (1982). A rasch model for partial credit scoring. Psychometrika, 47(2), 149–174. https://doi.org/10.1007/BF02296272

Maydeu-Olivares, A. (2013). Why should we assess the goodness-of-fit of IRT models? Measurement: Interdisciplinary Research & Perspective, 11(3), 127–137. https://doi.org/10.1080/15366367.2013.841511

Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71(4), 713–732. https://doi.org/10.1007/s11336-005-1295-9

Miyapuram, K. P., & Pammi, V. S. C. (2013). Understanding decision neuroscience: A multidisciplinary perspective and neural substrates. Progress in Brain Research, 202, 239–266. https://doi.org/10.1016/B978-0-444-62604-2.00014-9

Pabst, S., Brand, M., & Wolf, O. T. (2013). Stress and decision making: A few minutes make all the difference. Behavioural Brain Research, 250, 39–45. https://doi.org/10.1016/j.bbr.2013.04.046

Peng, J., Feng, T., Zhang, J., Zhao, L., Zhang, Y., Chang, Y., Zhang, Y., & Xiao, W. (2019). Measuring decision‐making competence in Chinese adults. Journal of Behavioral Decision Making, 32(3), 266–279. https://doi.org/10.1002/bdm.2114

Peng, J., Li, H., Miao, D., Feng, X., & Xiao, W. (2013). Five different types of framing effects in medical situation: A preliminary exploration. Iranian Red Crescent Medical Journal, 15(3), 161–165. https://doi.org/10.5812/ircmj.8469

Permatasari, A. W., Budiarti, L., & Srirejeki, K. (2020). The effect of sunk cost, framing effect, and educational background on the escalation of commitment. Journal of Contemporary Accounting, 2(1), 49–61. https://doi.org/10.20885/jca.vol2.iss1.art5

Petscher, Y., Mitchell, A. M., & Foorman, B. R. (2015). Improving the reliability of student scores from speeded assessments: An illustration of conditional item response theory using a computer-administered measure of vocabulary. Reading and Writing, 28(1), 31–56. https://doi.org/10.1007/s11145-014-9518-z

Piñon, A., & Gambara, H. (2005). A meta-analytic review of framing effect: Risky, attibute and goal framing. Psicothema, 17(2), 325–331. http://www.psicothema.com/pi?pii=3107

Rachev, N. R., Geiger, S. J., Vintr, J., Kirilova, D., Nabutovsky, A., & Nelsson, J. (2022). Actively open-minded thinking, bullshit receptivity, and susceptibility to framing: Evaluating the dual-process account in North America and Bulgaria. European Journal of Psychological Assessment, 38(6), 440–451. https://doi.org/10.1027/1015-5759/a000685

Rahayu, W., Hayat, B., & Putra, M. D. K. (2023). Analisis Rasch: Aplikasi dan interpretasi. UNJ Press.

Reise, S. P. (1999). Personality measurement issues viewed through the eyes of IRT. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every psychologist and educator should know (pp. 219–241). Erlbaum. https://doi.org/10.4324/9781410603593

Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika, 34(Suppl 1), 1–97. https://doi.org/10.1007/BF03372160

Solomon, E., & Kopelman, R. E. (1984). Questionnaire format and scale reliability: An examination of three modes of item presentation. Psychological Reports, 54(2), 447–452. https://doi.org/10.2466/pr0.1984.54.2.447

Su, S., Wang, C., & Weiss, D. J. (2021). Performance of the S−χ2 statistic for the multidimensional graded response model. Educational and Psychological Measurement, 81(3), 491–522. https://doi.org/10.1177/0013164420958060

Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd

Toplak, M. E. (2022). Cognitive sophistication and the development of judgment and decision-making. Academic Press Inc.

Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683

Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions. The Journal of Business, 59(4), S251–S278. https://www.jstor.org/stable/2352759

Weller, J. A., Ceschi, A., & Randolph, C. (2015). Decision-making competence predicts domain-specific risk attitudes. Frontiers in Psychology, 6. https://doi.org/10.3389/fpsyg.2015.00540

Wright, B. D. (1977). Misunderstanding the rasch model. Journal of Educational Measurement, 14(3), 219–225. https://doi.org/10.1111/j.1745-3984.1977.tb00039.x

Wright, B. D. (1996). Reliability and separation. Rasch Measurement Transactions, 9(4), 472. https://www.rasch.org/rmt/rmt94n.htm

Wyszynski, M., & Diederich, A. (2023). Individual differences moderate effects in an unusual disease paradigm: A psychophysical data collection lab approach and an online experiment. Frontiers in Psychology, 14. https://doi.org/10.3389/fpsyg.2023.1086699

Zhen, S., & Yu, R. (2016). All framing effects are not created equal: Low convergent validity between two classic measurements of framing. Scientific Reports, 6(1), 30071. https://doi.org/10.1038/srep30071

Full Text: PDF

DOI: 10.15408/jp3i.v14i1.38854