Qutest Construction and Psychometric Evaluation as Test of Attention and Willpower for Employee Selection Screening
Abstract
Attention and willpower are some of the primary modalities for work, so it is necessary to evaluate those potentials. However, standard tests for assessing the two aspects, such as the Pauli or Kraepelin test, are bias-prone to numerical ability. Due to that concern, Qutest was designed as an attention and willpower (screening) test with symbolic stimulus adapted from the d2 test (Brickencamp, 1998). Two studies were conducted for Qutest psychometric properties evaluation. In the first study, 261 participants finished Qutest on a web-based platform with a maximum of 96 pages, each containing 24 symbols of key stimuli (a symbol Q with two dots) and distractors that had to complete in a maximum of 30 minutes. Analyses of internal consistency and correlation with Pauli test results were performed to establish test psychometric properties. The results supported adequate internal consistency and convergent validity claim for Qutest. In addition to internal consistency, a second study was conducted to develop further and evaluate parallel-form consistency between mobile and computer versions. In the second study, 371 participants completed a maximum of 192-page Qutest with a similar amount of time to the previous study. Participants took the mobile version first and then the computer versions of the test with a minimum of one lag day. Statistically significant correlation index of results across devices supported parallel-form reliability of both mobile and laptop/desktop Qutest version. A follow-up analysis of answering aids (mouse, trackpad, touch screen) for the computer version recommended using a mouse or touch screen instead of a trackpad due to statistically significant difference in performance among the three tools. Thus, Qutest can be used as a psychometrically adequate test and be flexible in various devices to assess attention and willpower for (prospective) employee screening.
Keywords
References
Bates, M. E., & Lemay, Jr., E. P. (2004). The d2 test of attention: Construct validity and extensions in scoring techniques. Journal of the International Neuropsychological Society, 10, 392–400, https://doi.org/10.1017/s135561770410307X
Blotenberg, I, & Schmidt-Atzert, L. (2019). On the locus of the practice effect in sustained attention tests. Journal of Intelligence, 7(2), 12-26, https://doi.org/10.3390/jintelligence7020012
Brickenkamp, R., & Zillmer, E. (1998). The d2: Test of attention. Hogrefe & Huber Publishers.
Chadha, N. K. (2009). Applied psychometry. SAGE Publications India Pvt Ltd
Cohen, J., LaRue, C., & Cohen, H. H. (2017). Attention interrupted: Cognitive distraction & workplace safety. Professional Safety, 62(11), 28-34.
Ghosh, A., Nafalski, A., Tweedale, J., & Dollard, M. (2012). Using hybridized techniques to develop an online workplace risk assessment tool. Informatyka, Automatyka, Pomiary w Gospodarce i Ochronie Środowiska, 42-45.
Gomez-Ramirez, M., Hysaj, K., & Niebur, E. (2016). Neural mechanisms of selective attention in the somatosensory system. Journal of Neurophysiology, 116(3), 1218-1231, https://doi.org/10.1152/ jn.00637.2015
Grahn, J. A., & Manly, T. (2012). Common neural recruitment across diverse sustained attention tasks. PLoS ONE, 7(11), e49556, https://doi.org/10.1371/journal.pone.0049556
Ha, K. S., Kwon, J. S., & Lyoo, I. K. (2002). Development and standardization of the computerized attention assessment for Korean adults. Journal of Korean Neuropsychiatric Association, 335-346.
Hasanah, M. N., & Herliansyah, M. K. (2016). Analisis tes kecepatan dan ketelitian kerja yang representatif pada kondisi ruang kerja ideal. Undergraduate Thesis, Industrial Engineering Department, Universitas Gadjah Mada.
Langsrud, Ø. (2003). ANOVA for unbalanced data: Use Type II instead of Type III sums of squares. Statistics and Computing, 13(2), 163-167.
Lee, P., Lu, W-S., Liu, C-H., Lin, H-Y., & Hsieh, C-L. (2018). Test–retest reliability and minimal detectable change of the d2 test of attention in patients with schizophrenia. Archives of Clinical Neuropsychology, 33(8), 1060–1068, https://doi.org/10.1093/arclin/acx123
Lord, W., & Clarke, P. (2005). Use of the d2 test of attention as a predictor of driving proficiency. In L. Dorn. (ed.), Driver behaviour and training (first edition). (pp. 437-442). Routledge.
Lozano, J. H., Capote, E., & Fernández, M. P. (2015). Convergent validity of the measures of attention and impulsivity in the Trees: Simple Visual Discrimination Test (DiViSA-UAM). Anales de Psicologίa/Annals of Psychology, 31(1), 74–83. https://doi.org/10.6018/analesps.31.1.161371
Luf, D., & Fichman, N. (2012). Development and use of a computerized test, MATH—CPT, to assess attention. Perceptual and Motor Skills, 114(1), 59-74, https://doi.org/10.2466/08.11.22.PMS.114.1.59-74
Pradipta, R. T. D., Hidayat, R. (2019). Konvergensi dan divergensi tes kraepelin dan tes pauli pada komunitas psikolog. Master Thesis, Professional Program of Psychology, Universitas Gadjah Mada.
Raghavendra, B. R., & Telles, S. (2012). Performance in attentional tasks following meditative focusing and focusing without meditation. Ancient Science of Life, 32(1), 49–53. https://doi.org/10.4103/0257-7941.113799
Reyes-Parra, P., Jimenez-Martinez, M., & Lara-Baez, C. (2017). Selective and sustained attention of workers in two different shifts of the day. Psychologia, 11(2), 113-120, https://doi.org/10.21500/19002386.3059
Santacreu, J., & Quiroga, M. A. (2015). The DiViSA's predictive validity by age: An objective online test of attention, The Spanish Journal of Psychology, 18, 1–11, https://doi.org/10.1017/sjp.2015.103
Steinborn, M. B., Langner, R., Flehmig, H. C., & Huestegge, L. (2018). Methodology of performance scoring in the d2 sustained-attention test: Cumulative-reliability functions and practical guidelines. Psychological Assessment, 30(3), 339-359, https://doi.org/10.1037/pas0000482.
DOI: 10.15408/jp3i.v11i1.22352
Refbacks
- There are currently no refbacks.
Copyright (c) 2022 Iqbal Maesa Febriawan
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
