Identifying quality responses using an analysis of response times: the RTcutoff function in R
IntroductionThe present study aims to develop an R function to develop and visualize thresholds that describe the response time of individuals concerning their sample. The function utilizes the cumulative proportion correct (CUMP) approach, to estimate item-specific time threshold, which originated...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-11-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1479249/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846170514418040832 |
|---|---|
| author | Georgios Sideridis Georgios Sideridis Mohammed Alghamdi |
| author_facet | Georgios Sideridis Georgios Sideridis Mohammed Alghamdi |
| author_sort | Georgios Sideridis |
| collection | DOAJ |
| description | IntroductionThe present study aims to develop an R function to develop and visualize thresholds that describe the response time of individuals concerning their sample. The function utilizes the cumulative proportion correct (CUMP) approach, to estimate item-specific time threshold, which originated in the work of Guo and his colleagues. Besides the CUMP approach, the present function presents response time profiles on a measure using the mean of the sample and + 1SD times so that it can discern between thoughtful engagement and processing with an item (termed problem-solving behavior) and rapid responding, guessing, and disengagement with the test. The advantage of the CUMP model utilized here is that it simultaneously engages both response time and response correctness to establish thresholds that differentiate engaged from disengaged participants.MethodsGiven data on a measure of reading comprehension for students in Saudi Arabia (n = 494) using the Progress in International Reading Literacy Study (PIRLS 2021) international assessment, high and low-achieving individuals that engaged in different behavior patterns were identified and plotted against their sample.Results and conclusionResults pointed to the importance and necessity of the RTcutoff function to identify variable forms of engagement that have implications for person-score validity but also have implications for test validity and the need to increase measurement precisio. |
| format | Article |
| id | doaj-art-36b2597e7a2e431b9a3b7bc0d41cc811 |
| institution | Kabale University |
| issn | 1664-1078 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj-art-36b2597e7a2e431b9a3b7bc0d41cc8112024-11-11T17:11:00ZengFrontiers Media S.A.Frontiers in Psychology1664-10782024-11-011510.3389/fpsyg.2024.14792491479249Identifying quality responses using an analysis of response times: the RTcutoff function in RGeorgios Sideridis0Georgios Sideridis1Mohammed Alghamdi2Boston Children’s Hospital, Harvard Medical School, Boston, MA, United StatesNational and Kapodistrian University of Athens, Athens, GreeceDepartment of Self-Development Skills, King Saud University, Riyadh, Saudi ArabiaIntroductionThe present study aims to develop an R function to develop and visualize thresholds that describe the response time of individuals concerning their sample. The function utilizes the cumulative proportion correct (CUMP) approach, to estimate item-specific time threshold, which originated in the work of Guo and his colleagues. Besides the CUMP approach, the present function presents response time profiles on a measure using the mean of the sample and + 1SD times so that it can discern between thoughtful engagement and processing with an item (termed problem-solving behavior) and rapid responding, guessing, and disengagement with the test. The advantage of the CUMP model utilized here is that it simultaneously engages both response time and response correctness to establish thresholds that differentiate engaged from disengaged participants.MethodsGiven data on a measure of reading comprehension for students in Saudi Arabia (n = 494) using the Progress in International Reading Literacy Study (PIRLS 2021) international assessment, high and low-achieving individuals that engaged in different behavior patterns were identified and plotted against their sample.Results and conclusionResults pointed to the importance and necessity of the RTcutoff function to identify variable forms of engagement that have implications for person-score validity but also have implications for test validity and the need to increase measurement precisio.https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1479249/fullresponse timerapid guessingpsychometricstest validityresponse accuracyCUMP method |
| spellingShingle | Georgios Sideridis Georgios Sideridis Mohammed Alghamdi Identifying quality responses using an analysis of response times: the RTcutoff function in R Frontiers in Psychology response time rapid guessing psychometrics test validity response accuracy CUMP method |
| title | Identifying quality responses using an analysis of response times: the RTcutoff function in R |
| title_full | Identifying quality responses using an analysis of response times: the RTcutoff function in R |
| title_fullStr | Identifying quality responses using an analysis of response times: the RTcutoff function in R |
| title_full_unstemmed | Identifying quality responses using an analysis of response times: the RTcutoff function in R |
| title_short | Identifying quality responses using an analysis of response times: the RTcutoff function in R |
| title_sort | identifying quality responses using an analysis of response times the rtcutoff function in r |
| topic | response time rapid guessing psychometrics test validity response accuracy CUMP method |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1479249/full |
| work_keys_str_mv | AT georgiossideridis identifyingqualityresponsesusingananalysisofresponsetimesthertcutofffunctioninr AT georgiossideridis identifyingqualityresponsesusingananalysisofresponsetimesthertcutofffunctioninr AT mohammedalghamdi identifyingqualityresponsesusingananalysisofresponsetimesthertcutofffunctioninr |