Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers
Given the rapid adoption of machine learning methods by education researchers, and the growing acknowledgment of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing on an integr...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SAGE Publishing
2024-12-01
|
| Series: | AERA Open |
| Online Access: | https://doi.org/10.1177/23328584241303495 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846116788362805248 |
|---|---|
| author | Kylie Anglin |
| author_facet | Kylie Anglin |
| author_sort | Kylie Anglin |
| collection | DOAJ |
| description | Given the rapid adoption of machine learning methods by education researchers, and the growing acknowledgment of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing on an integrative literature review and extending a well-known framework for theorizing validity in the social sciences, this article provides both an overview of threats to validity in supervised machine learning and plausible approaches for addressing such threats. It collates a list of current best practices, brings supervised learning challenges into a unified conceptual framework, and offers a straightforward reference guide on crucial validity considerations. Finally, it proposes a novel research protocol for researchers to use during project planning and for reviewers and scholars to use when evaluating the validity of supervised machine learning applications. |
| format | Article |
| id | doaj-art-06e12c2789ef4d4388c604ce255ac0db |
| institution | Kabale University |
| issn | 2332-8584 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | SAGE Publishing |
| record_format | Article |
| series | AERA Open |
| spelling | doaj-art-06e12c2789ef4d4388c604ce255ac0db2024-12-18T14:04:07ZengSAGE PublishingAERA Open2332-85842024-12-011010.1177/23328584241303495Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education ResearchersKylie AnglinGiven the rapid adoption of machine learning methods by education researchers, and the growing acknowledgment of their inherent risks, there is an urgent need for tailored methodological guidance on how to improve and evaluate the validity of inferences drawn from these methods. Drawing on an integrative literature review and extending a well-known framework for theorizing validity in the social sciences, this article provides both an overview of threats to validity in supervised machine learning and plausible approaches for addressing such threats. It collates a list of current best practices, brings supervised learning challenges into a unified conceptual framework, and offers a straightforward reference guide on crucial validity considerations. Finally, it proposes a novel research protocol for researchers to use during project planning and for reviewers and scholars to use when evaluating the validity of supervised machine learning applications.https://doi.org/10.1177/23328584241303495 |
| spellingShingle | Kylie Anglin Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers AERA Open |
| title | Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers |
| title_full | Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers |
| title_fullStr | Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers |
| title_full_unstemmed | Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers |
| title_short | Addressing Threats to Validity in Supervised Machine Learning: A Framework and Best Practices for Education Researchers |
| title_sort | addressing threats to validity in supervised machine learning a framework and best practices for education researchers |
| url | https://doi.org/10.1177/23328584241303495 |
| work_keys_str_mv | AT kylieanglin addressingthreatstovalidityinsupervisedmachinelearningaframeworkandbestpracticesforeducationresearchers |