Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inap...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Algorithms |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-4893/17/12/556 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846106308546134016 |
|---|---|
| author | Saeed Ur Rehman Anwar Ali Adil Mehmood Khan Cynthia Okpala |
| author_facet | Saeed Ur Rehman Anwar Ali Adil Mehmood Khan Cynthia Okpala |
| author_sort | Saeed Ur Rehman |
| collection | DOAJ |
| description | With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems. |
| format | Article |
| id | doaj-art-e2308b02075d406ca3548f159c93f454 |
| institution | Kabale University |
| issn | 1999-4893 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Algorithms |
| spelling | doaj-art-e2308b02075d406ca3548f159c93f4542024-12-27T14:05:13ZengMDPI AGAlgorithms1999-48932024-12-01171255610.3390/a17120556Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable SensorsSaeed Ur Rehman0Anwar Ali1Adil Mehmood Khan2Cynthia Okpala3Faculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UKDepartment of Electronic and Electrical Engineering, Swansea University Bay Campus, Swansea SA1 8EN, UKFaculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UKFaculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UKWith the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.https://www.mdpi.com/1999-4893/17/12/556machine learningLOSOhuman activity recognition |
| spellingShingle | Saeed Ur Rehman Anwar Ali Adil Mehmood Khan Cynthia Okpala Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors Algorithms machine learning LOSO human activity recognition |
| title | Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors |
| title_full | Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors |
| title_fullStr | Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors |
| title_full_unstemmed | Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors |
| title_short | Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors |
| title_sort | human activity recognition a comparative study of validation methods and impact of feature extraction in wearable sensors |
| topic | machine learning LOSO human activity recognition |
| url | https://www.mdpi.com/1999-4893/17/12/556 |
| work_keys_str_mv | AT saeedurrehman humanactivityrecognitionacomparativestudyofvalidationmethodsandimpactoffeatureextractioninwearablesensors AT anwarali humanactivityrecognitionacomparativestudyofvalidationmethodsandimpactoffeatureextractioninwearablesensors AT adilmehmoodkhan humanactivityrecognitionacomparativestudyofvalidationmethodsandimpactoffeatureextractioninwearablesensors AT cynthiaokpala humanactivityrecognitionacomparativestudyofvalidationmethodsandimpactoffeatureextractioninwearablesensors |