Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods
Abstract BackgroundArtificial intelligence (AI) is rapidly being adopted to build products and aid in the decision-making process across industries. However, AI systems have been shown to exhibit and even amplify biases, causing a growing concern among people worldwide. Thus,...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2024-11-01
|
| Series: | JMIR Medical Informatics |
| Online Access: | https://medinform.jmir.org/2024/1/e60272 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846160026823032832 |
|---|---|
| author | Magnus Gray Mariofanna Milanova Leihong Wu |
| author_facet | Magnus Gray Mariofanna Milanova Leihong Wu |
| author_sort | Magnus Gray |
| collection | DOAJ |
| description |
Abstract
BackgroundArtificial intelligence (AI) is rapidly being adopted to build products and aid in the decision-making process across industries. However, AI systems have been shown to exhibit and even amplify biases, causing a growing concern among people worldwide. Thus, investigating methods of measuring and mitigating bias within these AI-powered tools is necessary.
ObjectiveIn natural language processing applications, the word embedding association test (WEAT) is a popular method of measuring bias in input embeddings, a common area of measure bias in AI. However, certain limitations of the WEAT have been identified (ie, their nonrobust measure of bias and their reliance on predefined and limited groups of words or sentences), which may lead to inadequate measurements and evaluations of bias. Thus, this study takes a new approach at modifying this popular measure of bias, with a focus on making it more robust and applicable in other domains.
MethodsIn this study, we introduce the SD-WEAT, which is a modified version of the WEAT that uses the SD of multiple permutations of the WEATs to calculate bias in input embeddings. With the SD-WEAT, we evaluated the biases and stability of several language embedding models, including Global Vectors for Word Representation (GloVe), Word2Vec, and bidirectional encoder representations from transformers (BERT).
ResultsThis method produces results comparable to those of the WEAT, with strong correlations between the methods’ bias scores or effect sizes (rPr
ConclusionsThus, the SD-WEAT shows promise for robustly measuring bias in the input embeddings fed to AI language models. |
| format | Article |
| id | doaj-art-e8f9317ca64643c9abf1824c2ca28d6a |
| institution | Kabale University |
| issn | 2291-9694 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Informatics |
| spelling | doaj-art-e8f9317ca64643c9abf1824c2ca28d6a2024-11-22T15:01:15ZengJMIR PublicationsJMIR Medical Informatics2291-96942024-11-0112e60272e6027210.2196/60272Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of MethodsMagnus Grayhttp://orcid.org/0009-0009-6355-7627Mariofanna Milanovahttp://orcid.org/0000-0003-0995-1921Leihong Wuhttp://orcid.org/0000-0002-4093-3708 Abstract BackgroundArtificial intelligence (AI) is rapidly being adopted to build products and aid in the decision-making process across industries. However, AI systems have been shown to exhibit and even amplify biases, causing a growing concern among people worldwide. Thus, investigating methods of measuring and mitigating bias within these AI-powered tools is necessary. ObjectiveIn natural language processing applications, the word embedding association test (WEAT) is a popular method of measuring bias in input embeddings, a common area of measure bias in AI. However, certain limitations of the WEAT have been identified (ie, their nonrobust measure of bias and their reliance on predefined and limited groups of words or sentences), which may lead to inadequate measurements and evaluations of bias. Thus, this study takes a new approach at modifying this popular measure of bias, with a focus on making it more robust and applicable in other domains. MethodsIn this study, we introduce the SD-WEAT, which is a modified version of the WEAT that uses the SD of multiple permutations of the WEATs to calculate bias in input embeddings. With the SD-WEAT, we evaluated the biases and stability of several language embedding models, including Global Vectors for Word Representation (GloVe), Word2Vec, and bidirectional encoder representations from transformers (BERT). ResultsThis method produces results comparable to those of the WEAT, with strong correlations between the methods’ bias scores or effect sizes (rPr ConclusionsThus, the SD-WEAT shows promise for robustly measuring bias in the input embeddings fed to AI language models.https://medinform.jmir.org/2024/1/e60272 |
| spellingShingle | Magnus Gray Mariofanna Milanova Leihong Wu Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods JMIR Medical Informatics |
| title | Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods |
| title_full | Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods |
| title_fullStr | Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods |
| title_full_unstemmed | Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods |
| title_short | Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods |
| title_sort | enhancing bias assessment for complex term groups in language embedding models quantitative comparison of methods |
| url | https://medinform.jmir.org/2024/1/e60272 |
| work_keys_str_mv | AT magnusgray enhancingbiasassessmentforcomplextermgroupsinlanguageembeddingmodelsquantitativecomparisonofmethods AT mariofannamilanova enhancingbiasassessmentforcomplextermgroupsinlanguageembeddingmodelsquantitativecomparisonofmethods AT leihongwu enhancingbiasassessmentforcomplextermgroupsinlanguageembeddingmodelsquantitativecomparisonofmethods |