Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
Objectives To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency.Setting Swiss National Science Foundation (SNSF).Design Cross-sectional analysis of peer review reports submitted from 2009 to 2...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2020-08-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/10/8/e035058.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846142500168794112 |
|---|---|
| author | Joao Martins Matthias Egger Anna Severin Rachel Heyard François Delavy Anne Jorstad |
| author_facet | Joao Martins Matthias Egger Anna Severin Rachel Heyard François Delavy Anne Jorstad |
| author_sort | Joao Martins |
| collection | DOAJ |
| description | Objectives To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency.Setting Swiss National Science Foundation (SNSF).Design Cross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression models adjusted for research topic, applicant’s age, nationality, affiliation and calendar period.Participants External peer reviewers.Primary outcome measure Overall score on a scale from 1 (worst) to 6 (best).Results Analyses included 38 250 reports on 12 294 grant applications from medicine, architecture, biology, chemistry, economics, engineering, geology, history, linguistics, mathematics, physics, psychology and sociology submitted by 26 829 unique peer reviewers. In univariable analysis, male applicants received more favourable evaluation scores than female applicants (+0.18 points; 95% CI 0.14 to 0.23), and male reviewers awarded higher scores than female reviewers (+0.11; 95% CI 0.08 to 0.15). Applicant-nominated reviewers awarded higher scores than reviewers nominated by the SNSF (+0.53; 95% CI 0.50 to 0.56), and reviewers from outside of Switzerland more favourable scores than reviewers affiliated with Swiss institutions (+0.53; 95% CI 0.49 to 0.56). In multivariable analysis, differences between male and female applicants were attenuated (+0.08; 95% CI 0.04 to 0.13) whereas results changed little for source of nomination and affiliation of reviewers. The gender difference increased after September 2011, when new evaluation forms were introduced (p=0.033 from test of interaction).Conclusions Peer review of grant applications at SNSF might be prone to biases stemming from different applicant and reviewer characteristics. The SNSF abandoned the nomination of peer reviewers by applicants. The new form introduced in 2011 may inadvertently have given more emphasis to the applicant’s track record. We encourage other funders to conduct similar studies, in order to improve the evidence base for rational and fair research funding. |
| format | Article |
| id | doaj-art-a84d9b5fafdb45c2bfb89d497ef515c5 |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2020-08-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-a84d9b5fafdb45c2bfb89d497ef515c52024-12-03T10:05:08ZengBMJ Publishing GroupBMJ Open2044-60552020-08-0110810.1136/bmjopen-2019-035058Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reportsJoao Martins0Matthias Egger1Anna Severin2Rachel Heyard3François Delavy4Anne Jorstad5MedicilLisboa, Lisboa, PortugalResearch Council, Schweizerischer Nationalfonds zur Förderung der wissenschaftlichen Forschung, Bern, SwitzerlandGraduate School for Health Sciences, University of Bern, Bern, SwitzerlandCenter for Reproducible Science, Epidemiology, Biostatistics and Prevention Institute, University of Zurich, Zurich, SwitzerlandStrategy Support, Swiss National Science Foundation, Bern, SwitzerlandData Team, Swiss National Science Foundation, Bern, SwitzerlandObjectives To examine whether the gender of applicants and peer reviewers and other factors influence peer review of grant proposals submitted to a national funding agency.Setting Swiss National Science Foundation (SNSF).Design Cross-sectional analysis of peer review reports submitted from 2009 to 2016 using linear mixed effects regression models adjusted for research topic, applicant’s age, nationality, affiliation and calendar period.Participants External peer reviewers.Primary outcome measure Overall score on a scale from 1 (worst) to 6 (best).Results Analyses included 38 250 reports on 12 294 grant applications from medicine, architecture, biology, chemistry, economics, engineering, geology, history, linguistics, mathematics, physics, psychology and sociology submitted by 26 829 unique peer reviewers. In univariable analysis, male applicants received more favourable evaluation scores than female applicants (+0.18 points; 95% CI 0.14 to 0.23), and male reviewers awarded higher scores than female reviewers (+0.11; 95% CI 0.08 to 0.15). Applicant-nominated reviewers awarded higher scores than reviewers nominated by the SNSF (+0.53; 95% CI 0.50 to 0.56), and reviewers from outside of Switzerland more favourable scores than reviewers affiliated with Swiss institutions (+0.53; 95% CI 0.49 to 0.56). In multivariable analysis, differences between male and female applicants were attenuated (+0.08; 95% CI 0.04 to 0.13) whereas results changed little for source of nomination and affiliation of reviewers. The gender difference increased after September 2011, when new evaluation forms were introduced (p=0.033 from test of interaction).Conclusions Peer review of grant applications at SNSF might be prone to biases stemming from different applicant and reviewer characteristics. The SNSF abandoned the nomination of peer reviewers by applicants. The new form introduced in 2011 may inadvertently have given more emphasis to the applicant’s track record. We encourage other funders to conduct similar studies, in order to improve the evidence base for rational and fair research funding.https://bmjopen.bmj.com/content/10/8/e035058.full |
| spellingShingle | Joao Martins Matthias Egger Anna Severin Rachel Heyard François Delavy Anne Jorstad Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports BMJ Open |
| title | Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports |
| title_full | Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports |
| title_fullStr | Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports |
| title_full_unstemmed | Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports |
| title_short | Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports |
| title_sort | gender and other potential biases in peer review cross sectional analysis of 38 250 external peer review reports |
| url | https://bmjopen.bmj.com/content/10/8/e035058.full |
| work_keys_str_mv | AT joaomartins genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports AT matthiasegger genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports AT annaseverin genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports AT rachelheyard genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports AT francoisdelavy genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports AT annejorstad genderandotherpotentialbiasesinpeerreviewcrosssectionalanalysisof38250externalpeerreviewreports |