Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons
Objectives Ordinal scales provide means for communicating the severity of a condition, but they are affected by cognitive biases, they introduce statistical problems and they sacrifice resolution. Clinicians discern more details than contained in scales, for example, when assessing radiographs, but...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
BMJ Publishing Group
2021-03-01
|
| Series: | BMJ Open |
| Online Access: | https://bmjopen.bmj.com/content/11/3/e041793.full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846161911278731264 |
|---|---|
| author | Mads Møller Pedersen Kristian Breds Geoffroy Mongelard Anne Mørup-Petersen Karl Bang Christensen Anders Odgaard |
| author_facet | Mads Møller Pedersen Kristian Breds Geoffroy Mongelard Anne Mørup-Petersen Karl Bang Christensen Anders Odgaard |
| author_sort | Mads Møller Pedersen |
| collection | DOAJ |
| description | Objectives Ordinal scales provide means for communicating the severity of a condition, but they are affected by cognitive biases, they introduce statistical problems and they sacrifice resolution. Clinicians discern more details than contained in scales, for example, when assessing radiographs, but clinicians’ distinctions are often based on experience-based rules of thumb, that is, heuristics. The objectives of this study are to compare clinicians’ heuristic assessments to ordinal grading, to identify case elements that influence clinicians’ judgements and to present a method for quantifying heuristic assessments.Design Clinicians were presented with 17 207 random pairs from a set of 1087 knee radiographs. For each pair, the radiograph with more severe osteoarthritis was selected. The Bradley-Terry model was used to calculate an osteoarthritis strength parameter for each radiograph. Similarly, strength parameters were determined for 12 morphological features with five additional features being considered either present or absent. All radiographs were also graded according to conventional ordinal systems (Kellgren-Lawrence and Ahlbäck). Relations between clinicians’ judgements and (1) the heuristics-based osteoarthritis strength, (2) conventional ordinal systems and (3) morphological features were investigated.Results Receiver operating characteristic analysis showed that the Bradley-Terry model provided a good description of clinicians’ assessments (area under the curve (AUC)=0.97, 95% CI 0.968 to 0.972). Morphological features (AUC=0.90, 95% CI 0.900 to 0.908) provided a superior description of clinicians’ choices compared with conventional ordinal systems (AUC=0.88, 95% CI 0.878 to 0.887 and AUC=0.80, 95% CI 0.796 to 0.809) for Ahlbäck and Kellgren-Lawrence, respectively). The features most strongly associated with osteoarthritis strength were medial joint space width, flattening of the medial femoral and tibial condyles, medial osteophytes and alignment.Conclusions Heuristics-based assessments give a better distinction than conventional grading systems of knee osteoarthritis. The example presents a general approach to evaluate which features are part of experts’ heuristics. The data suggest that experts discern more details than included in conventional ordinal grading systems. Quantitative heuristic assessments may replace ordinal scales. |
| format | Article |
| id | doaj-art-947a751e069b4b78b324b152f3beed15 |
| institution | Kabale University |
| issn | 2044-6055 |
| language | English |
| publishDate | 2021-03-01 |
| publisher | BMJ Publishing Group |
| record_format | Article |
| series | BMJ Open |
| spelling | doaj-art-947a751e069b4b78b324b152f3beed152024-11-21T00:55:07ZengBMJ Publishing GroupBMJ Open2044-60552021-03-0111310.1136/bmjopen-2020-041793Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisonsMads Møller Pedersen0Kristian Breds Geoffroy Mongelard1Anne Mørup-Petersen2Karl Bang Christensen3Anders Odgaard4Section of Biostatistics, University of Copenhagen, Copenhagen, DenmarkDepartment of Radiology, Copenhagen University Hospital Herlev-Gentofte, Copenhagen, DenmarkDepartment of Orthopedic Surgery, Copenhagen University Hospital Herlev-Gentofte, Copenhagen, DenmarkSection of Biostatistics, University of Copenhagen, Copenhagen, Denmark3Herlev and Gentofte University Hospital, Department of Orthopaedic Surgery, Hellerup, DenmarkObjectives Ordinal scales provide means for communicating the severity of a condition, but they are affected by cognitive biases, they introduce statistical problems and they sacrifice resolution. Clinicians discern more details than contained in scales, for example, when assessing radiographs, but clinicians’ distinctions are often based on experience-based rules of thumb, that is, heuristics. The objectives of this study are to compare clinicians’ heuristic assessments to ordinal grading, to identify case elements that influence clinicians’ judgements and to present a method for quantifying heuristic assessments.Design Clinicians were presented with 17 207 random pairs from a set of 1087 knee radiographs. For each pair, the radiograph with more severe osteoarthritis was selected. The Bradley-Terry model was used to calculate an osteoarthritis strength parameter for each radiograph. Similarly, strength parameters were determined for 12 morphological features with five additional features being considered either present or absent. All radiographs were also graded according to conventional ordinal systems (Kellgren-Lawrence and Ahlbäck). Relations between clinicians’ judgements and (1) the heuristics-based osteoarthritis strength, (2) conventional ordinal systems and (3) morphological features were investigated.Results Receiver operating characteristic analysis showed that the Bradley-Terry model provided a good description of clinicians’ assessments (area under the curve (AUC)=0.97, 95% CI 0.968 to 0.972). Morphological features (AUC=0.90, 95% CI 0.900 to 0.908) provided a superior description of clinicians’ choices compared with conventional ordinal systems (AUC=0.88, 95% CI 0.878 to 0.887 and AUC=0.80, 95% CI 0.796 to 0.809) for Ahlbäck and Kellgren-Lawrence, respectively). The features most strongly associated with osteoarthritis strength were medial joint space width, flattening of the medial femoral and tibial condyles, medial osteophytes and alignment.Conclusions Heuristics-based assessments give a better distinction than conventional grading systems of knee osteoarthritis. The example presents a general approach to evaluate which features are part of experts’ heuristics. The data suggest that experts discern more details than included in conventional ordinal grading systems. Quantitative heuristic assessments may replace ordinal scales.https://bmjopen.bmj.com/content/11/3/e041793.full |
| spellingShingle | Mads Møller Pedersen Kristian Breds Geoffroy Mongelard Anne Mørup-Petersen Karl Bang Christensen Anders Odgaard Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons BMJ Open |
| title | Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons |
| title_full | Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons |
| title_fullStr | Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons |
| title_full_unstemmed | Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons |
| title_short | Clinicians’ heuristic assessments of radiographs compared with Kellgren-Lawrence and Ahlbäck ordinal grading: an exploratory study of knee radiographs using paired comparisons |
| title_sort | clinicians heuristic assessments of radiographs compared with kellgren lawrence and ahlback ordinal grading an exploratory study of knee radiographs using paired comparisons |
| url | https://bmjopen.bmj.com/content/11/3/e041793.full |
| work_keys_str_mv | AT madsmøllerpedersen cliniciansheuristicassessmentsofradiographscomparedwithkellgrenlawrenceandahlbackordinalgradinganexploratorystudyofkneeradiographsusingpairedcomparisons AT kristianbredsgeoffroymongelard cliniciansheuristicassessmentsofradiographscomparedwithkellgrenlawrenceandahlbackordinalgradinganexploratorystudyofkneeradiographsusingpairedcomparisons AT annemøruppetersen cliniciansheuristicassessmentsofradiographscomparedwithkellgrenlawrenceandahlbackordinalgradinganexploratorystudyofkneeradiographsusingpairedcomparisons AT karlbangchristensen cliniciansheuristicassessmentsofradiographscomparedwithkellgrenlawrenceandahlbackordinalgradinganexploratorystudyofkneeradiographsusingpairedcomparisons AT andersodgaard cliniciansheuristicassessmentsofradiographscomparedwithkellgrenlawrenceandahlbackordinalgradinganexploratorystudyofkneeradiographsusingpairedcomparisons |