Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system.
<h4>Background</h4>A new undergraduate medical programme was instituted at the University of Botswana in 2009. In 2016, the Faculty of Medicine decided to conduct a comprehensive review of the programme. Participants at a planning workshop decided the review had to lead to an in-depth un...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2024-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0312730 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841555546924122112 |
|---|---|
| author | Detlef R Prozesky Masego B Kebaetse Mpho S Mogodi Mmoloki C Molwantwa |
| author_facet | Detlef R Prozesky Masego B Kebaetse Mpho S Mogodi Mmoloki C Molwantwa |
| author_sort | Detlef R Prozesky |
| collection | DOAJ |
| description | <h4>Background</h4>A new undergraduate medical programme was instituted at the University of Botswana in 2009. In 2016, the Faculty of Medicine decided to conduct a comprehensive review of the programme. Participants at a planning workshop decided the review had to lead to an in-depth understanding of the programme. The challenge was to develop an evaluation process to achieve this aim.<h4>Approach</h4>Standards and theories used in other programme evaluations were investigated. A steering committee developed a 'six-step process' for the review of the programme as a complex system. The process used three evaluation models and various sources of evaluation standards to derive 90 evaluation questions. Quantitative and qualitative data were collected using 96 different instruments, with data triangulation prominently featured. Data analysis used an interpretivist approach. The review process was validated against the Cilliers's ten features of a complex system.<h4>Validation</h4>The review process was validated against Cilliers's ten features of a complex system. We found that the 'six-step process' illuminated each of these features in the MBBS programme in turn, and was therefore a valid way of evaluating this programme as a complex system.<h4>Discussion</h4>In the process of designing and using the 'six-step process' to evaluate a complex medical programme important lessons were learnt: starting the process with complexity theory at the forefront; being as inclusive as possible in data collection; also applying complexity theory to the evaluation of smaller programme components as 'mini-complex systems'; and managing one's inevitable insider bias.<h4>Conclusion and utility</h4>The 'six-step process' as it stands or in adaptation is likely to be useful in similar situations, where evaluators perceive the object of their evaluation to be a complex system, or a component of such a system. |
| format | Article |
| id | doaj-art-11cc186b9b724c1f8ef41424307e9b47 |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-11cc186b9b724c1f8ef41424307e9b472025-01-08T05:32:07ZengPublic Library of Science (PLoS)PLoS ONE1932-62032024-01-011912e031273010.1371/journal.pone.0312730Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system.Detlef R ProzeskyMasego B KebaetseMpho S MogodiMmoloki C Molwantwa<h4>Background</h4>A new undergraduate medical programme was instituted at the University of Botswana in 2009. In 2016, the Faculty of Medicine decided to conduct a comprehensive review of the programme. Participants at a planning workshop decided the review had to lead to an in-depth understanding of the programme. The challenge was to develop an evaluation process to achieve this aim.<h4>Approach</h4>Standards and theories used in other programme evaluations were investigated. A steering committee developed a 'six-step process' for the review of the programme as a complex system. The process used three evaluation models and various sources of evaluation standards to derive 90 evaluation questions. Quantitative and qualitative data were collected using 96 different instruments, with data triangulation prominently featured. Data analysis used an interpretivist approach. The review process was validated against the Cilliers's ten features of a complex system.<h4>Validation</h4>The review process was validated against Cilliers's ten features of a complex system. We found that the 'six-step process' illuminated each of these features in the MBBS programme in turn, and was therefore a valid way of evaluating this programme as a complex system.<h4>Discussion</h4>In the process of designing and using the 'six-step process' to evaluate a complex medical programme important lessons were learnt: starting the process with complexity theory at the forefront; being as inclusive as possible in data collection; also applying complexity theory to the evaluation of smaller programme components as 'mini-complex systems'; and managing one's inevitable insider bias.<h4>Conclusion and utility</h4>The 'six-step process' as it stands or in adaptation is likely to be useful in similar situations, where evaluators perceive the object of their evaluation to be a complex system, or a component of such a system.https://doi.org/10.1371/journal.pone.0312730 |
| spellingShingle | Detlef R Prozesky Masego B Kebaetse Mpho S Mogodi Mmoloki C Molwantwa Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. PLoS ONE |
| title | Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. |
| title_full | Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. |
| title_fullStr | Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. |
| title_full_unstemmed | Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. |
| title_short | Developing an evaluation approach for the in-depth review of a new undergraduate medical programme as a complex system. |
| title_sort | developing an evaluation approach for the in depth review of a new undergraduate medical programme as a complex system |
| url | https://doi.org/10.1371/journal.pone.0312730 |
| work_keys_str_mv | AT detlefrprozesky developinganevaluationapproachfortheindepthreviewofanewundergraduatemedicalprogrammeasacomplexsystem AT masegobkebaetse developinganevaluationapproachfortheindepthreviewofanewundergraduatemedicalprogrammeasacomplexsystem AT mphosmogodi developinganevaluationapproachfortheindepthreviewofanewundergraduatemedicalprogrammeasacomplexsystem AT mmolokicmolwantwa developinganevaluationapproachfortheindepthreviewofanewundergraduatemedicalprogrammeasacomplexsystem |