The human-on-the-loop evaluation of multiobjective optimization algorithms for solving a real-world problem that integrates the food-energy-water nexus security and climate change vulnerability
Human-on-the-loop (HOTL) evaluation enables decision-makers to provide qualitative insights that support standard performance metrics. While these metrics are designed to assess convergence and diversity of solutions using statistical measures, they often fail to grasp problem-specific characteristi...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-09-01
|
| Series: | Results in Control and Optimization |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2666720725000918 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Human-on-the-loop (HOTL) evaluation enables decision-makers to provide qualitative insights that support standard performance metrics. While these metrics are designed to assess convergence and diversity of solutions using statistical measures, they often fail to grasp problem-specific characteristics inherent in noisy and uncertain real-world problems, subsequently generating inaccurate or uninterpretable numerical results. Additionally, the lack of qualitative insights and decision-maker preferences in performance evaluation compromises the effectiveness of identifying the appropriate algorithm to solve a specific real-world multiobjective optimization problem (MOP). This research applies graphical analysis, examination of objectives, and decision variable values to complement statistical analysis. The HOTL concept is explored as an “Interaction after a complete run (IAR)” mechanism for evaluating optimization and performance results through human intervention. It is achieved by implementing the inter-rater reliability (IRR) evaluation facilitated by computing Krippendorff’s alpha scores. Based on the results, the Fuzzy Decision Variable (FDV) algorithm is selected as the most suitable algorithm to solve a multiobjective problem that integrates the Food-Energy-Water Nexus (FEWN) and Climate Change Vulnerability (CCV). The findings highlight the HOTL's importance in addressing the shortcomings of the existing performance metrics, such as ambiguous and inaccurate results. Based on the identified knowledge gaps, the findings, and the limitations of this research, we propose future research areas that can be undertaken to improve the performance evaluation of algorithms. Such research areas include incorporating machine learning to predict, using performance data, the most suitable algorithm to solve a specific problem, and advancing interactive learning and user adjustments of the optimization process. |
|---|---|
| ISSN: | 2666-7207 |