Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation
The autonomous decision-making model for ship navigation requires extensive interaction and trial-and-error in real, complex environments to ensure optimal decision-making performance and efficiency across various scenarios. However, existing approaches still encounter significant challenges in addr...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Journal of Marine Science and Engineering |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2077-1312/13/3/514 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849342929915609088 |
|---|---|
| author | Ruolan Zhang Xinyu Qin Mingyang Pan Shaoxi Li Helong Shen |
| author_facet | Ruolan Zhang Xinyu Qin Mingyang Pan Shaoxi Li Helong Shen |
| author_sort | Ruolan Zhang |
| collection | DOAJ |
| description | The autonomous decision-making model for ship navigation requires extensive interaction and trial-and-error in real, complex environments to ensure optimal decision-making performance and efficiency across various scenarios. However, existing approaches still encounter significant challenges in addressing the temporal features of state space and tackling complex dynamic collision avoidance tasks, primarily due to factors such as environmental uncertainty, the high dimensionality of the state space, and limited decision robustness. This paper proposes an adaptive temporal decision-making model based on reinforcement learning, which utilizes Long Short-Term Memory (LSTM) networks to capture temporal features of the state space. The model integrates an enhanced Proximal Policy Optimization (PPO) algorithm for efficient policy iteration optimization. Additionally, a simulation training environment is constructed, incorporating multi-factor coupled physical properties and ship dynamics equations. The environment maps variables such as wind speed, current velocity, and wave height, along with dynamic ship parameters, while considering the International Regulations for Preventing Collisions at Sea (COLREGs) in training the autonomous navigation decision-making model. Experimental results demonstrate that, compared to other neural network-based reinforcement learning methods, the proposed model excels in environmental adaptability, collision avoidance success rate, navigation stability, and trajectory optimization. The model’s decision resilience and state-space mapping align with real-world navigation scenarios, significantly improving the autonomous decision-making capability of ships in dynamic sea conditions and providing critical support for the advancement of intelligent shipping. |
| format | Article |
| id | doaj-art-9f02f56c86b04d31a6e0461b267f7ca3 |
| institution | Kabale University |
| issn | 2077-1312 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Journal of Marine Science and Engineering |
| spelling | doaj-art-9f02f56c86b04d31a6e0461b267f7ca32025-08-20T03:43:12ZengMDPI AGJournal of Marine Science and Engineering2077-13122025-03-0113351410.3390/jmse13030514Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship NavigationRuolan Zhang0Xinyu Qin1Mingyang Pan2Shaoxi Li3Helong Shen4Navigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaNavigation College, Dalian Maritime University, Dalian 116026, ChinaThe autonomous decision-making model for ship navigation requires extensive interaction and trial-and-error in real, complex environments to ensure optimal decision-making performance and efficiency across various scenarios. However, existing approaches still encounter significant challenges in addressing the temporal features of state space and tackling complex dynamic collision avoidance tasks, primarily due to factors such as environmental uncertainty, the high dimensionality of the state space, and limited decision robustness. This paper proposes an adaptive temporal decision-making model based on reinforcement learning, which utilizes Long Short-Term Memory (LSTM) networks to capture temporal features of the state space. The model integrates an enhanced Proximal Policy Optimization (PPO) algorithm for efficient policy iteration optimization. Additionally, a simulation training environment is constructed, incorporating multi-factor coupled physical properties and ship dynamics equations. The environment maps variables such as wind speed, current velocity, and wave height, along with dynamic ship parameters, while considering the International Regulations for Preventing Collisions at Sea (COLREGs) in training the autonomous navigation decision-making model. Experimental results demonstrate that, compared to other neural network-based reinforcement learning methods, the proposed model excels in environmental adaptability, collision avoidance success rate, navigation stability, and trajectory optimization. The model’s decision resilience and state-space mapping align with real-world navigation scenarios, significantly improving the autonomous decision-making capability of ships in dynamic sea conditions and providing critical support for the advancement of intelligent shipping.https://www.mdpi.com/2077-1312/13/3/514Maritime Autonomous Surface Shipsstate space mappingProximal Policy Optimizationdynamic collision avoidancedecision-making resilience |
| spellingShingle | Ruolan Zhang Xinyu Qin Mingyang Pan Shaoxi Li Helong Shen Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation Journal of Marine Science and Engineering Maritime Autonomous Surface Ships state space mapping Proximal Policy Optimization dynamic collision avoidance decision-making resilience |
| title | Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation |
| title_full | Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation |
| title_fullStr | Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation |
| title_full_unstemmed | Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation |
| title_short | Adaptive Temporal Reinforcement Learning for Mapping Complex Maritime Environmental State Spaces in Autonomous Ship Navigation |
| title_sort | adaptive temporal reinforcement learning for mapping complex maritime environmental state spaces in autonomous ship navigation |
| topic | Maritime Autonomous Surface Ships state space mapping Proximal Policy Optimization dynamic collision avoidance decision-making resilience |
| url | https://www.mdpi.com/2077-1312/13/3/514 |
| work_keys_str_mv | AT ruolanzhang adaptivetemporalreinforcementlearningformappingcomplexmaritimeenvironmentalstatespacesinautonomousshipnavigation AT xinyuqin adaptivetemporalreinforcementlearningformappingcomplexmaritimeenvironmentalstatespacesinautonomousshipnavigation AT mingyangpan adaptivetemporalreinforcementlearningformappingcomplexmaritimeenvironmentalstatespacesinautonomousshipnavigation AT shaoxili adaptivetemporalreinforcementlearningformappingcomplexmaritimeenvironmentalstatespacesinautonomousshipnavigation AT helongshen adaptivetemporalreinforcementlearningformappingcomplexmaritimeenvironmentalstatespacesinautonomousshipnavigation |