Robust reinforcement learning algorithm based on pigeon-inspired optimization
Reinforcement learning(RL) is an artificial intelligence algorithm with the advantages of clear calculation logic and easy expansion of the model.Through interacting with the environment and maximizing value functions on the premise of obtaining little or no prior information, RL can optimize the pe...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
POSTS&TELECOM PRESS Co., LTD
2022-10-01
|
| Series: | 网络与信息安全学报 |
| Subjects: | |
| Online Access: | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2022064 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841529720250826752 |
|---|---|
| author | Mingying ZHANG Bing HUA Yuguang ZHANG Haidong LI Mohong ZHENG |
| author_facet | Mingying ZHANG Bing HUA Yuguang ZHANG Haidong LI Mohong ZHENG |
| author_sort | Mingying ZHANG |
| collection | DOAJ |
| description | Reinforcement learning(RL) is an artificial intelligence algorithm with the advantages of clear calculation logic and easy expansion of the model.Through interacting with the environment and maximizing value functions on the premise of obtaining little or no prior information, RL can optimize the performance of strategies and effectively reduce the complexity caused by physical models .The RL algorithm based on strategy gradient has been successfully applied in many fields such as intelligent image recognition, robot control and path planning for automatic driving.However, the highly sampling-dependent characteristics of RL determine that the training process needs a large number of samples to converge, and the accuracy of decision making is easily affected by slight interference that does not match with the simulation environment.Especially when RL is applied to the control field, it is difficult to prove the stability of the algorithm because the convergence of the algorithm cannot be guaranteed.Considering that swarm intelligence algorithm can solve complex problems through group cooperation and has the characteristics of self-organization and strong stability, it is an effective way to be used for improving the stability of RL model.The pigeon-inspired optimization algorithm in swarm intelligence was combined to improve RL based on strategy gradient.A RL algorithm based on pigeon-inspired optimization was proposed to solve the strategy gradient in order to maximize long-term future rewards.Adaptive function of pigeon-inspired optimization algorithm and RL were combined to estimate the advantages and disadvantages of strategies, avoid solving into an infinite loop, and improve the stability of the algorithm.A nonlinear two-wheel inverted pendulum robot control system was selected for simulation verification.The simulation results show that the RL algorithm based on pigeon-inspired optimization can improve the robustness of the system, reduce the computational cost, and reduce the algorithm’s dependence on the sample database. |
| format | Article |
| id | doaj-art-90cb9d5afb1d4aa88c11093f9dced888 |
| institution | Kabale University |
| issn | 2096-109X |
| language | English |
| publishDate | 2022-10-01 |
| publisher | POSTS&TELECOM PRESS Co., LTD |
| record_format | Article |
| series | 网络与信息安全学报 |
| spelling | doaj-art-90cb9d5afb1d4aa88c11093f9dced8882025-01-15T03:16:09ZengPOSTS&TELECOM PRESS Co., LTD网络与信息安全学报2096-109X2022-10-018667459575045Robust reinforcement learning algorithm based on pigeon-inspired optimizationMingying ZHANGBing HUAYuguang ZHANGHaidong LIMohong ZHENGReinforcement learning(RL) is an artificial intelligence algorithm with the advantages of clear calculation logic and easy expansion of the model.Through interacting with the environment and maximizing value functions on the premise of obtaining little or no prior information, RL can optimize the performance of strategies and effectively reduce the complexity caused by physical models .The RL algorithm based on strategy gradient has been successfully applied in many fields such as intelligent image recognition, robot control and path planning for automatic driving.However, the highly sampling-dependent characteristics of RL determine that the training process needs a large number of samples to converge, and the accuracy of decision making is easily affected by slight interference that does not match with the simulation environment.Especially when RL is applied to the control field, it is difficult to prove the stability of the algorithm because the convergence of the algorithm cannot be guaranteed.Considering that swarm intelligence algorithm can solve complex problems through group cooperation and has the characteristics of self-organization and strong stability, it is an effective way to be used for improving the stability of RL model.The pigeon-inspired optimization algorithm in swarm intelligence was combined to improve RL based on strategy gradient.A RL algorithm based on pigeon-inspired optimization was proposed to solve the strategy gradient in order to maximize long-term future rewards.Adaptive function of pigeon-inspired optimization algorithm and RL were combined to estimate the advantages and disadvantages of strategies, avoid solving into an infinite loop, and improve the stability of the algorithm.A nonlinear two-wheel inverted pendulum robot control system was selected for simulation verification.The simulation results show that the RL algorithm based on pigeon-inspired optimization can improve the robustness of the system, reduce the computational cost, and reduce the algorithm’s dependence on the sample database.http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2022064pigeon-inspired optimization algorithmstrengthen learningpolicy gradientrobustness |
| spellingShingle | Mingying ZHANG Bing HUA Yuguang ZHANG Haidong LI Mohong ZHENG Robust reinforcement learning algorithm based on pigeon-inspired optimization 网络与信息安全学报 pigeon-inspired optimization algorithm strengthen learning policy gradient robustness |
| title | Robust reinforcement learning algorithm based on pigeon-inspired optimization |
| title_full | Robust reinforcement learning algorithm based on pigeon-inspired optimization |
| title_fullStr | Robust reinforcement learning algorithm based on pigeon-inspired optimization |
| title_full_unstemmed | Robust reinforcement learning algorithm based on pigeon-inspired optimization |
| title_short | Robust reinforcement learning algorithm based on pigeon-inspired optimization |
| title_sort | robust reinforcement learning algorithm based on pigeon inspired optimization |
| topic | pigeon-inspired optimization algorithm strengthen learning policy gradient robustness |
| url | http://www.cjnis.com.cn/thesisDetails#10.11959/j.issn.2096-109x.2022064 |
| work_keys_str_mv | AT mingyingzhang robustreinforcementlearningalgorithmbasedonpigeoninspiredoptimization AT binghua robustreinforcementlearningalgorithmbasedonpigeoninspiredoptimization AT yuguangzhang robustreinforcementlearningalgorithmbasedonpigeoninspiredoptimization AT haidongli robustreinforcementlearningalgorithmbasedonpigeoninspiredoptimization AT mohongzheng robustreinforcementlearningalgorithmbasedonpigeoninspiredoptimization |