Gradient descent Sarsa(?)algorithm based on the adaptive potential function shaping reward mechanism

In the reinforcement leaning tasks with continuous state spaces,the algorithms are usually facing the problems of ill initial performance and low convergence speed.In order to solve these problems,the potential function shaping reward mechanism was proposed to improve the reinforcement learning algo...

Full description

Saved in:
Bibliographic Details
Main Authors: Fei XIAO, Quan LIU, Qi-ming FU, Hong-kun SUN, Long GAO
Format: Article
Language:zho
Published: Editorial Department of Journal on Communications 2013-01-01
Series:Tongxin xuebao
Subjects:
Online Access:http://www.joconline.com.cn/zh/article/doi/1000-436X(2013)01-0077-12/
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:In the reinforcement leaning tasks with continuous state spaces,the algorithms are usually facing the problems of ill initial performance and low convergence speed.In order to solve these problems,the potential function shaping reward mechanism was proposed to improve the reinforcement learning algorithms.This mechanism propagates model knowledge to the learner adaptively in the form of the additional reward signal,so that the initial performance and convergence speed could be improved effectively.In view of the good performance and existing problems of the radial basis function (RBF) network,the adaptive normalized RBF (ANRBF) network was put forward to use as a potential function to generate the shaping rewards.A gradient descent (GD)algorithm named ANRBF-GD-Sarsa(?) was proposed based on the ANRBF network.The convergence of ANRBF-GD-Sarsa(?) algorithm was analyzed theoretically.Extensive experiments are conducted to show the good initial performance and high convergence speed of the proposed algorithm.
ISSN:1000-436X