Optimized algorithm for value iteration based on topological sequence backups - Wei HUANG; Quan LIU; Hong-kun SUN; Qi-ming FU; HOUXiao-ke Z (Tongxin xuebao)
In order to improve the convergence performance, an optimized value iteration based on topological sequence backups, VI-TS, is proposed. The key idea of VI-TS is to circumvent the problem of unnecessary backups by dividing an MDP into strongly-connected components and solving these components in topological sequences after detecting the structure of MDP. The experiment results show that VI-TS has a better convergence performance and robustness for state space growth when applied to classical planning experiment scenarios.