TY - JOUR
A2 - Ma, Zhiqiang
AU - Liu, Quan
AU - Mu, Xiang
AU - Huang, Wei
AU - Fu, Qiming
AU - Zhang, Yonggang
PY - 2013
DA - 2013/12/05
TI - A Sarsa(*λ*) Algorithm Based on Double-Layer Fuzzy Reasoning
SP - 561026
VL - 2013
AB - Solving reinforcement learning problems in continuous space with function approximation is currently a research hotspot of machine learning. When dealing with the continuous space problems, the classic Q-iteration algorithms based on lookup table or function approximation converge slowly and are difficult to derive a continuous policy. To overcome the above weaknesses, we propose an algorithm named DFR-Sarsa(λ) based on double-layer fuzzy reasoning and prove its convergence. In this algorithm, the first reasoning layer uses fuzzy sets of state to compute continuous actions; the second reasoning layer uses fuzzy sets of action to compute the components of Q-value. Then, these two fuzzy layers are combined to compute the Q-value function of continuous action space. Besides, this algorithm utilizes the membership degrees of activation rules in the two fuzzy reasoning layers to update the eligibility traces. Applying DFR-Sarsa(λ) to the Mountain Car and Cart-pole Balancing problems, experimental results show that the algorithm not only can be used to get a continuous action policy, but also has a better convergence performance.
SN - 1024-123X
UR - https://doi.org/10.1155/2013/561026
DO - 10.1155/2013/561026
JF - Mathematical Problems in Engineering
PB - Hindawi Publishing Corporation
KW -
ER -