Exploiting the structural properties of the underlying markov decision problem in the Q-learning algorithm

Kunnumkal, S and Topaloglu, H (2008) Exploiting the structural properties of the underlying markov decision problem in the Q-learning algorithm. INFORMS Journal on Computing, 20 (2). pp. 288-301. ISSN 1526-5528

Full text not available from this repository. (Request a copy)

Abstract

This paper shows how to exploit the structural properties of the underlying Markov decision problem to improve the convergence behavior of the Q-learning algorithm. In particular, we consider infinite-horizon discounted-cost Markov decision problems where there is a natural ordering between the states of the system and the value function is known to be monotone in the state. We propose a new variant of the Q-learning algorithm that ensures that the value function approximations obtained during the intermediate iterations are also monotone in the state. We establish the convergence of the proposed algorithm and experimentally show that it significantly improves the convergence behavior of the standard version of the Q-learning algorithm

Item Type: Article
Subjects: Business and Management
Date Deposited: 01 Nov 2014 17:12
Last Modified: 11 Jul 2023 17:54
URI: https://eprints.exchange.isb.edu/id/eprint/119

Actions (login required)

View Item
View Item