Abstract

This paper deals with Markov decision processes with a countable state space. We demonstrate that a single, relatively simple condition suffices to guarantee that the value-iteration method converges and that an optimal policy can be computed via this method, once the existence of a solution to the average cost optimality equation has been established via any of the many available sets of existence conditions.

Authors
Yossi Aviv and Awi Federgruen
Format
Journal Article
Publication Date
Journal
Operations Research Letters

Full Citation

Aviv, Yossi and Awi Federgruen
. “The value iteration method for countable state Markov decision processes.”
Operations Research Letters
vol.
24
, (June 01, 1999):
223
-
234
.