Abstract

This paper considers an undiscounted semi-Markov decision problem with denumerable state space and compact metric action spaces. Recurrence conditions on the transition probability matrices associated with the stationary policies are considered and relations between these conditions are established. Also it is shown that under each of these conditions the optimality equation for the average costs has a bounded solution.

Authors
Awi Federgruen, A. Hordijk, and H. C. Tijms
Format
Chapter
Publication Date
Book
Dynamic Programming and Its Applications

Full Citation

Federgruen, Awi, A. Hordijk, and H. C. Tijms
. “Recurrence Conditions in Denumerable State Markov Decision Processes.” In
Dynamic Programming and Its Applications
, edited by
Martin L. Puterman
,
3
-
22
.
New York
:
Academic Press
, 1978.