Abstract

This paper establishes the existence of a solution to the optimality equations in undiscounted semi-Markov decision models with countable state space, under conditions generalizing the hitherto obtained results. In particular, we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy, instead of the traditional and restrictive assumption that ever stationary policy has a single irreducible set of states. A replacement model and an inventory model illustrate why this extension is essential. Our approach differs fundamentally from classical approaches; we convert the optimality equations into a form suitable for the application of a fixed point theorem.

Authors
Awi Federgruen, Paul Schweitzer, and H. C. Tijms
Format
Journal Article
Publication Date
Journal
Mathematics of Operations Research

Full Citation

Federgruen, Awi, Paul Schweitzer, and H. C. Tijms
. “Denumerable undiscounted semi-Markov decision processes with unbounded rewards.”
Mathematics of Operations Research
vol.
8
, (May 01, 1983):
298
-
313
.