A general Markov decision method II: Applications

In a preceding paper we have introduced a new approach for solving a wide class of Markov decision problems in which the state-space may be general and the system may be continuously controlled. The criterion is the average cost. This paper discusses two applications of this approach. The first application concerns a house-selling problem in which a constructor builds houses which may be sold at any stage of the construction and potential customers make offers depending on the stage of the construction. The second application considers an M/M/c queuing problem in which the number of operating servers can be controlled by turning servers on or off.