Files
Abstract
We discuss a new class of applied probability models. There is a system whose evolution is described by a Markov chain with known transition matrix in a discrete state space. At each moment of a discrete time a decision maker can apply one of three possible actions: to continue, to quit, or to restart Markov chain to the "restarting point", where restarting point is a fixed state of the Markov chain. The decision maker is earning a reward (fee), which is the function of the state and chosen action. The goal for the decision maker is to maximize expected total discounted reward on an infinite time horizon.Such model is a generalization of a model of Katehakis and Veinott (1987), where restart to a unique point is allowed without any fee, and quit action is absent. Both models are related to Gittins index and another index defined in a Whittle family of stopping retirement problems. We propose a transparent recursive finite algorithm to find an optimal strategy in O(n^3) operations.