Continue, quit, restart probability model and related problems

1 online resource (131 pages) : PDF

2011

University of North Carolina at Charlotte

We discuss a new class of applied probability models. There is a system whose evolution is described by a Markov chain with known transition matrix in a discrete state space. At each moment of a discrete time a decision maker can apply one of three possible actions: to continue, to quit, or to restart Markov chain to the "restarting point", where restarting point is a fixed state of the Markov chain. The decision maker is earning a reward (fee), which is the function of the state and chosen action. The goal for the decision maker is to maximize expected total discounted reward on an infinite time horizon.Such model is a generalization of a model of Katehakis and Veinott (1987), where restart to a unique point is allowed without any fee, and quit action is absent. Both models are related to Gittins index and another index defined in a Whittle family of stopping retirement problems. We propose a transparent recursive finite algorithm to find an optimal strategy in O(n^3) operations.

doctoral dissertations

Operations research
Mathematics

Ph.D.

Elimination Algorithm
Gittins Index
Markov Chain
Optimal Stopping

Applied Mathematics

Sonin, Isaac

Sonin, Stanislav
Molchanov, Joseph
Quinn, Yu

Thesis (Ph.D.)--University of North Carolina at Charlotte, 2011.

This Item is protected by copyright and/or related rights. You are free to use this Item in any way that is permitted by the copyright and related rights legislation that applies to your use. For other uses you need to obtain permission from the rights-holder(s). For additional information, see http://rightsstatements.org/page/InC/1.0/.

Copyright is held by the author unless otherwise indicated.

Steinberg_uncc_0694D_10276

http://hdl.handle.net/20.500.13093/etd:1447