| Scott Moura | 
  |
Projects
Identifiability and Adaptive Control of Markov ChainsScott Moura and Ben Pence
December 12, 2009
Stochastic Control (EECS 558)
Professor Demosthenis Teneketzis
University of Michigan
This project reviews three papers from the identification and adaptive control literature for Markov chain models. The reviewed papers are by Mandl and Borkar & Varaiya. Each paper addresses the problem of controlling a Markov chain in which the transition probability matrices depend on an unknown parameter vector. A key issue in adaptive control, as discussed in the reviewed papers, is the "dual control" problem - that is, when to apply controls to improve performance and when to apply controls to improve parameter estimation. To motivate why this tradeoff is important, this reports focuses on the concept of "identifiability".
REPORT | SLIDES