Errata Page

Handbook of Markov Decision Processes

(updated 9 December 2001)

No. Introduction; Title/pages Errata/links Updated
1. Introduction; pages 1--17.  
2. Finite State and Action MDPs; pages 21--88.

Page 73 Reference [66]: the authors are E.B. Dynkin and A.A. Yushkevich

12/09/01
3. Bias Optimality; pages 89--112.  
4. Singular Perturbations of Markov Chains and Decision Processes; pages 113--150.

Example 2.1:  p.132

11/27/01
5. Average Reward Optimization Theory for Denumerable State Spaces; pages 153--172.

Page 159 formula 5.17--omit the "*"

12/04/01
6. Total Reward Criteria; pages 173--208.  
7. Mixed Criteria; pages 209--230.  
8. Blackwell Optimality; pages 231--268.  
9. The Poisson Equation for Countable Markov Chains: Probabilistic Methods and Interpretations; pages 269--304.  
10. Stability, Performance Evaluation, and Optimization; pages 305--346.  
11. Convex Analytic Methods in Markov Decision Processes;  pages 347--376.  
12. The Linear Programming Approach; pages 377--408.  
13. Invariant Gambling Problems and Markov Decision Processes; pages 409--430.  
14. Neuro-Dynamic Programming: Overview and Recent Trends; pages 431--460.  
15. Markov Decision Processes in Finance and Dynamic Options; pages 461--488.

Eq.(15.8): p. 464, some typos

1/17/02
16. Applications of Markov Decision Processes in Communication Networks; pages 489--536.  
17. Water Reservoir Applications of Markov Decision Processes; pages 537--558.