Kluwer HomepageCustomer ServiceWebmasterOrdering InformationCatalogue HomepageSearch






Special Offer 

Write to Editors

Errata Page
Comments on Open Problems



Handbook of Markov Decision Processes

Methods and Applications




edited by
Eugene A. Feinberg
SUNY at Stony Brook, USA
Adam Shwartz
Technion Israel Institute of Technology, Haifa, Israel

Kluwer 2002 (565 pages)

Contents and Contributors 
(links to introduction of each chapter)

1. Introduction; E.A. Feinberg, A. Shwartz.

Part I: Finite State and Action Models.

2. Finite State and Action MDPs; L. Kallenberg.
3. Bias Optimality; M.E. Lewis, M.L. Puterman.
4. Singular Perturbations of Markov Chains and Decision Processes
    K.E. Avrachenkov, J. Filar, M. Haviv

Part II: Infinite State Models.

5. Average Reward Optimization Theory for Denumerable State Spaces; L.I. Sennott.
6. Total Reward Criteria; E.A. Feinberg.
7. Mixed Criteria; E.A. Feinberg, A. Shwartz.
8. Blackwell Optimality; A. Hordijk, A.A. Yushkevich.
9. The Poisson Equation for Countable Markov Chains: Probabilistic Methods and Interpretations;
    A.M. Makowski, A. Shwartz.
10. Stability, Performance Evaluation, and Optimization; S.P. Meyn.
11. Convex Analytic Methods in Markov Decision ProcessesV.S. Borkar.
12. The Linear Programming Approach; O. Hernández-Lerma,  J.B. Lasserre.
13. Invariant Gambling Problems and Markov Decision Processes
      L.E. Dubins, A.P. Maitra, W.D. Sudderth

Part III: Applications.

14. Neuro-Dynamic Programming: Overview and Recent TrendsB. Van Roy.
15. Markov Decision Processes in Finance and Dynamic OptionsM. Schäl.
16. Applications of Markov Decision Processes in Communication Networks; E. Altman.
17. Water Reservoir Applications of Markov Decision Processes
      B.F. Lamond, A. Boukhtouta.