Eugene A

Eugene A. Feinberg

Department of Applied Mathematics and Statistics
State University of New York at Stony Brook
Stony Brook, NY, 11794-3600
tel: (631) 632-7189
fax: (631) 632-8490
email: Eugene.Feinberg@sunysb.edu

Books

Handbook of Markov Decision Processes: Methods and Algorithms (with A. Shwartz, editors), Kluwer, Boston, 2002

Papers in Refereed Journals and Books:

“On Controlled Finite State Markov Processes with Compact Control Sets,” SIAM Theory Probability Appl., 20, pp. 856-862, 1975 PDF

“Control of Queueing Systems” (with M. A. Feinberg), Radioelectronics, no. 3, pp. 3-34, 1975 (in Russian)

“On e-optimal Control of Discrete Time Markov Chains,” Transactions of Moscow Institute of Transport Engineering, 510, pp. 33-36, 1976 (in Russian)

“Railway Transport Radio Communication Models” (with L. M. Zuravliova), Railway Transport, no. 6, pp. 81-83, 1976 (in Russian)

“On Finite Controlled Markov Chains,” Uspehi Math. Nauk, 32, no 3, pp. 181-182, 1977 (in Russian)

“The Existence of a Stationary e-optimal Policy for a Finite Markov Chain,” SIAM Theory Probability Appl., 23, pp. 297-313, 1978 PDF

“On Homogeneous Markov Decision Models with Continuous Time and Finite or Countable State Spaces” (with A. A. Yushkevich), SIAM Theory Probability Appl., 24, pp. 156-161, 1979 PDF PDF

“On Ergodic Conditions for Finite Controlled Markov Chains with Arbitrary Decision Sets,” Transactions of Moscow Institute of Transport Engineering, 640, pp. 51-59, 1979 (in Russian)

“An e-optimal Control of a Finite Markov Chain,” SIAM Theory Probability Appl., 25, pp. 70-81, 1980 PDF

“Optimization Models for Railway Radio Communication Procedures” (with L. M. Zuravliova), Railway Transport, no. 4, pp. 65-67, 1980 (in Russian)

“An Optimal Strategy for Activation of a Service Device in One Class of Systems with Group Service” (with L. B. Boguslavsky), Problems of Information Transmission, no. 3,pp. 243-253, 1980.

“Optimization of Number of Vans in a Group in Railway Operations” (with S. M. Rezer), Transactions of Moscow Institute of Transport Engineering, 653, pp. 55-63, 1981 (in Russian)

“Nonrandomized Markov and Semi-Markov Strategies in Dynamic Programming,” SIAM Theory Probability Appl., 27, pp. 116-126, 1982 PDF

“Controlled Markov Processes with Arbitrary Numerical Criteria,” SIAM Theory Probability Appl., 27, pp. 486-503, 1982. PDF

“Stationary Strategies in Dynamic Programming with a Countable State Space” (with I. M. Sonin), Studies on Mathemat. Econom. and Control Theory, Central Econom.-Math. Institute Press, Moscow, pp. 137-173, 1983 (in Russian)

“Stationary and Markov Policies in Countable State Dynamic Programming” (with I. M. Sonin), Lecture Notes in Math., 1021, pp. 111-129, 1982 PDF

“Sufficient Classes of Strategies in Controllable Markov Chains with Total Criterion” (with I. M. Sonin), Soviet Math. Dokl., 29, no. 2, pp. 308-311, 1984

“Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming” (with I. M. Sonin), Statistics and Control of Stochastic Processes, Steklov Seminar, Optimization Software, NY, pp. 69-101, 1985

“Optimal Specification of Railway Platform Sections,” Transactions of Moscow Institute of Transport Engineering, 758, pp. 58-62, 1984 (in Russian)

“Optimal Control of Parameters of Multiple Access Protocols” (with Ya. A. Kogan and A. N. Smirnov), Automatic Control and Computer Science, no. 6, pp. 48-54, 1984.

“Optimal Control by the Retransmission Probability in Slotted ALOHA Systems” (with Ya. A. Kogan and A. N. Smirnov), Performance Evaluation, 5, pp. 85-96, 1985

“Sufficient Classes of Strategies in Discrete Dynamic Programming. I: Decomposition of Randomized Strategies and Embedded Models,” SIAM Theory Probability Appl., 31, pp. 658-668, 1986 PDF

“The Structure of Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming,” Lecture Notes in Control and Inform. Sci., 81, pp. 22-31, 1986

“Sufficient Classes of Strategies in Discrete Dynamic Programming. II: Locally Stationary Strategies,” SIAM Theory Probability Appl., 32, No. 3, pp. 478-493, 1987 PDF

“Parametric Stochastic Dynamic Programming,” Statistics and Control of Stochastic Processes, Steklov Seminar, 2, Optimization Software, NY, pp. 103-120, 1989

“Nonrandomized Strategies in Stochastic Decision Processes,” Annals of Operations Research, 29, pp. 315-332, 1991

“A Markov Decision Model of a Search Process,” Contemporary Mathematics, 125, pp. 87-96, 1992

“On Stationary Strategies in Borel Dynamic Programming,” Mathematics of Operations Research, 17, pp. 393-397, 1992

“Finite State Markov Decision Models with Average Reward Criteria” (with H. Park), Stochastic Processes Appl., 49, pp. 159-177, 1994

“Markov Decision Models with Weighted Discounted Criteria” (with A. Shwartz), Mathematics of Operations Research, 19, pp. 152-168, 1994

“A Generalization of “Expectation Equals Reciprocal of Intensity” to Nonstationary Distributions,” J. Appl. Probability, 31, pp. 262-267, 1994. . Download PS or PDF

“Constrained Semi-Markov Decision Processes with Average Rewards,” ZOR - Mathematical Methods of Operations Research, 39, pp. 257-288, 1994

“Optimality of Randomized Trunk Reservation” (with M. Reiman), Probability in Engineering and Informational Sciences, 8, pp. 483-489, 1994

“Constrained Markov Decision Models with Weighted Discounting” (with A. Shwartz), Mathematics of Operations Research, 20, pp. 302-320, 1995

“Bicriterion Optimization of an M/G/1 Queue with a Removable Server” (with D.J. Kim), Probability in Engineering and Informational Sciences, 10, pp. 57-73, 1996. Download PDF

“Letter to the Editor,” Operations Research, 44, p. 526, 1996

“Constrained Discounted Dynamic Programming” (with A. Shwartz), Mathematics of Operations Research, 21, pp. 922-945, 1996. . Download PS or DVI

“Notes on Equivalent Stationary Policies in Markov Decision Processes with Total Rewards” (with I.M. Sonin), ZOR - Mathematical Methods of Operations Research, 44, pp. 205-221, 1996.

“On Measurability and Representation of Strategic Measures in Markov Decision Processes”, in Statistics, Probability and Game Theory Papers in Honor of David Blackwell (eds. T.S. Ferguson et al.), IMS Lecture Notes - Monograph Series, 30, pp. 29-43, 1996 PDF

“Stochastic Monotonicity for Stationary Recurrence Times of First Passage Heights” (with E.V. Denardo and O. Kella), Annals of Applied Probability, 7, pp. 326-339, 1997. Download PDF

“On Future Organization of Hybrid Chip Manufacturing” (with S. Luryi), in Future Trends in Microelectronics , J. Wiley & Sons (eds. S. Luryi and A. Zaslavsky), pp. 47-54, 1999.

“Constrained Dynamic Programming with Two Discount Factors: Applications and an Algorithm”; (with A. Shwartz), IEEE Transactions on Automatic Control, 44, pp. 628-631, 1999. Download PS or DVI

“ Weighted Discounted Stochastic Games with Perfect Information” (with E. Altman and A. Shwartz), Annals of the International Society of Dynamic Games, 5, pp. 303-324, 2000. Download PDF

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles” Mathematics of Operations Research, 25, pp. 130-140, 2000. Download PDF

“A Note on the Existence of Optimal Policies in Continuous Dynamic Programs with Compact Action Sets” (with R. Cavazos-Cadena and R. Montes-de-Oca), Mathematics of Operations Research, 25, pp. 657-666, 2000. Download PS or DVI

“Multiple Objective Nonatomic Markov Decision Processes with Total Reward Criteria” (with A.B. Piunovskiy) ), Journal of Mathematical Analysis and Applications, 247, pp. 45-66, 2000. Download PS or DVI

“Perturbed Zero-Sum Games with Applications to Stochastic and Repeated Games” (with E. Altman, J. Filar, and V. Gaitsgory), Annals of the International Society of Dynamic Games, 6, pp. 165-181, 2001. Download PS or DVI

“Introduction” (with A. Shwartz), Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp. 1-17, 2002.

“Mixed Criteria” (with A. Shwartz) , Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 209-230, 2002. Download PS or DVI

“Total Reward Criteria,” Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 173-207, 2002.

“Constrained Discounted Semi-Markov Decision Processes,” Markov Processes and Controlled Markov Chains (Z. How, J.A. Filar, A. Chen, eds), Kluwer, Norwell, MA, pp. 231-242, 2002. Download PS

“Nonatomic Total Reward Markov Decision Processes with Multiple Criteria” (with A.B. Piunovskiy), Journal of Mathematical Analysis and Applications, 273, 93-111, 2002. Download PDF

“Optimality of D-policies for an M/G/1 Queue with a Removable Server” (with O. Kella), Queueing Systems – Theory and Applications (QUESTA) 42, pp. 355-376, 2002. Download PDF

“Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach,” Mathematics of Operations Research, 29, pp. 492-524, 2004. Download PDF

“Optimality of Four-Threshold Policies in Inventory Systems with Customer Returns and Borrowing/Storage Options,” (with M.E. Lewis), Probability in Engineering and Informational Sciences, 19, 45-71, 2005. Download PDF

“Generalized Pinwheel Problem,” (with M.T. Curry), Mathematical Methods of Operations Research, 62, 99-122, 2005. Download PDF

“Load Forecasting,” (with D. Genethliou), Applied Mathematics for Restructured Electric Power Systems: Optimization, Control, and Computational Intelligence (J. H. Chow, F.F. Wu, and J.J. Momoh, eds.), Spinger, pp. 269-285, 2005. Download PDF

“On Essential Information in Stochastic Decision Processes,” Mathematical Methods of Operations Research, 62, 399-410, 2005. Download PDF

“On Dvoretzky-Wald-Wolfowitz Theorem on Nonrandomized Statistical Decisions,” (with A.B. Piunovskiy), SIAM Theory Probability Appl., 50, 463-466, 2006. Download PDF

“Optimality of Randomized Trunk Reservation for a Problem with a Single Constraint,” (with X. Fan-Orzechowski), Advances in Applied Probability, 38, 199-220, 2006. Download PDF

“Quickest Detection of Drift Change for Brownian motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Statistics & Decisions, 24, 445-470, 2006. Download PDF

“Optimality of Randomized Trunk Reservation for a Problem with Multiple Constraints,” (with X. Fan-Orzechowski), Probability in Engineering and Informational Sciences, 21, 189-200, 2007. Download PDF

“Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem,” (with M.E. Lewis), Mathematics of Operations Research, 32, 769-783, 2007. Download PDF

“Non-Randomized Policies for Constrained Markov Decision Processes,” (with R.C. Chen), Mathematical Methods of Operations Research, 66, 165-179, 2007. Download PDF

“Optimality of Deterministic Policies for Certain Stochastic Control Problems with Multiple Criteria and Constraints,” 2007. Download PDF

“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang) 2008. Download PDF

Papers in Conference Proceedings:

“Quality Evaluation and Optimization of Data Transmission Procedures in Computer Networks” (with L. B. Boguslavsky, E. Gelembe, Ya. A. Kogan), Proceedings of 7-th National Conference on Coding Theory and Data Transmission, Vilnius, 3, pp. 25-30, 1978 (in Russian)

“Markov and Semi-Markov Strategies in Dynamic Programming,” Proceedings of 14-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp. 30-31, 1980 (in Russian)

“Nonrandomized Strategies in Markov Decision Processes,” Proceedings of the Conference on Mathematical Methods of Optimization in Large Economic and Technical Systems,” Baku, pp. 188-190, 1980 (in Russian)

“Discrete Time Stochastic Decision Processes with Arbitrary Real-Valued Criteria,” Abstracts of 3-rd Vilnius Conference on Probability Theory and Mathematical Statistics, 3, pp. 88-89, 1981

“On Markov Decision Models with Measurable Value Functions,” Proceedings of 15-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp.12--13, 1981 (in Russian)

“Markov Policies in Infinite Horizon Dynamic Programming Problems with Bounded Value Functions” (with I. M. Sonin), Abstracts of 4-th USSR - Japan Symposium on Probability Theory and Mathematical Statistics, Tbilisi, 1, pp. 209-210, 1982

“Persistently Nearly Optimal Strategies in Countable State Markov Decision Chains with the Total Reward Criterion” (with I. M. Sonin), Abstracts of International Conference on Stochastic Optimization, Kiev, pp. 74-76, 1984

“Uniformly Optimal Strategies in Controlled Random Sequences,” Proceedings of 18-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 44, 1984 (in Russian)

“Optimal Control of Parameters in Random Multivariate Access” (with Ya. A. Kogan and A. N. Smirnov), Proceedings of 9-th Russian Conference on Computer Networks, Puschino, 2.1, pp. 41-44, 1984 (in Russian)

“On the Existence of Good Strategies in Stochastic Dynamic Programming,” Abstracts of 4-th Vilnius Conference on Probability Theory and Mathematical Statistics, 4, pp. 70-72, 1985

“Controlled Markov Models with Decision Set Selection,” Proceedings of 19-th Conference on Probability Theor and Mathematical Statistics, Bakuriani, p. 58, 1985 (in Russian)

“Optimization of a railway traffic” (with S. M. Rezer and L. N. Schennikov), Proceedings of the Conference “Optimization and Control of Transportation Systems”, Gomel, pp. 169-171, 1985 (in Russian).

“Additive Functionals of Jump Processes: a Discrete Time Representation,” Abstracts of I-st World Congress of Bernoulli Society, Tashkent, 1, p. 289, 1986

“Parametric Dynamic Programming,” Proceedings of 20-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 56, 1986 (in Russian)

“Optimality of Pure Strategies in Stochastic Decision Processes,” Proceedings of the 29-th IEEE Conference on Decision and Control, 4, pp. 2149-2154, 1990

“Weighted Discounted Dynamic Programming” (with A. Shwartz), Proceedings of the 30-th IEEE Conference on Decision and Control, 2, pp. 485-486, 1991.

“Optimal Switching Policies for M/G/1 Queues with Two Performance Criteria” (with D.J. Kim), Proceedings of the International Conference on Operations Research, pp. 227-232, 1994, Berlin, Springer-Verlag.

“Multiple Objective Optimal Control of Production Systems with Stochastic Demand” (with D.J. Kim), Proceedings of the First Regional Symposium on Manufacturing Science and Technology, Stony Brook, NY, pp. 37-43, 1995.

“Optimization of Jump Stochastic Systems,” Proceedings of the 1996 NSF Design and Manufacturing Grantees Conference, Albuquerque, NM, pp.497-498, 1996.

“Weighted Discounted Stochastic Games with Perfect Information,” (with E. Altman and A. Shwartz), Proceedings of the Seventh International Symposium on Dynamic Games and Applications, December 16-18, Kanagawa, Japan, 1, 18 - 31, 1996.

“Optimization of Discounted Constrained Jump Stochastic Systems,” Proceedings of the 1997 NSF Design and Manufacturing Grantees Conference, Seattle, WA, pp. 457-458, 1997.

“Geographic Scheduling System for Customer Service,” (with A. Zhukovsky), Proceedings of the 18^th National Conference of the American Society of Engineering Management, Virginia Beach, pp. 561 - 563, 1997.

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles,” Proceedings of the 36-th IEEE Conference on Decision and Control, 3, pp. 2821 - 2826, 1997.

“Optimization of Jump Stochastic Systems: Constrained Dynamic Programming and Hamiltonian Cycles,” Proceedings of the 1997 NSF Design and Manufacturing Grantees Conference, NSF, Arlington, VA, pp. 217-218, 1998.

“Perturbed Zero-Sum Games with Applications to Dynamic Games” (with E. Altman, J. Filar, and V. Gaitsgory), Proceedings of the Seventh International Symposium on Dynamic Games and Applications, July 5-8, Maastricht, The Netherlands, pp. 45 - 51, 1998.

“Optimization of Jump Stochastic Systems,” Proceedings of the 1999 NSF Design and Manufacturing Grantees Conference, Long Beach, CA, January 5-8, 1999 (CD).

“New Approach to Optimization of Discounted Stochastic Continuous-Time Discrete-Event Systems,” Proceedings of the 38‑th IEEE Conference on Decision and Control 1, pp. 937 – 941, 1999.

“Optimality of Pure Strategies in Continuous Dynamic Programming Problems with Multiple Criteria, “ (with A.B. Piunovskiy) Proceedings of the 2000 NSF Design and Manufacturing Grantees Conference, Vancouver, Canada, January 2000 (CD).

“Nonatomic Total Reward Markov Decision Processes with Multiple Criteria,” (with A.B. Piunovskiy), Proceedings of the 39‑th IEEE Conference on Decision and Control, 1, pp. 723 – 728, 2000.

“Nonatomic Total Reward Markov Decision Processes,” (with A. B. Piunovskiy), Proceedings of the 2001 NSF Design and Manufacturing Research Conference, Tampa, Florida, January 6-9, 2001 (CD).

“Optimization of Jump Stochastic Systems,” Proceedings of the 2002 NSF Design and Manufacturing Research Conference, San Juan, Puerto Rico, January 7-10, 2002 (CD).

“Sensor Resource Management for an Airborne Early Warning Radar,” (with M. Bender, M. Curry, D. Huang, T. Koutsoudis, J. Bernstein), Signal and Data Processing of Small Targets, Proceedings of SPIE – The International Society of Optical Engineering, 2-4 April 2002, Orlando, 4728, pp. 145-156, 2002.

“Online Scheduling with Hard Constraints: a Semi-Markov Decision Process Approach,” (with M. Curry), Control and Applications, Proceedings of the IASTED International Conference, Cancun, MX, May 20-22, pp. 104-109, 2002.

“Load Pocket Forecasting,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Second IASTED International Conference, Crete, Greece, June 25-28, pp. 50-54, 2002. Download PDF

“Constrained Finite Continuous-Time Markov Decision Processes with Average Rewards,” Proceedings of IEEE 2002 Conference on Decisions and Control, Las Vegas, December 10-13, pp. 3805-3810, 2002.

“Average reward constrained continuous-time finite Markov decision processes,” Proceedings of the 2002 NSF Design and Manufacturing Research Conference, Birminham, AL, January 6-9, 2003 (CD).

“Statistical Load Modeling,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Seventh IASTED International Multi-Conference, Palm Springs, CA, February 24-26, pp. 88-91, 2003. Download PDF

“Control of Finite Continuous-Time Markov Chains with Average Rewards and Constraints,” Kolmogorov and Contemporary Mathematics, Abstracts, Moscow State University, June 16-21, 2003, pp. 427-428.

“Online Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of the 42^nd IEEE Conference on Decision and Control, Maui, December 9-12, 2003, pp. 4333-4338.

“Real Time Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of the 2004 NSF Design and Manufacturing Research Conference, Dallas, TX, January 4-7, 2004 (CD).

“Load Pocket Forecasting Software,” (with D. Genethliou, J.T. Hajagos, B.G. Irrgang, and R.R. Rossin),

Proceedings of 2004 IEEE Power Systems Conference & Exposition, New York, October 10-13, 2004, Download PDF

“Optimality Inequalities for Average Cost MDPs and their Inventory Control Applications,” (with M.E. Lewis), Proceedings of the 43^nd IEEE Conference on Decision and Control, Bahamas, December 14-17, 2004, pp. 4411-4416, Download PDF

“Optimality of Randomized Trunk Reservation for a Problem with a Single Constraint,” (with X. Fan-Orzechowski), Proceedings of the 2004 NSF Design and Manufacturing Research Conference, Scottsdale, AZ, January 3-6, 2005 (CD).

“Buffer Insertion for Bridges and Optimal Buffer Sizing for Communication Sub-System of Systems-on-Chip,” (with S.

Kallakuri, A. Doboli), Proceedings of Design Automation and Test in Europe, Munich, Germany, March 6-10, 2005, pp 826-

827.

“A Continuous Time Markov Decision Process Based On-Chip Buffer Allocation Methodology,” (with S. Kallakuri, N. Thepayasuwan, and A. Doboli), Proceedings of GLSVLSI (Great Lakes Symposium on VLSI), Chicago, IL, April 17-19, 2005, pp. 345-348.

“Optimal Admission Control for a Markovian Queue under the Quality of Service Constraint,” (with Xiaofei Fan-Orzechowski), Proceedings of the 44^nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, December 12-15, 2005, pp. 1729-1734.

“On State Space Reduction in Sequential Decision Processes,” Proceedings of the 44^nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, December 12-15, 2005, pp. 7314-7319.

“Non-Randomized Control of Constrained Markov Decision Processes,” (with R.C. Chen ), Proceedings of the 2006 American Control Conference, Minneapolis, MN, June 14-16, 2006.

“Optimality of Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), Proceedings of the 2006 NSF Design, Service, and Manufacturing Grantees and Research Conference, St. Louis, MO, July 24-27, 2006 (CD).

“Quickest Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, TN, January 7-10, 2008 (CD).

“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, TN, January 7-10, 2008 (CD).