Eugene A. Feinberg
Department of Applied Mathematics and
Statistics
State University of New York at Stony Brook
Stony Brook, NY, 11794-3600
tel: (631) 632-7189
fax: (631) 632-8490
email: Eugene.Feinberg@sunysb.edu
Handbook of Markov Decision Processes: Methods and
Algorithms (with A. Shwartz, editors), Kluwer,
Papers in Refereed Journals and Books:
“On Controlled Finite State Markov Processes with Compact Control Sets,”
“Control of Queueing Systems” (with M. A. Feinberg), Radioelectronics, no. 3, pp. 3-34, 1975 (in Russian)
“On e-optimal Control of Discrete Time Markov Chains,” Transactions of Moscow Institute of Transport Engineering, 510, pp. 33-36, 1976 (in Russian)
“Railway Transport Radio Communication Models” (with L. M. Zuravliova), Railway Transport, no. 6, pp. 81-83, 1976 (in Russian)
“On Finite Controlled Markov Chains,” Uspehi Math. Nauk, 32, no 3, pp. 181-182, 1977 (in Russian)
“The Existence of a Stationary e-optimal Policy for a Finite Markov Chain,” SIAM Theory Probability Appl., 23, pp. 297-313, 1978 PDF
“On Homogeneous Markov Decision Models with Continuous Time and Finite or
Countable State Spaces” (with A. A. Yushkevich),
“On Ergodic Conditions for Finite Controlled
Markov Chains with Arbitrary Decision Sets,” Transactions of
“An e-optimal Control of a Finite Markov Chain,” SIAM Theory Probability Appl., 25, pp. 70-81, 1980 PDF
“Optimization Models for Railway Radio Communication Procedures” (with L. M. Zuravliova), Railway Transport, no. 4, pp. 65-67, 1980 (in Russian)
“An Optimal Strategy for Activation of a Service Device in One Class of Systems with Group Service” (with L. B. Boguslavsky), Problems of Information Transmission, no. 3,pp. 243-253, 1980.
“Optimization of Number of Vans in a Group in Railway Operations” (with S. M. Rezer), Transactions of Moscow Institute of Transport Engineering, 653, pp. 55-63, 1981 (in Russian)
“Nonrandomized Markov and Semi-Markov Strategies in Dynamic Programming,”
“Controlled Markov Processes with Arbitrary Numerical Criteria,” SIAM Theory Probability Appl., 27, pp. 486-503, 1982. PDF
“Stationary Strategies in Dynamic Programming with a Countable State Space”
(with
“Stationary and Markov Policies in Countable State Dynamic Programming”
(with
“Sufficient Classes of Strategies in Controllable Markov Chains with Total
Criterion” (with
“Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming” (with I. M. Sonin), Statistics and Control of Stochastic Processes, Steklov Seminar, Optimization Software, NY, pp. 69-101, 1985
“Optimal Specification of Railway Platform Sections,” Transactions of Moscow Institute of Transport Engineering, 758, pp. 58-62, 1984 (in Russian)
“Optimal Control of Parameters of Multiple Access Protocols” (with Ya. A. Kogan and A. N. Smirnov), Automatic Control and Computer Science, no. 6, pp. 48-54, 1984.
“Optimal Control by the Retransmission Probability in Slotted ALOHA Systems” (with Ya. A. Kogan and A. N. Smirnov), Performance Evaluation, 5, pp. 85-96, 1985
“Sufficient Classes of Strategies in Discrete Dynamic
Programming. I: Decomposition of Randomized Strategies and Embedded
Models,”
“The Structure of Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming,” Lecture Notes in Control and Inform. Sci., 81, pp. 22-31, 1986
“Sufficient Classes of Strategies in Discrete Dynamic
Programming. II: Locally Stationary Strategies,”
“Parametric Stochastic Dynamic Programming,” Statistics and Control of Stochastic Processes, Steklov Seminar, 2, Optimization Software, NY, pp. 103-120, 1989
“Nonrandomized Strategies in Stochastic Decision Processes,” Annals of Operations Research, 29, pp. 315-332, 1991
“A Markov Decision Model of a Search Process,” Contemporary Mathematics, 125, pp. 87-96, 1992
“On Stationary Strategies in Borel Dynamic Programming,” Mathematics of Operations Research, 17, pp. 393-397, 1992
“
“Markov Decision Models with Weighted Discounted Criteria” (with A. Shwartz), Mathematics of Operations Research, 19, pp. 152-168, 1994
“A Generalization of “Expectation Equals Reciprocal of Intensity” to Nonstationary Distributions,” J. Appl. Probability, 31, pp. 262-267, 1994. . Download PS or PDF
“Constrained Semi-Markov Decision Processes with Average Rewards,” ZOR - Mathematical Methods of Operations Research, 39, pp. 257-288, 1994
“Optimality of Randomized Trunk Reservation” (with M. Reiman), Probability in Engineering and Informational Sciences, 8, pp. 483-489, 1994
“Constrained Markov Decision Models with Weighted Discounting” (with A. Shwartz), Mathematics of Operations Research, 20, pp. 302-320, 1995
“Bicriterion Optimization of an M/G/1 Queue with a Removable Server” (with D.J. Kim), Probability in Engineering and Informational Sciences, 10, pp. 57-73, 1996. Download PDF
“Letter to the Editor,” Operations Research, 44, p. 526, 1996
“Constrained Discounted Dynamic Programming” (with A. Shwartz), Mathematics of Operations Research, 21, pp. 922-945, 1996. . Download PS or DVI
“Notes on Equivalent Stationary Policies in Markov Decision Processes with Total Rewards” (with I.M. Sonin), ZOR - Mathematical Methods of Operations Research, 44, pp. 205-221, 1996.
“On Measurability and Representation of Strategic Measures in Markov
Decision Processes”, in Statistics, Probability and Game Theory Papers in
Honor of David Blackwell (eds. T.S. Ferguson et al.), IMS Lecture Notes - Monograph
Series, 30, pp. 29-43, 1996 PDF
“Stochastic Monotonicity for Stationary Recurrence Times of
“On Future Organization of Hybrid Chip Manufacturing”
(with S. Luryi), in Future Trends in Microelectronics , J. Wiley & Sons (eds.
“Constrained Dynamic Programming with Two Discount Factors: Applications and an Algorithm”; (with A. Shwartz), IEEE Transactions on Automatic Control, 44, pp. 628-631, 1999. Download PS or DVI
“ Weighted Discounted Stochastic Games with Perfect
Information” (with
“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles” Mathematics of Operations Research, 25, pp. 130-140, 2000. Download PDF
“A Note on the Existence of Optimal Policies in Continuous Dynamic Programs with Compact Action Sets” (with R. Cavazos-Cadena and R. Montes-de-Oca), Mathematics of Operations Research, 25, pp. 657-666, 2000. Download PS or DVI
“Multiple Objective Nonatomic Markov Decision Processes with Total Reward Criteria” (with A.B. Piunovskiy) ), Journal of Mathematical Analysis and Applications, 247, pp. 45-66, 2000. Download PS or DVI
“Perturbed Zero-Sum Games with Applications to Stochastic
and Repeated Games” (with
“Introduction” (with A. Shwartz), Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp. 1-17, 2002.
“Mixed Criteria” (with A. Shwartz) , Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 209-230, 2002. Download PS or DVI
“Total Reward Criteria,” Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 173-207, 2002.
“Constrained Discounted Semi-Markov
Decision Processes,” Markov Processes and Controlled Markov Chains (Z.
How, J.A. Filar, A. Chen, eds), Kluwer,
“Nonatomic Total Reward Markov Decision Processes with Multiple Criteria” (with A.B. Piunovskiy), Journal of Mathematical Analysis and Applications, 273, 93-111, 2002. Download PDF
“Optimality of D-policies for an M/G/1 Queue with a Removable Server” (with O. Kella), Queueing Systems – Theory and Applications (QUESTA) 42, pp. 355-376, 2002. Download PDF
“Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach,” Mathematics of Operations Research, 29, pp. 492-524, 2004. Download PDF
“Optimality of Four-Threshold Policies in Inventory Systems with Customer Returns and Borrowing/Storage Options,” (with M.E. Lewis), Probability in Engineering and Informational Sciences, 19, 45-71, 2005. Download PDF
“Generalized Pinwheel Problem,” (with M.T. Curry), Mathematical Methods of Operations Research, 62, 99-122, 2005. Download PDF
“Load Forecasting,” (with D. Genethliou), Applied Mathematics for Restructured Electric Power Systems: Optimization, Control, and Computational Intelligence (J. H. Chow, F.F. Wu, and J.J. Momoh, eds.), Spinger, pp. 269-285, 2005. Download PDF
“On Essential Information in Stochastic Decision Processes,” Mathematical Methods of Operations Research, 62, 399-410, 2005. Download PDF
“On Dvoretzky-Wald-Wolfowitz Theorem on Nonrandomized Statistical Decisions,” (with A.B. Piunovskiy), SIAM Theory Probability Appl., 50, 463-466, 2006. Download PDF
“Optimality of Randomized Trunk Reservation for a Problem with a Single Constraint,” (with X. Fan-Orzechowski), Advances in Applied Probability, 38, 199-220, 2006. Download PDF
“Quickest Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Statistics & Decisions, 24, 445-470, 2006. Download PDF
“Optimality of Randomized Trunk Reservation for a Problem with Multiple Constraints,” (with X. Fan-Orzechowski), Probability in Engineering and Informational Sciences, 21, 189-200, 2007. Download PDF
“Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem,” (with M.E. Lewis), Mathematics of Operations Research, 32, 769-783, 2007. Download PDF
“Non-Randomized Policies for Constrained Markov Decision Processes,” (with R.C. Chen), Mathematical Methods of Operations Research, 66, 165-179, 2007. Download PDF
“Optimality of Deterministic Policies for Certain Stochastic Control Problems with Multiple Criteria and Constraints,” in Mathematical Control Theory and Finance, A. Sarychev et al (eds), Springer, Berlin, pp. 137-148, 2008. Download PDF
“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), Operations Research Letters, 36, pp. 527-530, 2008. Download PDF
“On Asymptotic Optimality of the Second Order in the Minimax Quickest Detection Problem of Drift Change for Brownian Motion,” (with E.V. Burnaev and A.N. Shiryaev), SIAM Theory Probability Appl., 53, No 3, pp. 519-536, 2009. PDF
“An Inequality for Variances of the Discounted Rewards,” Journal of Applied Probability, (with J. Fei), 46, 1209-1212, 2009. Download PDF
“Compactness of the Space of Non-Randomized Policies in Countable-State Sequential Decision Processes,” (with R.C. Chen), Mathematical Methods of Operations Research., 71, 307-323, 2010. Download PDF
“On Strongly Equivalent Nonrandomized Transition Probabilities,” (with A.B. Piunovskiy), SIAM Theory Probab. Appl., 54, 300-307, 2010. Download PDF
“Total Expected Discounted Reward MDPs: Existence of Optimal Policies,” Wiley Encyclopedia of Operations Research and Management Science, Wiley Online Library, 2011, DOI: 10.1002/9780470400531.eorms0906. Download PDF
“On
“Optimality of Trunk Reservation for an M/M/k/N Queue with Several Customer types and Holding Costs,” (with F. Yang), Probability in Engineering and Informational Sciences, 25, pp. 537-560, 2011. Download PDF
“Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes,” (with U.G. Rothblum), Mathematics of Operations Research, 37, pp. 129-153, 2012. Download PDF
“Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs,” in Optimization, Control, and Applications of Stochastic Systems, D. Hernandez and A. Minjarez (eds), Birkhauser, Springer, pp. 77-98, 2012. Download PDF
“Average-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities,” (with P.O. Kasyanov and N.V. Zadoianchuk), Mathematics of Operations Research 37, pp.591-607, 2012. Download PDF
“Berge's Theorem for Noncompact Image Sets,” (with P.O. Kasyanov and N.V. Zadoianchuk), Journal of Mathematical Analysis and Applications, 397, pp. 255-259, 2013. Download PDF
“Variance Minimization for Constrained Discounted Continuous-Time MDPs with
Exponentially Distributed Stopping Times,” (with J. Fei),
Annals of Operations Research, 208,
pp. 433-450, 2013. Download PDF
“The Multi-Armed Bandit, with Constraints,” (with E.V. Denardo and U.G. Rothblum), Annals of Operations Research , 208, pp. 37-62, 2013. Download PDF
“Strong polynomiality of policy iterations for average-cost MDPs modeling replacement and maintenance problems,” (with J. Huang), Operations Research Letters, 41, pp. 249-251, 2013. Download PDF
“Fatou’s Lemma for Weakly Converging Probabilities,” (with P.O. Kasyanov, N.V. Zadoianchuk) Theory Probab. Appl., 58, pp. 683-689, 2014. PDF
“Extension of Lyapunov's Convexity Theorem to Subranges,” (with P. Dai), Proceedings of the American Mathematical Society, 142, pp. 361-467, 2014. Download PDF
“On Solutions of Kolmogorov’s Equations for Jump Markov Processes,” (with M. Mandava, A.N. Shiryaev), Journal of Mathematical Analysis and Applications, 411, pp. 261-270, 2014. Download PDF
“The Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic Programming,” (with J. Huang), Operations Research Letters, 42, pp. 130-131, 2014. Download PDF
“Berge’s Maximum Theorem for Noncompact Image Sets,“ (with P.O. Kasyanov and M. Voorneveld), Journal of Mathematical Analysis and Applications, 413, pp. 1040-1046, 2014. Download PDF
“Examples Concerning Abel and Cesàro Limits,” (with C.J. Bishop and J. Zhang), Journal of Mathematical Analysis and Applications, 420, pp. 1654-1661, 2014. PDF
“Optimality Conditions for Partially Observable Markov Decision Processes,” (with P.O. Kasyanov and M.Z. Zgurovsky) in Continuous and Distributed Systems: Theory and Applications, M.Z. Zgurovsky, V.A. Sadovnichiy (editors), Springer, Cham, Switzeland, pp. 251-264, 2014.
“Modified Policy Iteration Algorithms are Not Strongly Polynomial for Discounted Dynamic Programming,” (with J. Huang, B. Scherrer), Operations Research Letters, 42, pp. 429-431, 2014. PDF
“Convergence of Probability Measures and Markov Decision Models with Incomplete Information,” (with P.O. Kasyanov and M.Z. Zgurovsky), Proceedings of the Steklov Institute of Mathematics, 287, pp. 96-117, 2014. PDF
“Optimal Switching on and off the Entire Service Capacity of a Parallel Queue,” (with X. Zhang), Probability in Engineering and Informational Sciences, 29, pp. 483-506, 2015. PDF
“Continuity of Minima: Local Results,” (with P.O. Kasyanov), Set-Valued and Variational Analysis, 23, pp. 485-499, 2015. PDF
“Optimal Pricing for a GI/M/k/N Queue with Several Customer Types and Holding Costs,” (with F. Yang), Queueing Systems, 82, pp. 103-120, 2016. PDF
“A Stochastic Search Algorithm for Voltage and Reactive Power
Control with Switching Costs and ZIP Load Model,” (with J. Hu and E. Yuan), Electric
Power Systems Research, 133, pp.
328-337, 2016. PDF
“Partially Observable
Total-Cost Markov Decision Processes with Weakly Continuous Transition
Probabilities,” (with P.O. Kasyanov and M.Z. Zgurovsky), Mathematics of
Operations Research, 41, pp. 656-681, 2016.
PD F
“Uniform Fatou’s Lemma,” (with Kasyanov and M.Z. Zgurovsky), Journal of Mathematical Analysis and Applications, 444, pp. 550-567, 2016. PD F
“Structure of
Optimal Solutions to Periodic-Review Total-Cost Stochastic Inventory Control
Problems,” (with
Y. Liang), ACM SIGMETRICS Performance Evaluation Review, 44, pp. 21-23, 2016. PD F
“Optimality
Conditions for Inventory Control,” Tutorials in Operations Research,
INFORMS, pp. 14-44, 2016. PD F
“On the Reduction of Total-Cost and
Average-Cost MDPs to Discounted MDPs,” (with J. Huang), Naval Research
Logistics, 2017, published online: DOI 10.1002/nav.21743. PD F
“On the Convergence of Optimal Actions for
Markov Decision Processes and the Optimality of (s,S) Policies for Inventory Control,” (with M.E. Lewis), Naval Research
Logistics, 2017, published online: DOI 10.1002/nav.21750. PD F
“Kolmogorov’s Equations for Jump Markov Processes with Unbounded Jump Rates,” (with M. Mandava and A.N. Shiryaev), Annals of Operations Research, 2017, published online: DOI 10.1007/s10479-017-2538-8. PD F
“On the Optimality Equation
for Average Cost Markov Decision Processes and its
Validity for Inventory Control,” (with Y. Liang), Annals of Operations Research, 2017,
published online: DOI
10.1007/s10479-017-2561-9.
PD F
“Structure of Optimal
Solutions to Periodic-Review Total-Cost Inventory Control Problems,” (with Y.
Liang), Annals of Operations Research, 2017, published online: DOI 10.1007/s10479-017-2548-6. PD F
Papers in Conference Proceedings:
“Quality Evaluation and Optimization of Data Transmission
Procedures in Computer Networks” (with L. B. Boguslavsky,
E. Gelembe, Ya. A. Kogan), Proceedings of 7-th National Conference
on Coding Theory and Data Transmission,
“Markov and Semi-Markov Strategies in Dynamic Programming,” Proceedings of 14-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp. 30-31, 1980 (in Russian)
“Nonrandomized Strategies in Markov Decision Processes,” Proceedings of
the Conference on Mathematical Methods of Optimization in Large Economic
and Technical Systems,”
“Discrete Time Stochastic Decision Processes with Arbitrary Real-Valued
Criteria,” Abstracts of 3-rd
“On Markov Decision Models with Measurable Value Functions,” Proceedings of 15-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp.12--13, 1981 (in Russian)
“Markov Policies in Infinite Horizon Dynamic Programming Problems with
Bounded Value Functions” (with I. M. Sonin), Abstracts
of 4-th
“Persistently Nearly Optimal Strategies in Countable State Markov Decision
Chains with the Total Reward Criterion” (with I. M. Sonin),
Abstracts of International Conference on Stochastic Optimization,
“Uniformly Optimal Strategies in Controlled Random Sequences,” Proceedings of 18-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 44, 1984 (in Russian)
“Optimal Control of Parameters in Random Multivariate Access” (with Ya. A. Kogan and A. N. Smirnov), Proceedings of 9-th Russian Conference on Computer Networks, Puschino, 2.1, pp. 41-44, 1984 (in Russian)
“On the Existence of Good Strategies in Stochastic Dynamic Programming,” Abstracts
of 4-th
“Controlled Markov Models with Decision Set Selection,” Proceedings of 19-th Conference on Probability Theor and Mathematical Statistics, Bakuriani, p. 58, 1985 (in Russian)
“Optimization of a railway traffic” (with S. M. Rezer and L. N. Schennikov), Proceedings of the Conference “Optimization and Control of Transportation Systems”, Gomel, pp. 169-171, 1985 (in Russian).
“Additive Functionals of Jump Processes: a
Discrete Time Representation,” Abstracts of I-st
World Congress of Bernoulli Society,
“Parametric Dynamic Programming,” Proceedings of 20-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 56, 1986 (in Russian)
“Optimality of Pure Strategies in Stochastic Decision Processes,” Proceedings of the 29-th IEEE Conference on Decision and Control, 4, pp. 2149-2154, 1990
“Weighted Discounted Dynamic Programming” (with A. Shwartz), Proceedings of the 30-th IEEE Conference on Decision and Control, 2, pp. 485-486, 1991.
“Optimal Switching Policies for M/G/1 Queues with Two Performance Criteria”
(with D.J. Kim), Proceedings of the International Conference on Operations
Research, pp. 227-232, 1994,
“Multiple Objective Optimal Control of Production Systems with Stochastic Demand” (with D.J. Kim), Proceedings of the First Regional Symposium on Manufacturing Science and Technology, Stony Brook, NY, pp. 37-43, 1995.
“Optimization of Jump Stochastic Systems,” Proceedings of the 1996 NSF Design and Manufacturing Grantees Conference, Albuquerque, NM, pp.497-498, 1996.
“Weighted Discounted Stochastic Games with Perfect Information,” (with E. Altman and A. Shwartz), Proceedings of the Seventh International Symposium on Dynamic Games and Applications, December 16-18, Kanagawa, Japan, 1, 18 - 31, 1996.
“Optimization of Discounted Constrained Jump Stochastic Systems,” Proceedings of the 1997 NSF Design and Manufacturing Grantees Conference, Seattle, WA, pp. 457-458, 1997.
“Geographic Scheduling System for Customer Service,” (with A. Zhukovsky), Proceedings of the 18th National Conference of the American Society of Engineering Management, Virginia Beach, pp. 561 - 563, 1997.
“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles,” Proceedings of the 36-th IEEE Conference on Decision and Control, 3, pp. 2821 - 2826, 1997.
“Optimization of Jump Stochastic Systems: Constrained Dynamic Programming
and Hamiltonian Cycles,” Proceedings of the 1997 NSF Design and
Manufacturing Grantees Conference, NSF,
“Perturbed Zero-Sum Games with Applications to Dynamic Games” (with E. Altman, J. Filar, and V. Gaitsgory), Proceedings of the Seventh International Symposium on Dynamic Games and Applications, July 5-8, Maastricht, The Netherlands, pp. 45 - 51, 1998.
“Optimization of Jump Stochastic Systems,” Proceedings
of the 1999 NSF Design and Manufacturing Grantees Conference,
“New Approach to Optimization of Discounted Stochastic Continuous-Time Discrete-Event Systems,” Proceedings of the 38‑th IEEE Conference on Decision and Control 1, pp. 937 – 941, 1999.
“Optimality of Pure Strategies in Continuous Dynamic
Programming Problems with Multiple Criteria, “ (with
A.B. Piunovskiy)
Proceedings of the 2000 NSF Design and Manufacturing Grantees
Conference,
“Nonatomic Total Reward Markov Decision Processes with Multiple Criteria,” (with A.B. Piunovskiy), Proceedings of the 39‑th IEEE Conference on Decision and Control, 1, pp. 723 – 728, 2000.
“Nonatomic Total Reward Markov
Decision Processes,” (with A. B. Piunovskiy), Proceedings
of the 2001 NSF Design and Manufacturing Research Conference,
“Optimization of Jump Stochastic Systems,” Proceedings of the 2002 NSF Design and Manufacturing Research Conference, San Juan, Puerto Rico, January 7-10, 2002 (CD).
“Sensor Resource Management for an Airborne Early Warning Radar,” (with M. Bender, M. Curry, D. Huang, T. Koutsoudis, J. Bernstein), Signal and Data Processing of Small Targets, Proceedings of SPIE – The International Society of Optical Engineering, 2-4 April 2002, Orlando, 4728, pp. 145-156, 2002.
“Online Scheduling with Hard Constraints: a Semi-Markov Decision Process Approach,” (with M. Curry), Control and Applications, Proceedings of the IASTED International Conference, Cancun, MX, May 20-22, pp. 104-109, 2002.
“Load Pocket Forecasting,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Second IASTED International Conference, Crete, Greece, June 25-28, pp. 50-54, 2002. Download PDF
“Constrained Finite Continuous-Time Markov Decision Processes with Average Rewards,” Proceedings of IEEE 2002 Conference on Decisions and Control, Las Vegas, December 10-13, pp. 3805-3810, 2002. Download PDF
“Average reward constrained continuous-time finite Markov
decision processes,” Proceedings
of the 2002 NSF Design and Manufacturing Research Conference,
“Statistical Load Modeling,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Seventh IASTED International Multi-Conference, Palm Springs, CA, February 24-26, pp. 88-91, 2003. Download PDF
“Control of Finite Continuous-Time Markov Chains with
Average Rewards and
Constraints,” Kolmogorov and Contemporary Mathematics,
Abstracts,
“Online Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of the 42nd IEEE Conference on Decision and Control,
“Real Time Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of the 2004 NSF
Design and Manufacturing Research Conference,
“Load Pocket Forecasting Software,” (with D. Genethliou, J.T. Hajagos, B.G. Irrgang, and R.R. Rossin),
Proceedings of 2004 IEEE Power Systems
“Optimality Inequalities for Average Cost MDPs and their Inventory Control Applications,” (with M.E. Lewis), Proceedings of the 43nd IEEE Conference on Decision and Control, Bahamas, December 14-17, 2004, pp. 4411-4416, Download PDF
“Optimality of Randomized Trunk Reservation for a Problem
with a Single Constraint,” (with X. Fan-Orzechowski), Proceedings
of the 2004 NSF Design and Manufacturing Research Conference,
“Buffer Insertion for Bridges and Optimal Buffer Sizing for Communication Sub-System of Systems-on-Chip,” (with S.
Kallakuri, A. Doboli), Proceedings of Design Automation and Test in
Europe,
827.
“A Continuous Time Markov Decision Process Based On-Chip
Buffer Allocation Methodology,” (with S. Kallakuri,
N. Thepayasuwan, and A. Doboli), Proceedings of GLSVLSI (Great Lakes
Symposium on VLSI),
“Optimal Admission Control for a Markovian Queue under the Quality of Service Constraint,” (with Xiaofei Fan-Orzechowski), Proceedings of the 44nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, December 12-15, 2005, pp. 1729-1734.
“On State Space Reduction in Sequential Decision Processes,” Proceedings of the 44nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, December 12-15, 2005, pp. 7314-7319.
“Non-Randomized
Control of Constrained Markov Decision Processes,” (with R.C. Chen), Proceedings
of the 2006 American Control Conference,
“Optimality of Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), Proceedings of the 2006 NSF Design, Service, and Manufacturing Grantees and Research Conference, St. Louis, MO, July 24-27, 2006 (CD).
“Optimality of Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), Proceedings of the 2006 NSF Design, Service, and Manufacturing Grantees and Research Conference, St. Louis, MO, July 24-27, 2006 (CD).
“Quickest
Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Proceedings
of the 2008 NSF Engineering
Research and Innovation Conference,
“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, TN, January 7-10, 2008 (CD).
“Polynomial Classification Algorithms for Markov Decision Processes,” (with Fenghsu Yang), Proceedings of the 47th IEEE Conference on Decision and Control, Cancun, Mexico, December 9-11, 2008, pp.4485 – 4490.
“Adaptive Computation of Optimal Nonrandomized Policies in Constrained Average-Reward MDPs,” 2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPLR 2009) Proceedings, March 30 – April 2, 2009, Nashville, TN, pp. 96 – 100
“Efficient Computation of Optimal
Nonrandomized Policies in Constrained Average-Reward Markov Decision
Processes,” Proceedings of NSF CMMI Research and Innovation Conference 2009,”
“Optimal Admission for a Queue with Several Types of Customers and Holding Costs,” (with F. Yang), In Modern Trends in Controlled Stochastic Processes: Theory and Applications, A.B. Piunovskiy (ed.), pp. 304 – 322, Luniver Press, UK, 2010
“Optimal Admission to an M/M/k/N Queue with Several Customer Types and Holding Costs,” (with F. Yang), Proceedings of the 49th IEEE Conference on Decision and Control, Atlanta, GA, December 15-17, 2010, pp.2791 – 2796.
“Variance Minimization for Constrained Discounted Continuous-Time MDPs with Exponentially Distributed Stopping Times,” (with J. Fei), Proceedings of NSF CMMI Research and Innovation Conference 2011,” Atlanta, GA, January 4-7, 2011 (CD).
“Optimal Admission Policies for M/M/k/N Queue with Multiple Customer Types and Holding Costs,” (with F. Yang), Proceedings of NSF CMMI Research and Innovation Conference 2011,” Atlanta, GA, January 4-7, 2011 (CD).
“Smart Grid Software Applications for Distribution Network Forecasting,” (with Jun Fei, J.T. Hajagos, and R.J. Rossin), Proceedings of ENERGY 2011: The First International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, May 21-27, Venice/Mestre, Italy, 2011, pp. 76-80. PDF
“A Rolling Horizon Approach to Distribution Feeder Reconfiguration with Switching Costs,” (J. Hu, and K. Huang), Proceedings of the Second IEEE International Conference on Smart Grid Communications, Brussels, Belgium, 17-20 October 2011, pp. 357-362.
“Switching
on and off the full capacity of an M/M/infinity queue,” (with X. Zhang), Proceedings of the 50th IEEE Conference on Decision and Control and
European Control Conference,
“A Parallel Rolling Horizon Scheme for Large Scale Security Constrained Unit Commitment Problems with Wind Power Generation,” (with J. Hu and E. Yuan), Proceedings of ENERGY 2012: The Second International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, March 25-30, 2012, St. Maarten, pp. 13-21. Downloiad PDF
E.A. Feinberg, and U.G. Rothblum, “Splitting in a finite Markov decision problem.” (with E.V Denardo and U.G. Rothblum), SIGMETRICS Performance Evaluation Review, 39, p. 38, 2012.
“The multi-armed bandit, with constraints.” (with E.V Denardo and U.G. Rothblum), SIGMETRICS Performance Evaluation Review, 39, p. 39, 2012.
“Sufficiency of Markov policies for continuous-time Markov decision processes and solutions to Kolmogorov’s forward equation for jump Markov processes,” (with M. Mandava and A.N. Shiryaev), Proceedings of the 52th IEEE Conference on Decision and Control and European Control Conference, Florence, Italy, December 10-13, pp. 5716-5721, 2013.
“Optimality
conditions for total-cost partially observable Markov decision processes,”
(with P.O. Kasyanov, M.Z. Zgurovsky), Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference,
”Voltage
and reactive power control using approximate stochastic annealing,” (with J. Hu, E. Yuan), Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference,
“Convergence of Value Iterations for Total-Cost MDPs and POMDPs with General State and Action Sets,” (with P.O. Kasyanov and M.Z. Zgurovsky), Proceedings of 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), IEEE SSCI 2014, Orlando, FL, December 9-12, pp. 41-48, 2014.
“Sensor Placement for Real-Time Power Flow Calculations in Transmission Networks,” (with M. Li, R. Samulyak, B. Fardanesh, G. Stefopoulos), ENERGY 2015: The Fifth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, Rome, Italy, May 24-29, pp. 12-17, 2015.