**Eugene
A. Feinberg**

Department of Applied Mathematics and
Statistics

State University of New York at Stony Brook

Stony Brook, NY, 11794-3600

tel: (631) 632-7189

fax: (631) 632-8490

email: Eugene.Feinberg@sunysb.edu

Handbook of Markov Decision Processes: Methods and
Algorithms (with A. Shwartz, editors), Kluwer,

**Papers in Refereed Journals and Books:**

“On Controlled Finite State Markov Processes with Compact Control Sets,” __SIAM____
Theory__ __Probability Appl.__, **20**, pp. 856-862, 1975 PDF

“Control of Queueing Systems” (with M. A. Feinberg), __Radioelectronics__,
no. 3, pp. 3-34, 1975 (in Russian)

“On e-optimal Control of Discrete
Time Markov Chains,” __Transactions of Moscow__ __Institute of Transport
Engineering__, **510**, pp. 33-36, 1976 (in Russian)

“Railway Transport Radio Communication Models” (with L. M. Zuravliova), __Railway__ __Transport__, no. 6, pp.
81-83, 1976 (in Russian)

“On Finite Controlled Markov Chains,”__ Uspehi Math.____ Nauk__,
**32**, no 3, pp. 181-182, 1977 (in Russian)

“The Existence of a Stationary e-optimal
Policy for a Finite Markov Chain,” __SIAM Theory__ __Probability Appl.__,
**23**, pp. 297-313, 1978 PDF

“On Homogeneous Markov Decision Models with Continuous Time and Finite or
Countable State Spaces” (with A. A. Yushkevich), __SIAM____
Theory Probability Appl.__, **24**, pp. 156-161, 1979 PDF PDF

“On Ergodic Conditions for Finite Controlled
Markov Chains with Arbitrary Decision Sets,” __Transactions of __,

“An e-optimal Control of a Finite
Markov Chain,” __SIAM Theory Probability Appl.__, **25**, pp. 70-81, 1980
PDF

“Optimization Models for Railway Radio Communication Procedures” (with L. M.
Zuravliova), __Railway Transport__, no. 4, pp.
65-67, 1980 (in Russian)

“An Optimal Strategy for Activation of a Service Device in One Class of
Systems with Group Service” (with L. B. Boguslavsky),
__Problems of Information Transmission__, no. 3,pp.
243-253, 1980.

“Optimization of Number of Vans in a Group in Railway Operations” (with S.
M. Rezer), __Transactions of Moscow Institute of
Transport Engineering__,** 653**, pp. 55-63, 1981 (in Russian)

“Nonrandomized Markov and Semi-Markov Strategies in Dynamic Programming,” __SIAM____
Theory Probability Appl.__, **27**, pp. 116-126, 1982 PDF

“Controlled Markov Processes with Arbitrary Numerical
Criteria,” __SIAM Theory Probability__ __Appl.__, **27**, pp. 486-503,
1982. PDF

“Stationary Strategies in Dynamic Programming with a Countable State Space”
(with __Studies
on Mathemat. Econom. and Control Theory__, Central Econom.-Math. Institute Press,

“Stationary and Markov Policies in Countable State Dynamic Programming”
(with __Lecture
Notes in Math.__,** 1021**, pp. 111-129, 1982 PDF

“Sufficient Classes of Strategies in Controllable Markov Chains with Total
Criterion” (with __Soviet Math. Dokl.,__**
29**, no. 2, pp. 308-311, 1984

“Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming”
(with I. M. Sonin), __Statistics and Control of
Stochastic Processes__, Steklov Seminar,
Optimization Software, NY, pp. 69-101, 1985

“Optimal Specification of Railway Platform Sections,” __Transactions of
Moscow Institute__ __of Transport Engineering__, **758**, pp. 58-62,
1984 (in Russian)

“Optimal Control of Parameters of Multiple Access
Protocols” (with Ya. A. Kogan and A. N. Smirnov), __Automatic Control and Computer
Science__, no. 6, pp. 48-54, 1984.

“Optimal Control by the Retransmission Probability in
Slotted ALOHA Systems” (with Ya. A. Kogan and A. N. Smirnov), __Performance Evaluation__,**
5**, pp. 85-96, 1985

“Sufficient Classes of Strategies in Discrete Dynamic
Programming. I: Decomposition of Randomized Strategies and Embedded
Models,” __SIAM____
Theory Probability Appl.__, **31**, pp. 658-668, 1986 PDF

“The Structure of Persistently Nearly Optimal Strategies
in Stochastic Dynamic Programming,” __Lecture Notes in Control and Inform.____
Sci.__, **81**, pp. 22-31, 1986

“Sufficient Classes of Strategies in Discrete Dynamic
Programming. II: Locally Stationary Strategies,” __SIAM____ Theory Probability
Appl.__, **32**, No. 3, pp. 478-493, 1987 PDF

“Parametric Stochastic Dynamic Programming,” __Statistics and Control of
Stochastic__ __Processes__, Steklov Seminar,**
2**, Optimization Software, NY, pp. 103-120, 1989

“Nonrandomized Strategies in Stochastic Decision Processes,” __Annals of
Operations__ __Research__, **29**, pp. 315-332, 1991

“A Markov Decision Model of a Search Process,” __Contemporary Mathematics__,**
125**, pp. 87-96, 1992

“On Stationary Strategies in Borel Dynamic
Programming,” __Mathematics of Operations__ __Research__, **17**, pp.
393-397, 1992

“__Stochastic Processes Appl.__, **49**,
pp. 159-177, 1994

“Markov Decision Models with Weighted Discounted Criteria” (with A. Shwartz), __Mathematics of Operations Research__, **19**,
pp. 152-168, 1994

“A Generalization of “Expectation Equals Reciprocal of Intensity” to Nonstationary Distributions,”__ J. Appl. Probability__, **31**,
pp. 262-267, 1994. . Download PS or PDF

“Constrained Semi-Markov Decision Processes with Average Rewards,” __ZOR -__
__Mathematical Methods of Operations Research__,** 39**, pp. 257-288,
1994

“Optimality of Randomized Trunk Reservation” (with M. Reiman),
__Probability in__ __Engineering and Informational Sciences__, **8**,
pp. 483-489, 1994

“Constrained Markov Decision Models with Weighted Discounting” (with A. Shwartz), __Mathematics of Operations Research__, **20**,
pp. 302-320, 1995

“Bicriterion Optimization of an M/G/1 Queue with a
Removable Server” (with D.J. Kim), __Probability in Engineering and
Informational Sciences__, **10**, pp. 57-73, 1996. Download PDF

“Letter to the Editor,” __Operations Research__, **44**, p. 526, 1996

“Constrained Discounted Dynamic Programming” (with A. Shwartz),
__Mathematics of__ __Operations Research__, **21**, pp. 922-945, 1996.
. Download PS
or DVI

“Notes on Equivalent Stationary Policies in Markov Decision Processes with
Total Rewards” (with I.M. Sonin), __ZOR -
Mathematical Methods of Operations Research__, **44**, pp. 205-221, 1996.

“On Measurability and Representation of Strategic Measures in Markov
Decision Processes”, in __Statistics, Probability and Game Theory Papers in
Honor of David__ __Blackwell__ (eds. T.S. Ferguson et al.), IMS Lecture Notes - Monograph
Series, **30**, pp. 29-43, 1996 PDF

“Stochastic Monotonicity for Stationary Recurrence Times of __Annals
of Applied Probability__, **7**, pp. 326-339, 1997. Download PDF

“On Future Organization of Hybrid Chip Manufacturing”
(with S. Luryi), in __Future Trends in Microelectronics ,__ J. Wiley & Sons (eds.

“Constrained Dynamic Programming with
Two Discount Factors: Applications and an Algorithm”; (with A. Shwartz), __IEEE Transactions on Automatic Control__, **44**, pp. 628-631, 1999. Download PS or DVI

“ Weighted Discounted Stochastic Games with Perfect
Information” (with __Annals of the International Society of Dynamic
Games__, **5**, pp. 303-324, 2000.
Download PDF

“Constrained Discounted Markov Decision Processes and
Hamiltonian Cycles” __Mathematics____
of Operations Research__, **25**, pp. 130-140, 2000. Download PDF

“A Note on the Existence of Optimal
Policies in Continuous Dynamic Programs with Compact Action Sets” (with R.
Cavazos-Cadena and R. Montes-de-Oca),
__Mathematics of Operations Research__, **25**, pp. 657-666, 2000. Download PS or DVI

“Multiple Objective Nonatomic
Markov Decision Processes with Total Reward Criteria” (with A.B. Piunovskiy) ), __Journal of Mathematical Analysis and
Applications__, **247**, pp. 45-66,
2000. Download PS or DVI

“Perturbed Zero-Sum Games with Applications to Stochastic
and Repeated Games” (with __Annals
of the International Society of Dynamic Games__, **6**, pp. 165-181,
2001. Download PS or DVI

“Introduction” (with A. Shwartz), Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp. 1-17, 2002.

“Mixed Criteria” (with A. Shwartz) , __Handbook of Markov Decision Processes: Methods and
Applications,__ Kluwer, pp 209-230, 2002. Download PS or DVI

“Total Reward Criteria,” Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 173-207, 2002.

“Constrained Discounted Semi-Markov
Decision Processes,” Markov Processes and Controlled Markov Chains (Z.
How, J.A. Filar, A. Chen, eds), Kluwer,

“Nonatomic Total Reward Markov
Decision Processes with Multiple Criteria” (with A.B. Piunovskiy), __Journal____
of Mathematical Analysis and Applications__, **273**, 93-111, 2002. Download PDF

“Optimality of D-policies for an M/G/1 Queue with a
Removable Server” (with O. Kella), __Queueing____
Systems – Theory and Applications (QUESTA)__ **42**, pp. 355-376,
2002. Download PDF

“Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event
Approach,” __Mathematics of Operations Research__, **29**, pp. 492-524,
2004. Download PDF

“Optimality of Four-Threshold Policies in Inventory Systems with Customer
Returns and Borrowing/Storage Options,” (with M.E. Lewis), __Probability____ in Engineering
and Informational Sciences__, **19**, 45-71, 2005. Download PDF

“Generalized Pinwheel Problem,” (with M.T. Curry), __Mathematical____ Methods of
Operations Research__, **62, **99-122, 2005. Download PDF

“Load Forecasting,” (with D. Genethliou), __Applied Mathematics for Restructured
Electric Power Systems: Optimization, Control, and Computational Intelligence__
(J. H. Chow, F.F. Wu, and J.J. Momoh, eds.), Spinger, pp. 269-285, 2005. Download PDF

“On Essential Information in Stochastic Decision
Processes,” __Mathematical Methods of Operations Research__, **62, **399-410,
2005. Download PDF

“On Dvoretzky-Wald-Wolfowitz Theorem on Nonrandomized
Statistical Decisions,” (with A.B. Piunovskiy), __SIAM
____Theory__ __Probability Appl.__, **50**, 463-466, 2006. Download PDF

“Optimality
of Randomized Trunk Reservation for a Problem with a Single Constraint,”
(with X. Fan-Orzechowski), __Advances____ in Applied Probability__, **38**, 199-220, 2006. Download PDF

“Quickest Detection of Drift Change for
Brownian Motion in Generalized Bayesian and Minimax
Settings,” (with A.N. Shiryaev), Statistics
& Decisions, **24**, 445-470, 2006. Download PDF

“Optimality of Randomized Trunk Reservation for a Problem with
Multiple Constraints,” (with X. Fan-Orzechowski), __Probability____
in Engineering and Informational Sciences__, **21**, 189-200, 2007. Download PDF

“Optimality Inequalities for Average Cost Markov Decision Processes and the
Stochastic Cash Balance Problem,” (with M.E. Lewis), __Mathematics of
Operations Research__, **32**,
769-783, 2007. Download PDF

“Non-Randomized Policies for Constrained Markov Decision Processes,” (with
R.C. Chen), __Mathematical Methods of Operations Research__, **66**, 165-179, 2007. Download PDF

“Optimality of Deterministic Policies for
Certain Stochastic Control Problems with Multiple Criteria and Constraints,” in
__Mathematical Control Theory and Finance__, A. Sarychev
et al (eds), Springer, Berlin, pp. 137-148,
2008. Download PDF

“On Polynomial Classification Problems for Markov Decision
Processes,” (with Fenghsu Yang), __Operations
Research Letters__, **36**, pp.
527-530, 2008. Download PDF

“On
Asymptotic Optimality of the Second Order in the Minimax
Quickest Detection Problem of Drift Change for Brownian Motion,” (with E.V. Burnaev and A.N. Shiryaev), __SIAM____
Theory Probability Appl__., **53**, No
3, pp. 519-536, 2009. PDF

“An Inequality for Variances of the Discounted
Rewards,” __Journal of Applied Probability__, (with J. Fei), ** 46**, 1209-1212, 2009. Download PDF

“Compactness
of the Space of Non-Randomized Policies in Countable-State Sequential Decision
Processes,” (with R.C. Chen), __Mathematical
Methods of Operations Research.,__ **71**, 307-323, 2010. Download PDF

“On
Strongly Equivalent Nonrandomized Transition Probabilities,” (with A.B. Piunovskiy), __SIAM____ Theory Probab. Appl., 54, 300-307,
2010.__ Download PDF

“Total
Expected Discounted Reward MDPs: Existence of Optimal Policies,” __Wiley____
Encyclopedia of Operations Research and Management Science__, Wiley Online
Library, 2011, DOI: 10.1002/9780470400531.eorms0906. Download PDF

“On __Proceedings____ of the American
Mathematical Society__, **139**, pp.
4497-4511, 2011. Download PDF

“Optimality of Trunk Reservation for an M/M/k/N Queue with Several Customer
types and Holding Costs,” (with F. Yang), __Probability in Engineering and
Informational Sciences__, **25**, pp.
537-560, 2011. Download
PDF

“Splitting Randomized Stationary Policies in Total-Reward Markov Decision
Processes,” (with U.G. Rothblum), __Mathematics____ of Operations
Research__, 37, pp. 129-153, 2012.
Download PDF

“Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward
Rates to Discrete-Time Total-Reward MDPs,” in __Optimization, Control, and
Applications of Stochastic Systems__, D. Hernandez and A. Minjarez
(eds), Birkhauser,
Springer, pp. 77-98, 2012. Download PDF

“Average-Cost Markov Decision
Processes with Weakly Continuous Transition Probabilities,” (with P.O. Kasyanov
and N.V. Zadoianchuk), __Mathematics____ of Operations
Research__ **37**, pp.591-607,
2012. Download PDF

“Berge's
Theorem for Noncompact Image Sets,” (with P.O.
Kasyanov and N.V. Zadoianchuk), __Journal____ of Mathematical
Analysis and Applications__, **397**, pp. 255-259, 2013. Download PDF

“Variance Minimization for Constrained Discounted Continuous-Time MDPs with
Exponentially Distributed Stopping Times,” (with J. Fei),
__Annals of Operations Research, 208__,
pp. 433-450, 2013. Download PDF

“The Multi-Armed Bandit, with Constraints,” (with E.V. Denardo
and U.G. Rothblum), __Annals____ of Operations
Research__ , **208**, pp. 37-62,
2013. Download PDF

“Strong polynomiality of policy
iterations for average-cost MDPs modeling replacement and maintenance
problems,” (with J. Huang), __Operations Research Letters__, 41, pp.
249-251, 2013. Download PDF

“Fatou’s Lemma for Weakly Converging
Probabilities,” (with P.O. Kasyanov, N.V. Zadoianchuk)
__Theory Probab.____ Appl., 58, pp. 683-689,
2014.__ PDF

“Extension
of Lyapunov's Convexity Theorem to Subranges,” (with P. Dai), __Proceedings____ of the American
Mathematical Society__, **142**, pp. 361-467, 2014. Download PDF

“On
Solutions of Kolmogorov’s Equations for Jump Markov Processes,” (with M. Mandava, A.N. Shiryaev), __Journal____
of Mathematical Analysis and Applications__, **411**, pp. 261-270, 2014.
Download PDF

“The
Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic
Programming,” (with J. Huang), __Operations Research Letters__, **42**,
pp. 130-131, 2014. Download PDF

“Berge’s
Maximum Theorem for Noncompact Image Sets,“ (with P.O. Kasyanov and M. Voorneveld), __Journal of Mathematical Analysis and
Applications__, **413**, pp. 1040-1046, 2014. Download PDF

“Examples
Concerning Abel and Cesàro Limits,” (with C.J. Bishop
and J. Zhang), __Journal____
of Mathematical Analysis and Applications__, **420**, pp. 1654-1661, 2014. PDF

“Optimality
Conditions for Partially Observable Markov Decision Processes,” (with P.O.
Kasyanov and M.Z. Zgurovsky) in __Continuous and
Distributed Systems: Theory and Applications__, M.Z. Zgurovsky,
V.A. Sadovnichiy (editors), Springer,
Cham, Switzeland, pp. 251-264, 2014.

“Modified
Policy Iteration Algorithms are Not Strongly Polynomial for Discounted Dynamic
Programming,” (with J. Huang, B. Scherrer), __Operations
Research Letters__, **42**, pp. 429-431, 2014. PDF

“Convergence
of Probability Measures and Markov Decision Models with Incomplete
Information,” (with P.O. Kasyanov and M.Z. Zgurovsky),
__Proceedings of the Steklov Institute of
Mathematics,__ **287**, pp. 96-117, 2014.
PDF

“Optimal
Switching on and off the Entire Service Capacity of a Parallel Queue,” (with X.
Zhang), __Probability____
in Engineering and Informational Sciences__,
**29**, pp. 483-506, 2015. PDF

“Continuity of Minima: Local Results,” (with P.O. Kasyanov), __Set-Valued
and Variational Analysis__, **23**, pp. 485-499, 2015. PDF

“Optimal Pricing for a GI/M/k/N Queue with Several Customer Types
and Holding Costs,” (with F. Yang), __Queueing____
Systems__, **82**, pp. 103-120, 2016. PDF

“A Stochastic Search Algorithm for Voltage and Reactive Power
Control with Switching Costs and ZIP Load Model,” (with J. Hu and E. Yuan), __Electric
Power Systems Research__, **133**, pp.
328-337, 2016. PDF

“Partially Observable
Total-Cost Markov Decision Processes with Weakly Continuous Transition
Probabilities,” (with P.O. Kasyanov and M.Z. Zgurovsky), __Mathematics of
Operations Research__, 41, pp. 656-681, 2016.
PD F

“Uniform Fatou’s
Lemma,” (with Kasyanov and M.Z. Zgurovsky), __Journal of Mathematical Analysis and
Applications__, **444**, pp. 550-567, 2016. PD F

“Structure of
Optimal Solutions to Periodic-Review Total-Cost Stochastic Inventory Control
Problems,” (with
Y. Liang), __ACM SIGMETRICS Performance Evaluation Review__, **44**, pp. 21-23, 2016. PD F

“Optimality
Conditions for Inventory Control,” __Tutorials in Operations Research__,
INFORMS, pp. 14-44, 2016. PD F

“On the Reduction of Total-Cost and
Average-Cost MDPs to Discounted MDPs,” (with J. Huang), __Naval Research
Logistics__, 2017, published online: DOI 10.1002/nav.21743. PD F

“On the Convergence of Optimal Actions for
Markov Decision Processes and the Optimality of (s,S) Policies for Inventory Control,” (with M.E. Lewis), __Naval Research
Logistics__, 2017, published online: DOI 10.1002/nav.21750. PD F

“Kolmogorov’s Equations for Jump
Markov Processes with Unbounded Jump Rates,” (with M. Mandava
and A.N. Shiryaev), __Annals of Operations Research__,
2017, published online: DOI 10.1007/s10479-017-2538-8. PD F

“On the Optimality Equation
for Average Cost Markov Decision Processes and its
Validity for Inventory Control,” (with Y. Liang), __Annals of Operations Research__, 2017,
published online: DOI
10.1007/s10479-017-2561-9.
PD F

“Structure of Optimal
Solutions to Periodic-Review Total-Cost Inventory Control Problems,” (with Y.
Liang), __Annals of Operations Research__, 2017, published online: DOI 10.1007/s10479-017-2548-6. PD F

**Papers in Conference Proceedings:**

“Quality Evaluation and Optimization of Data Transmission
Procedures in Computer Networks” (with L. B. Boguslavsky,
E. Gelembe, Ya. A. Kogan), __Proceedings of 7-th__ __National Conference
on Coding Theory and Data Transmission__, **3**, pp. 25-30, 1978 (in
Russian)

“Markov and Semi-Markov Strategies in Dynamic Programming,” __Proceedings
of 14-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, pp. 30-31, 1980 (in Russian)

“Nonrandomized Strategies in Markov Decision Processes,” __Proceedings of
the__ __Conference on Mathematical Methods of Optimization in Large Economic
and Technical__ __Systems__,”

“Discrete Time Stochastic Decision Processes with Arbitrary Real-Valued
Criteria,” __Abstracts of 3-rd __,

“On Markov Decision Models with Measurable Value Functions,” __Proceedings
of 15-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, pp.12--13, 1981 (in Russian)

“Markov Policies in Infinite Horizon Dynamic Programming Problems with
Bounded Value Functions” (with I. M. Sonin), __Abstracts
of 4-th USSR - Japan Symposium on__

“Persistently Nearly Optimal Strategies in Countable State Markov Decision
Chains with the Total Reward Criterion” (with I. M. Sonin),
__Abstracts of International Conference on Stochastic Optimization__,

“Uniformly Optimal Strategies in Controlled Random Sequences,” __Proceedings
of 18-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, p. 44, 1984 (in Russian)

“Optimal Control of Parameters in Random Multivariate
Access” (with Ya. A. Kogan
and A. N. Smirnov), __Proceedings of 9-th Russian Conference on Computer
Networks__, Puschino, **2.1**, pp. 41-44, 1984
(in Russian)

“On the Existence of Good Strategies in Stochastic Dynamic Programming,” __Abstracts
of__ __4-th __, 4, pp. 70-72,
1985

“Controlled Markov Models with Decision Set Selection,” __Proceedings of
19-th__ __Conference on Probability Theor and
Mathematical Statistics__, Bakuriani, p. 58, 1985
(in Russian)

“Optimization of a railway traffic” (with S. M. Rezer and L. N. Schennikov), __Proceedings__
__of the Conference “Optimization and Control of Transportation Systems”__,
Gomel, pp. 169-171, 1985 (in Russian).

“Additive Functionals of Jump Processes: a
Discrete Time Representation,” __Abstracts of__ __I-st
World Congress of Bernoulli Society__, **1**, p. 289, 1986

“Parametric Dynamic Programming,” __Proceedings of 20-th Conference on
Probability__ __Theory and Mathematical Statistics__, Bakuriani,
p. 56, 1986 (in Russian)

“Optimality of Pure Strategies in Stochastic Decision Processes,” __Proceedings
of the__ __29-th IEEE Conference on Decision and Control__, **4**, pp.
2149-2154, 1990

“Weighted Discounted Dynamic Programming” (with A. Shwartz),
__Proceedings of the__ __30-th IEEE Conference on Decision and Control__,**
2**, pp. 485-486, 1991.

“Optimal Switching Policies for M/G/1 Queues with Two Performance Criteria”
(with D.J. Kim), __Proceedings of the International Conference on Operations
Research__, pp. 227-232, 1994,

“Multiple Objective Optimal Control of Production Systems
with Stochastic Demand” (with D.J. Kim),__ Proceedings of the First Regional
Symposium on Manufacturing Science and Technology__, Stony Brook, NY, pp.
37-43, 1995.

“Optimization of Jump Stochastic Systems,”__ Proceedings
of the 1996 NSF Design and Manufacturing Grantees Conference__, Albuquerque,
NM, pp.497-498, 1996.

“Weighted Discounted Stochastic Games with Perfect Information,” (with E.
Altman and A. Shwartz), __Proceedings of the Seventh
International Symposium on Dynamic Games__ __and Applications__, December
16-18, Kanagawa, Japan,** 1**, 18 - 31, 1996.

“Optimization of Discounted Constrained Jump Stochastic
Systems,” __Proceedings of the 1997 NSF Design and Manufacturing Grantees
Conference__, Seattle, WA, pp. 457-458, 1997.

“Geographic Scheduling System for Customer Service,” (with A. Zhukovsky), __Proceedings of the 18 ^{th} National
Conference of the American Society of Engineering__

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles,” __Proceedings
of the 36-th IEEE Conference on Decision and Control__, **3**, pp. 2821 -
2826, 1997.

“Optimization of Jump Stochastic Systems: Constrained Dynamic Programming
and Hamiltonian Cycles,”__ Proceedings of the 1997 NSF Design and
Manufacturing Grantees Conference__, NSF,

“Perturbed Zero-Sum Games with Applications to Dynamic
Games” (with E. Altman, J. Filar, and V. Gaitsgory), __Proceedings
of the Seventh International Symposium on Dynamic Games and Applications__, July 5-8, Maastricht, The Netherlands, pp. 45 - 51,
1998.

“Optimization of Jump Stochastic Systems,” __Proceedings
of the 1999 NSF Design and Manufacturing Grantees Conference__,

“New Approach to Optimization of
Discounted Stochastic Continuous-Time Discrete-Event Systems,” __Proceedings
of the 38‑th IEEE Conference on Decision and Control__ **1**, pp.
937 – 941, 1999.

“Optimality of Pure Strategies in Continuous Dynamic
Programming Problems with Multiple Criteria, “ (with
A.B. Piunovskiy)
__Proceedings of the 2000 NSF Design and Manufacturing Grantees
Conference__,

“Nonatomic Total Reward Markov
Decision Processes with Multiple Criteria,” (with A.B. Piunovskiy),
__Proceedings of the 39‑th IEEE Conference on Decision and Control__,
1, pp. 723 – 728, 2000.

“Nonatomic Total Reward Markov
Decision Processes,” (with A. B. Piunovskiy), __Proceedings
of the 2001 NSF Design and Manufacturing Research Conference__,

“Optimization of Jump Stochastic
Systems,” __Proceedings of the 2002 NSF Design and Manufacturing Research
Conference__, San Juan, Puerto Rico, January 7-10, 2002 (CD).

“Sensor Resource Management for an Airborne Early Warning
Radar,” (with M. Bender, M. Curry, D. Huang, T. Koutsoudis,
J. Bernstein), __Signal and Data
Processing of Small Targets__, Proceedings of SPIE – The International
Society of Optical Engineering, 2-4 April 2002, Orlando, **4728**,
pp. 145-156, 2002.

“Online Scheduling with Hard
Constraints: a Semi-Markov Decision Process Approach,” (with M. Curry), __Control
and Applications__, Proceedings of the IASTED International Conference,
Cancun, MX, May 20-22, pp. 104-109, 2002.

“Load Pocket Forecasting,” (with D. Genethliou
and J.T. Hajagos), __Power and Energy Systems__,
Proceedings of the Second IASTED International Conference, Crete, Greece, June
25-28, pp. 50-54, 2002. Download PDF

“Constrained Finite Continuous-Time Markov Decision
Processes with Average Rewards,” __Proceedings of IEEE 2002 Conference on
Decisions and Control__, Las Vegas,
December 10-13, pp. 3805-3810, 2002. Download PDF

“Average reward constrained continuous-time finite Markov
decision processes,” __Proceedings____
of the 2002 NSF Design and Manufacturing Research Conference__,

“Statistical Load Modeling,” (with D. Genethliou
and J.T. Hajagos), __Power and Energy__ __Systems__,
Proceedings of the Seventh IASTED International Multi-Conference, Palm Springs,
CA, February 24-26, pp. 88-91, 2003. Download PDF

“Control of Finite Continuous-Time Markov Chains with
Average Rewards and
Constraints,” __Kolmogorov and Contemporary Mathematics__,
Abstracts,

“Online Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), __Proceedings of the 42 ^{nd} IEEE Conference on Decision and Control__,

“Real Time Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), __Proceedings__ __of the 2004 NSF
Design and Manufacturing Research Conference__,

“Load Pocket Forecasting Software,” (with D. Genethliou, J.T. Hajagos, B.G. Irrgang, and R.R. Rossin),

__Proceedings of 2004 IEEE Power Systems __

“Optimality Inequalities for Average Cost MDPs and their
Inventory Control Applications,” (with M.E. Lewis), __Proceedings of the 43 ^{nd} IEEE Conference on Decision and Control__,
Bahamas, December 14-17, 2004, pp. 4411-4416, Download PDF

“Optimality of Randomized Trunk Reservation for a Problem
with a Single Constraint,” (with X. Fan-Orzechowski), __Proceedings
of the 2004 NSF Design and Manufacturing Research Conference__,

“Buffer Insertion for Bridges and Optimal Buffer Sizing for Communication Sub-System of Systems-on-Chip,” (with S.

Kallakuri, A. Doboli), __Proceedings of Design Automation and Test in
Europe__,

827.

“A Continuous Time Markov Decision Process Based On-Chip
Buffer Allocation Methodology,” (with S. Kallakuri,
N. Thepayasuwan, and A. Doboli), __Proceedings of GLSVLSI (Great Lakes
Symposium on VLSI)__,

“Optimal Admission Control for a Markovian
Queue under the Quality of Service Constraint,” (with Xiaofei
Fan-Orzechowski), __Proceedings of the 44 ^{nd}
IEEE Conference on Decision and Control, and the European Control Conference
2005__, Seville, Spain, December
12-15, 2005, pp. 1729-1734.

“On State Space Reduction in Sequential Decision
Processes,” __Proceedings of the 44 ^{nd} IEEE Conference on Decision and
Control, and the European Control Conference 2005__, Seville, Spain, December
12-15, 2005, pp. 7314-7319.

“Non-Randomized
Control of Constrained Markov Decision Processes,” (with R.C. Chen), __Proceedings
of the 2006 American Control Conference__,

“Optimality of
Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), __Proceedings of the
2006 NSF Design, Service, and Manufacturing Grantees and Research Conference__,
St. Louis, MO, July 24-27, 2006 (CD).

“Optimality of
Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), __Proceedings of the
2006 NSF Design, Service, and Manufacturing Grantees and Research Conference__,
St. Louis, MO, July 24-27, 2006 (CD).

“Quickest
Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), __Proceedings____
of the ____2008 NSF Engineering
Research and Innovation Conference__,

“On Polynomial Classification Problems for Markov Decision
Processes,” (with Fenghsu Yang), __Proceedings of the ____2008 NSF Engineering Research and Innovation Conference__,
Knoxville, TN, January 7-10, 2008 (CD).

“Polynomial
Classification Algorithms for Markov Decision Processes,” (with Fenghsu Yang), __Proceedings
of the 47th IEEE Conference on Decision
and Control__, Cancun, Mexico, December 9-11, 2008, pp.4485 – 4490.

“Adaptive
Computation of Optimal Nonrandomized Policies in Constrained Average-Reward
MDPs,” __2009 IEEE Symposium on
Adaptive Dynamic Programming and Reinforcement Learning (ADPLR 2009)
Proceedings__, March 30 – April 2, 2009, Nashville, TN, pp. 96 – 100

“Efficient Computation of Optimal
Nonrandomized Policies in Constrained Average-Reward Markov Decision
Processes,” __Proceedings of NSF CMMI Research and Innovation Conference 2009__,”

“Optimal
Admission for a Queue with Several Types of Customers and Holding Costs,” (with
F. Yang), In __Modern Trends in Controlled Stochastic Processes: Theory and
Applications__, A.B. Piunovskiy (ed.), pp. 304 –
322, Luniver Press, UK, 2010

“Optimal
Admission to an M/M/k/N Queue with Several Customer Types and Holding Costs,”
(with F. Yang), __Proceedings of the
49th IEEE Conference on Decision and
Control__, Atlanta, GA, December 15-17, 2010,
pp.2791 – 2796.

“Variance Minimization for Constrained Discounted Continuous-Time
MDPs with Exponentially Distributed Stopping Times,” (with J. Fei), __Proceedings of NSF CMMI Research and Innovation
Conference 2011__,” Atlanta, GA, January 4-7, 2011 (CD).

“Optimal Admission Policies for M/M/k/N Queue with Multiple
Customer Types and Holding Costs,” (with F. Yang), __Proceedings of NSF CMMI Research and
Innovation Conference 2011__,” Atlanta, GA, January 4-7, 2011 (CD).

“Smart
Grid Software Applications for Distribution Network Forecasting,” (with Jun Fei, J.T. Hajagos, and R.J. Rossin), __Proceedings of ENERGY 2011: The First
International Conference on Smart Grids, Green Communications and IT
Energy-aware Technologies__, May 21-27, Venice/Mestre,
Italy, 2011, pp. 76-80. PDF

“A
Rolling Horizon Approach to Distribution Feeder Reconfiguration with Switching
Costs,” (J. Hu, and K. Huang), __Proceedings
of the Second IEEE International Conference on Smart Grid Communications__,
Brussels, Belgium, 17-20 October 2011, pp. 357-362.

“Switching
on and off the full capacity of an M/M/infinity queue,” (with X. Zhang), __Proceedings of the 50th IEEE Conference on Decision and Control and
European Control Conference__,

“A
Parallel Rolling Horizon Scheme for Large Scale Security Constrained Unit Commitment
Problems with Wind Power Generation,” (with J. Hu and E. Yuan), __Proceedings
of ENERGY 2012: The Second International Conference on Smart Grids, Green
Communications and IT Energy-aware Technologies__, March 25-30, 2012, St.
Maarten, pp. 13-21. Downloiad
PDF

E.A. Feinberg, and U.G. Rothblum,
“Splitting in a finite Markov decision problem.” (with E.V Denardo and U.G. Rothblum), __SIGMETRICS Performance Evaluation
Review__, **39**, p. 38, 2012.

“The multi-armed bandit, with constraints.” (with E.V Denardo and U.G. Rothblum), __SIGMETRICS Performance Evaluation
Review__, **39**, p. 39, 2012.

“Sufficiency
of Markov policies for continuous-time Markov decision processes and solutions to
Kolmogorov’s forward equation for jump Markov processes,” (with M. Mandava
and A.N. Shiryaev), __ Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference__, Florence, Italy, December 10-13, pp.
5716-5721, 2013.

“Optimality
conditions for total-cost partially observable Markov decision processes,”
(with P.O. Kasyanov, M.Z. Zgurovsky), __Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference__,

”Voltage
and reactive power control using approximate stochastic annealing,” (with J. Hu, E. Yuan), __Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference__,

“Convergence
of Value Iterations for Total-Cost MDPs and POMDPs with General State and
Action Sets,” (with P.O. Kasyanov and M.Z. Zgurovsky),
Proceedings of 2014 IEEE__ Symposium on
Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), IEEE SSCI 2014__,
Orlando, FL, December 9-12, pp. 41-48, 2014.

“Sensor
Placement for Real-Time Power Flow Calculations in Transmission Networks,”
(with M. Li, R. Samulyak, B. Fardanesh,
G. Stefopoulos), __ENERGY 2015: The Fifth
International Conference on Smart Grids, Green Communications and IT
Energy-aware Technologies__, Rome, Italy, May 24-29, pp. 12-17, 2015.