**Eugene
A. Feinberg**

Department of Applied Mathematics and
Statistics

State University of New York at Stony Brook

Stony Brook, NY, 11794-3600

tel: (631) 632-7189

fax: (631) 632-8490

email: Eugene.Feinberg@sunysb.edu

Handbook of Markov Decision Processes: Methods and
Algorithms (with A. Shwartz, editors), Kluwer,

**Papers in Refereed Journals and Books:**

“On Controlled Finite State Markov Processes with Compact Control Sets,” __SIAM____
Theory__ __Probability Appl.__, **20**, pp. 856-862, 1975 PDF

“Control of Queueing Systems” (with M. A. Feinberg), __Radioelectronics__,
no. 3, pp. 3-34, 1975 (in Russian)

“On e-optimal Control of Discrete
Time Markov Chains,” __Transactions of Moscow__ __Institute of Transport
Engineering__, **510**, pp. 33-36, 1976 (in Russian)

“Railway Transport Radio Communication Models” (with L. M. Zuravliova), __Railway__
__Transport__, no. 6, pp. 81-83, 1976 (in Russian)

“On Finite Controlled Markov Chains,”__ Uspehi Math. Nauk__, **32**,
no 3, pp. 181-182, 1977 (in Russian)

“The Existence of a Stationary e-optimal
Policy for a Finite Markov Chain,” __SIAM Theory__ __Probability Appl.__,
**23**, pp. 297-313, 1978 PDF

“On Homogeneous Markov Decision Models with Continuous Time and Finite or
Countable State Spaces” (with A. A. Yushkevich), __SIAM____ Theory Probability
Appl.__, **24**, pp. 156-161, 1979 PDF PDF

“On Ergodic Conditions for Finite Controlled Markov Chains with Arbitrary
Decision Sets,” __Transactions of __,

“An e-optimal Control of a Finite
Markov Chain,” __SIAM Theory Probability Appl.__, **25**, pp. 70-81, 1980
PDF

“Optimization Models for Railway Radio Communication Procedures” (with L. M.
Zuravliova), __Railway Transport__, no. 4, pp. 65-67, 1980 (in Russian)

“An Optimal Strategy for Activation of a Service Device in One Class of
Systems with Group Service” (with L. B. Boguslavsky), __Problems of
Information Transmission__, no. 3,pp. 243-253, 1980.

“Optimization of Number of Vans in a Group in Railway Operations” (with S.
M. Rezer), __Transactions of Moscow Institute of Transport Engineering__,**
653**, pp. 55-63, 1981 (in Russian)

“Nonrandomized Markov and Semi-Markov Strategies in Dynamic Programming,” __SIAM____
Theory Probability Appl.__, **27**, pp. 116-126, 1982 PDF

“Controlled Markov Processes with Arbitrary Numerical Criteria,” __SIAM
Theory Probability__ __Appl.__, **27**, pp. 486-503, 1982. PDF

“Stationary Strategies in Dynamic Programming with a Countable State Space”
(with __Studies on Mathemat.
Econom. and Control Theory__, Central Econom.-Math. Institute Press,

“Stationary and Markov Policies in Countable State Dynamic Programming”
(with __Lecture Notes in Math.__,**
1021**, pp. 111-129, 1982 PDF

“Sufficient Classes of Strategies in Controllable Markov Chains with Total
Criterion” (with __Soviet Math.
Dokl.__,** 29**, no. 2, pp. 308-311, 1984

“Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming”
(with I. M. Sonin), __Statistics and Control of Stochastic Processes__,
Steklov Seminar, Optimization Software, NY, pp. 69-101, 1985

“Optimal Specification of Railway Platform Sections,” __Transactions of
Moscow Institute__ __of Transport Engineering__, **758**, pp. 58-62,
1984 (in Russian)

“Optimal Control of Parameters of Multiple Access Protocols” (with Ya. A.
Kogan and A. N. Smirnov), __Automatic Control and Computer Science__, no. 6,
pp. 48-54, 1984.

“Optimal Control by the Retransmission Probability in Slotted ALOHA Systems”
(with Ya. A. Kogan and A. N. Smirnov), __Performance Evaluation__,** 5**,
pp. 85-96, 1985

“Sufficient Classes of Strategies in Discrete Dynamic Programming. I:
Decomposition of Randomized Strategies and Embedded Models,” __SIAM____ Theory Probability
Appl.__, **31**, pp. 658-668, 1986 PDF

“The Structure of Persistently Nearly Optimal Strategies in Stochastic
Dynamic Programming,” __Lecture Notes in Control and Inform. Sci.__, **81**,
pp. 22-31, 1986

“Sufficient Classes of Strategies in Discrete Dynamic Programming. II:
Locally Stationary Strategies,” __SIAM____
Theory Probability Appl.__, **32**, No. 3, pp. 478-493, 1987 PDF

“Parametric Stochastic Dynamic Programming,” __Statistics and Control of
Stochastic__ __Processes__, Steklov Seminar,** 2**, Optimization
Software, NY, pp. 103-120, 1989

“Nonrandomized Strategies in Stochastic Decision Processes,” __Annals of
Operations__ __Research__, **29**, pp. 315-332, 1991

“A Markov Decision Model of a Search Process,” __Contemporary Mathematics__,**
125**, pp. 87-96, 1992

“On Stationary Strategies in Borel Dynamic Programming,” __Mathematics of
Operations__ __Research__, **17**, pp. 393-397, 1992

“__Stochastic Processes Appl.__, **49**,
pp. 159-177, 1994

“Markov Decision Models with Weighted Discounted Criteria” (with A.
Shwartz), __Mathematics of Operations Research__, **19**, pp. 152-168,
1994

“A Generalization of “Expectation Equals Reciprocal of Intensity” to
Nonstationary Distributions,”__ J. Appl. Probability__, **31**, pp.
262-267, 1994. . Download PS or PDF

“Constrained Semi-Markov Decision Processes with Average Rewards,” __ZOR -__
__Mathematical Methods of Operations Research__,** 39**, pp. 257-288,
1994

“Optimality of Randomized Trunk Reservation” (with M. Reiman), __Probability
in__ __Engineering and Informational Sciences__, **8**, pp. 483-489,
1994

“Constrained Markov Decision Models with Weighted Discounting” (with A.
Shwartz), __Mathematics of Operations Research__, **20**, pp. 302-320,
1995

“Bicriterion Optimization of an M/G/1 Queue with a Removable Server” (with
D.J. Kim), __Probability in Engineering and Informational Sciences__, **10**,
pp. 57-73, 1996. Download PDF

“Letter to the Editor,” __Operations Research__, **44**, p. 526, 1996

“Constrained Discounted Dynamic Programming” (with A. Shwartz), __Mathematics
of__ __Operations Research__, **21**, pp. 922-945, 1996. . Download PS or DVI

“Notes on Equivalent Stationary Policies in Markov Decision Processes with
Total Rewards” (with I.M. Sonin), __ZOR - Mathematical Methods of Operations
Research__, **44**, pp. 205-221, 1996.

“On Measurability and Representation of Strategic Measures in Markov
Decision Processes”, in __Statistics, Probability and Game Theory Papers in
Honor of David__ __Blackwell__ (eds. T.S. Ferguson et al.), IMS Lecture Notes - Monograph Series, **30**, pp.
29-43, 1996 PDF

“Stochastic Monotonicity for Stationary Recurrence Times of __Annals of Applied Probability__, **7**, pp.
326-339, 1997. Download PDF

“On Future Organization of Hybrid Chip Manufacturing”
(with S. Luryi), in __Future Trends in Microelectronics__ , J. Wiley &
Sons (eds.

“Constrained Dynamic Programming with Two Discount
Factors: Applications and an Algorithm”; (with A. Shwartz), __IEEE
Transactions on Automatic Control__, **44**,
pp. 628-631, 1999. Download PS or DVI

“ Weighted Discounted Stochastic Games with Perfect Information” (with __Annals of the International
Society of Dynamic Games__, **5**, pp. 303-324, 2000. Download PDF

“Constrained Discounted Markov Decision Processes and
Hamiltonian Cycles” __Mathematics of
Operations Research__, **25**, pp. 130-140, 2000. Download PDF

“A Note on the Existence of Optimal Policies in Continuous
Dynamic Programs with Compact Action Sets” (with R. Cavazos-Cadena and R.
Montes-de-Oca), __Mathematics of Operations Research__, **25**, pp.
657-666, 2000. Download PS or DVI

“Multiple Objective Nonatomic Markov Decision Processes
with Total Reward Criteria” (with A.B. Piunovskiy) ), __Journal of Mathematical Analysis and
Applications__, **247**, pp. 45-66,
2000. Download PS or DVI

“Perturbed Zero-Sum Games with Applications to Stochastic
and Repeated Games” (with __Annals of the International Society of Dynamic Games__,
**6**, pp. 165-181, 2001. Download PS or DVI

“Introduction” (with A. Shwartz), Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp. 1-17, 2002.

“Mixed Criteria” (with A. Shwartz) , __Handbook of Markov
Decision Processes: Methods and Applications,__ Kluwer, pp 209-230,
2002. Download PS or DVI

“Total Reward Criteria,” Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 173-207, 2002.

“Constrained Discounted Semi-Markov Decision Processes,”
Markov Processes and Controlled Markov Chains (Z. How, J.A. Filar, A. Chen,
eds), Kluwer,

“Nonatomic Total Reward Markov Decision Processes with
Multiple Criteria” (with A.B. Piunovskiy),
__Journal of Mathematical Analysis and Applications__, **273**,
93-111, 2002. Download PDF

“Optimality of D-policies for an M/G/1 Queue with a
Removable Server” (with O. Kella), __Queueing
Systems – Theory and Applications (QUESTA)__ **42**, pp. 355-376,
2002. Download PDF

“Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event
Approach,” __Mathematics of Operations Research__, **29**, pp. 492-524,
2004. Download PDF

“Optimality of Four-Threshold Policies in Inventory Systems with Customer
Returns and Borrowing/Storage Options,” (with M.E. Lewis), __Probability in Engineering and
Informational Sciences__, **19**, 45-71, 2005. Download PDF

“Generalized Pinwheel Problem,” (with M.T. Curry), __Mathematical Methods of Operations
Research__, **62, **99-122, 2005.
Download PDF

“Load Forecasting,” (with D. Genethliou),
__Applied Mathematics for Restructured Electric Power Systems:
Optimization, Control, and Computational Intelligence__ (J. H. Chow, F.F. Wu,
and J.J. Momoh, eds.), Spinger, pp. 269-285, 2005. Download PDF

“On Essential Information in Stochastic Decision Processes,” __Mathematical
Methods of Operations Research__, **62, **399-410, 2005. Download PDF

“On
Dvoretzky-Wald-Wolfowitz Theorem on Nonrandomized Statistical Decisions,” (with
A.B. Piunovskiy), __SIAM ____Theory__ __Probability Appl.__, **50**,
463-466, 2006. Download PDF

“Optimality
of Randomized Trunk Reservation for a Problem with a Single Constraint,”
(with X. Fan-Orzechowski),
__Advances in Applied Probability__, **38**,
199-220, 2006. Download PDF

“Quickest Detection of Drift Change for Brownian Motion in
Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Statistics
& Decisions, **24**, 445-470, 2006. Download PDF

“Optimality of Randomized Trunk Reservation for a Problem with
Multiple Constraints,” (with X. Fan-Orzechowski), __Probability
in Engineering and Informational Sciences__, **21**, 189-200, 2007. Download PDF

“Optimality Inequalities for Average Cost Markov Decision Processes and the
Stochastic Cash Balance Problem,” (with M.E. Lewis), __Mathematics of
Operations Research__, **32**,
769-783, 2007. Download PDF

“Non-Randomized Policies for Constrained Markov Decision Processes,” (with
R.C. Chen), __Mathematical Methods of Operations Research__, **66**, 165-179, 2007. Download PDF

“Optimality of Deterministic Policies for
Certain Stochastic Control Problems with Multiple Criteria and Constraints,” in
__Mathematical Control Theory and Finance__, A. Sarychev et al (eds),
Springer, Berlin, pp. 137-148, 2008.
Download PDF

“On Polynomial Classification Problems for Markov Decision Processes,” (with
Fenghsu Yang), __Operations Research Letters__, **36**, pp. 527-530, 2008.
Download PDF

“On
Asymptotic Optimality of the Second Order in the Minimax Quickest Detection
Problem of Drift Change for Brownian Motion,” (with E.V. Burnaev and A.N.
Shiryaev), __SIAM Theory Probability
Appl__., **53**, No 3, pp. 519-536, 2009.
PDF

“An Inequality for Variances of the Discounted
Rewards,” __Journal of Applied Probability__, (with J. Fei), ** 46**,
1209-1212, 2009. Download PDF

“Compactness
of the Space of Non-Randomized Policies in Countable-State Sequential Decision
Processes,” (with R.C. Chen), __Mathematical
Methods of Operations Research.,__ **71**,
307-323, 2010. Download PDF

“On
Strongly Equivalent Nonrandomized Transition Probabilities,” (with A.B.
Piunovskiy), __SIAM Theory Probab. Appl__.,
**54, **300-307, 2010. Download
PDF

“Total
Expected Discounted Reward MDPs: Existence of Optimal Policies,” __Wiley Encyclopedia of Operations Research
and Management Science__, Wiley Online Library, 2011,
DOI: 10.1002/9780470400531.eorms0906.
Download PDF

“On __Proceedings of the American Mathematical
Society__, **139**, pp. 4497-4511,
2011. Download PDF

“Optimality of Trunk Reservation for an M/M/k/N Queue with Several Customer
types and Holding Costs,” (with F. Yang), __Probability in Engineering and
Informational Sciences__, **25**, pp.
537-560, 2011. Download PDF

“Splitting Randomized Stationary Policies in Total-Reward Markov Decision
Processes,” (with U.G. Rothblum), __Mathematics
of Operations Research__, 37, pp. 129-153, 2012. Download PDF

“Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward
Rates to Discrete-Time Total-Reward MDPs,” in __Optimization, Control, and
Applications of Stochastic Systems__, D. Hernandez and A. Minjarez (eds),
Birkhauser, Springer, pp. 77-98,
2012. Download PDF

“Average-Cost Markov Decision
Processes with Weakly Continuous Transition Probabilities,” (with P.O. Kasyanov
and N.V. Zadoianchuk), __Mathematics of
Operations Research__ **37**,
pp.591-607, 2012. Download PDF

“Berge's
Theorem for Noncompact Image Sets,” (with P.O. Kasyanov and N.V.
Zadoianchuk), __Journal of Mathematical
Analysis and Applications__, **397**, pp. 255-259, 2013. Download PDF

“Variance Minimization for Constrained Discounted Continuous-Time MDPs with
Exponentially Distributed Stopping Times,” (with J. Fei), __Annals of
Operations Research, __ **208**, pp.
433-450, 2013. Download PDF

“The Multi-Armed Bandit, with Constraints,” (with E.V. Denardo and U.G.
Rothblum), __Annals of Operations
Research__ , **208**, pp. 37-62,
2013. Download PDF

“Strong
polynomiality of policy iterations for average-cost MDPs modeling replacement
and maintenance problems,” (with J. Huang), __Operations Research Letters__,
41, pp. 249-251, 2013. Download PDF

“Fatou’s
Lemma for Weakly Converging Probabilities,” (with P.O. Kasyanov, N.V.
Zadoianchuk) __Theory Probab. Appl__., **58**, pp. 683-689, 2014. PDF

“Extension
of Lyapunov's Convexity Theorem to Subranges,” (with P. Dai), __Proceedings of the American Mathematical
Society__, **142**, pp. 361-467, 2014.
Download PDF

“On
Solutions of Kolmogorov’s Equations for Jump Markov Processes,” (with M.
Mandava, A.N. Shiryaev), __Journal of
Mathematical Analysis and Applications__, **411**, pp. 261-270, 2014.
Download PDF

“The
Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic
Programming,” (with J. Huang), __Operations Research Letters__, **42**,
pp. 130-131, 2014. Download PDF

“Berge’s
Maximum Theorem for Noncompact Image Sets,“ (with P.O. Kasyanov and M.
Voorneveld), __Journal of Mathematical
Analysis and Applications__, **413**, pp. 1040-1046, 2014. Download PDF

“Examples
Concerning Abel and Cesàro Limits,” (with C.J. Bishop and J. Zhang), __Journal of Mathematical Analysis and
Applications__, **420**, pp.
1654-1661, 2014. PDF

“Optimality
Conditions for Partially Observable Markov Decision Processes,” (with P.O.
Kasyanov and M.Z. Zgurovsky) in __Continuous and Distributed Systems: Theory
and Applications__, M.Z. Zgurovsky, V.A. Sadovnichiy (editors), Springer,
Cham, Switzeland, pp. 251-264,
2014.

“Modified
Policy Iteration Algorithms are Not Strongly Polynomial for Discounted Dynamic
Programming,” (with J. Huang, B. Scherrer), __Operations Research Letters__,
**42**, pp. 429-431, 2014. PDF

“Convergence
of Probability Measures and Markov Decision Models with Incomplete
Information,” (with P.O. Kasyanov and M.Z. Zgurovsky), __Proceedings of the
Steklov Institute of Mathematics,__ **287**, pp. 96-117, 2014. PDF

“Optimal
Switching on and off the Entire Service Capacity of a Parallel Queue,” (with X.
Zhang), __Probability in Engineering
and Informational Sciences__, **29**, pp. 483-506, 2015. PDF

“Continuity
of Minima: Local Results,” (with P.O. Kasyanov), __Set-Valued and Variational
Analysis__, **23**, pp. 485-499, 2015.
PDF

“Optimal
Pricing for a GI/M/k/N Queue with Several Customer Types and Holding Costs,”
(with F. Yang), __Queueing Systems__, **82**,
pp. 103-120, 2016. PDF

“A
Stochastic Search Algorithm for Voltage and Reactive Power Control with
Switching Costs and ZIP Load Model,” (with J. Hu and E. Yuan), __Electric
Power Systems Research__, **133**, pp.
328-337, 2016. PDF

“Partially Observable
Total-Cost Markov Decision Processes with Weakly Continuous Transition
Probabilities,” (with P.O. Kasyanov and M.Z. Zgurovsky), __Mathematics of
Operations Research__, 41, pp. 656-681, 2016.
PD F

“Uniform Fatou’s Lemma,” (with
Kasyanov and M.Z. Zgurovsky), __Journal
of Mathematical Analysis and Applications__,
**444**, pp. 550-567, 2016. PD F

“Structure of
Optimal Solutions to Periodic-Review Total-Cost Stochastic Inventory Control
Problems,” (with Y. Liang), __ACM
SIGMETRICS Performance Evaluation Review__, **44**, pp. 21-23, 2016. PD F

“Optimality
Conditions for Inventory Control,” __Tutorials in Operations Research__,
INFORMS, pp. 14-44, 2016. PD F

“On the Reduction of Total-Cost and
Average-Cost MDPs to Discounted MDPs,” (with J. Huang), __Naval Research
Logistics__, 2017, published online: DOI 10.1002/nav.21743. PD F

“On the Convergence of Optimal Actions for
Markov Decision Processes and the Optimality of (s,S) Policies for Inventory
Control,” (with M.E. Lewis), __Naval
Research Logistics__, 2017, published online: DOI 10.1002/nav.21750. PD F

“Kolmogorov’s Equations for Jump
Markov Processes with Unbounded Jump Rates,” (with M. Mandava and A.N.
Shiryaev), __Annals of Operations Research__, 2017, published online: DOI
10.1007/s10479-017-2538-8. PD F

“On the Optimality Equation
for Average Cost Markov Decision Processes and its
Validity for Inventory Control,” (with Y. Liang), __Annals of Operations Research__, 2017,
published online: DOI
10.1007/s10479-017-2561-9.
PD F

“Structure of Optimal Solutions
to Periodic-Review Total-Cost Inventory Control Problems,” (with Y. Liang), __Annals
of Operations Research__, 2017, published online: DOI 10.1007/s10479-017-2548-6. PD F

“Strongly
Polynomial Algorithms for Transient and Average-Cost MDPs,” (with J.
Huang), __ACM SIGMETRICS Performance Evaluation Review__, **45**(2), pp. 6-8, 2017. PD F

“Continuity of Equilibria
for Two-Person Zero-Sum Games with Noncompact Action Sets and Unbounded
Payoffs,” (with P.O. Kasyanov and M.Z. Zgurovsky), __Annals of Operations Research__, 2017,
published online: DOI
10.1007/s10479-017-2677-y. PD F

“Reduction of Total-Cost and
Average-Cost MDPs with Weakly Continuous Transition Probabilities to Discounted
MDPs,” with J. Huang), __Operations
Research Letters,__** 46**, p. 179-184,
2018. PD F

“An Example Showing that A-Lower
Semi-Continuity is Essential for Minimax Continuity Theorems,” (with P.O.
Kasyanov and M.Z. Zgurovsky), __Operations
Research Letters__, **46**, p.
385-388, 2018. PD F

“Two-Person Zero-Sum Games
with Unbounded Payoff Functions and Uncertain Expected Payoffs,” (with P.O.
Kasyanov and M.Z. Zgurovsky), http://arxiv.org/abs/1704.04564 .

“Stochastic
Setup-Cost Inventory Model with Backorders and Quasiconvex Cost Functions, (with
J. Liang), to appear in __Probability in the Engineering and Informational Sciences__, __http://arxiv.org/abs/1705.06814__ .

“Sufficiency of Deterministic Policies for
Atomless Discounted and Uniformly Absorbing MDPs with Multiple Criteria,” (with
A.B. Piunovskiy), to appear in __SIAM__
__Journal on Control and Optimization__, __https://arxiv.org/abs/1806.06170__ .

“Fatou’s Lemma for Weakly
Converging Measures under the Uniform Integrability Condition,” (P.O. Kasyanov and Y. Liang), __https://arxiv.org/abs/1807.07931__ .

**Papers in Conference Proceedings:**

“Quality Evaluation and Optimization of Data Transmission Procedures in
Computer Networks” (with L. B. Boguslavsky, E. Gelembe, Ya. A. Kogan), __Proceedings
of 7-th__ __National Conference on Coding Theory and Data Transmission__, **3**, pp.
25-30, 1978 (in Russian)

“Markov and Semi-Markov Strategies in Dynamic Programming,” __Proceedings
of 14-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, pp. 30-31, 1980 (in Russian)

“Nonrandomized Strategies in Markov Decision Processes,” __Proceedings of
the__ __Conference on Mathematical Methods of Optimization in Large Economic
and Technical__ __Systems__,”

“Discrete Time Stochastic Decision Processes with Arbitrary Real-Valued
Criteria,” __Abstracts of 3-rd __,

“On Markov Decision Models with Measurable Value Functions,” __Proceedings
of 15-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, pp.12--13, 1981 (in Russian)

“Markov Policies in Infinite Horizon Dynamic Programming Problems with
Bounded Value Functions” (with I. M. Sonin), __Abstracts of 4-th USSR - Japan
Symposium on__

“Persistently Nearly Optimal Strategies in Countable State Markov Decision
Chains with the Total Reward Criterion” (with I. M. Sonin), __Abstracts of
International Conference on Stochastic Optimization__,

“Uniformly Optimal Strategies in Controlled Random Sequences,” __Proceedings
of 18-th__ __Conference on Probability Theory and Mathematical Statistics__,
Bakuriani, p. 44, 1984 (in Russian)

“Optimal Control of Parameters in Random Multivariate Access” (with Ya. A.
Kogan and A. N. Smirnov), __Proceedings of 9-th Russian Conference on Computer
Networks__, Puschino, **2.1**, pp. 41-44, 1984 (in Russian)

“On the Existence of Good Strategies in Stochastic Dynamic Programming,” __Abstracts
of__ __4-th __, 4, pp. 70-72,
1985

“Controlled Markov Models with Decision Set Selection,” __Proceedings of
19-th__ __Conference on Probability Theor and Mathematical Statistics__,
Bakuriani, p. 58, 1985 (in Russian)

“Optimization of a railway traffic” (with S. M. Rezer and L. N. Schennikov),
__Proceedings__ __of the Conference “Optimization and Control of
Transportation Systems”__, Gomel, pp. 169-171, 1985 (in Russian).

“Additive Functionals of Jump Processes: a Discrete Time Representation,” __Abstracts
of__ __I-st World Congress of Bernoulli Society__, **1**, p. 289, 1986

“Parametric Dynamic Programming,” __Proceedings of 20-th Conference on
Probability__ __Theory and Mathematical Statistics__, Bakuriani, p. 56,
1986 (in Russian)

“Optimality of Pure Strategies in Stochastic Decision Processes,” __Proceedings
of the__ __29-th IEEE Conference on Decision and Control__, **4**, pp.
2149-2154, 1990

“Weighted Discounted Dynamic Programming” (with A. Shwartz), __Proceedings
of the__ __30-th IEEE Conference on Decision and Control__,** 2**, pp.
485-486, 1991.

“Optimal Switching Policies for M/G/1 Queues with Two Performance Criteria”
(with D.J. Kim), __Proceedings of the International Conference on Operations
Research__, pp. 227-232, 1994,

“Multiple Objective Optimal Control of Production Systems with Stochastic
Demand” (with D.J. Kim),__ Proceedings of the First Regional Symposium on
Manufacturing Science and Technology__, Stony Brook, NY, pp. 37-43, 1995.

“Optimization of Jump Stochastic Systems,”__ Proceedings of the 1996 NSF
Design and Manufacturing Grantees Conference__, Albuquerque, NM, pp.497-498,
1996.

“Weighted Discounted Stochastic Games with Perfect Information,” (with E.
Altman and A. Shwartz), __Proceedings of the Seventh International Symposium
on Dynamic Games__ __and Applications__, December 16-18, Kanagawa, Japan,**
1**, 18 - 31, 1996.

“Optimization of Discounted Constrained Jump Stochastic Systems,” __Proceedings
of the 1997 NSF Design and Manufacturing Grantees Conference__, Seattle, WA,
pp. 457-458, 1997.

“Geographic Scheduling System for Customer Service,” (with A. Zhukovsky), __Proceedings
of the 18 ^{th} National Conference of the American Society of
Engineering__

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles,” __Proceedings
of the 36-th IEEE Conference on Decision and Control__, **3**, pp. 2821 -
2826, 1997.

“Optimization of Jump Stochastic Systems: Constrained Dynamic Programming
and Hamiltonian Cycles,”__ Proceedings of the 1997 NSF Design and
Manufacturing Grantees Conference__, NSF,

“Perturbed Zero-Sum Games with Applications to Dynamic
Games” (with E. Altman, J. Filar, and V. Gaitsgory), __Proceedings of the Seventh International
Symposium on Dynamic Games and Applications__, July 5-8, Maastricht, The Netherlands, pp. 45 - 51,
1998.

“Optimization of Jump Stochastic Systems,” __Proceedings
of the 1999 NSF Design and Manufacturing Grantees Conference__,

“New Approach to Optimization of Discounted Stochastic
Continuous-Time Discrete-Event Systems,” __Proceedings of the 38‑th IEEE
Conference on Decision and Control__ **1**, pp. 937 – 941, 1999.

“Optimality of Pure Strategies in Continuous Dynamic
Programming Problems with Multiple Criteria, “ (with A.B. Piunovskiy) __Proceedings of the 2000 NSF Design and
Manufacturing Grantees Conference__,

“Nonatomic Total Reward Markov Decision Processes with
Multiple Criteria,” (with A.B.
Piunovskiy), __Proceedings of the 39‑th IEEE Conference on Decision and
Control__, 1, pp. 723 – 728, 2000.

“Nonatomic Total Reward Markov Decision Processes,” (with
A. B. Piunovskiy), __Proceedings of the 2001 NSF Design and Manufacturing
Research Conference__,

“Optimization of Jump Stochastic Systems,” __Proceedings
of the 2002 NSF Design and Manufacturing Research Conference__, San Juan,
Puerto Rico, January 7-10, 2002 (CD).

“Sensor Resource Management for an Airborne Early Warning
Radar,” (with M. Bender, M. Curry, D. Huang, T. Koutsoudis, J. Bernstein), __Signal
and Data Processing of Small Targets__,
Proceedings of SPIE – The International Society of Optical Engineering, 2-4
April 2002, Orlando, **4728**, pp. 145-156, 2002.

“Online Scheduling with Hard Constraints: a Semi-Markov
Decision Process Approach,” (with M. Curry), __Control and Applications__,
Proceedings of the IASTED International Conference, Cancun, MX, May 20-22, pp.
104-109, 2002.

“Load Pocket Forecasting,” (with D. Genethliou and J.T.
Hajagos), __Power and Energy Systems__, Proceedings of the Second IASTED
International Conference, Crete, Greece, June 25-28, pp. 50-54, 2002. Download PDF

“Constrained Finite Continuous-Time Markov Decision
Processes with Average Rewards,” __Proceedings of IEEE 2002 Conference on
Decisions and Control__, Las Vegas,
December 10-13, pp. 3805-3810, 2002. Download PDF

“Average reward constrained continuous-time finite Markov
decision processes,” __Proceedings of
the 2002 NSF Design and Manufacturing Research Conference__,

“Statistical Load Modeling,” (with D. Genethliou and J.T.
Hajagos), __Power and Energy__ __Systems__, Proceedings of the Seventh
IASTED International Multi-Conference, Palm Springs, CA, February 24-26, pp.
88-91, 2003. Download PDF

“Control of Finite Continuous-Time Markov Chains with
Average Rewards and Constraints,” __Kolmogorov
and Contemporary Mathematics__, Abstracts,

“Online Scheduling: Generalized Pinwheel Problem,”(with
M.T. Curry), __Proceedings of the 42 ^{nd} IEEE Conference on Decision and Control__,

“Real Time Scheduling: Generalized Pinwheel Problem,”(with
M.T. Curry), __Proceedings__ __of the 2004 NSF Design and Manufacturing
Research Conference__,

“Load Pocket Forecasting Software,” (with D. Genethliou, J.T. Hajagos, B.G. Irrgang, and R.R. Rossin),

__Proceedings of 2004 IEEE Power Systems __

“Optimality Inequalities for Average Cost MDPs and their
Inventory Control Applications,” (with M.E. Lewis), __Proceedings of the 43 ^{nd} IEEE Conference on Decision and Control__,
Bahamas, December 14-17, 2004, pp. 4411-4416, Download PDF

“Optimality of Randomized Trunk Reservation for a Problem
with a Single Constraint,” (with X. Fan-Orzechowski), __Proceedings of the 2004 NSF Design and
Manufacturing Research Conference__,

“Buffer Insertion for Bridges and Optimal Buffer Sizing for Communication Sub-System of Systems-on-Chip,” (with S.

Kallakuri, A. Doboli), __Proceedings of Design Automation
and Test in Europe__,

827.

“A Continuous Time Markov Decision Process Based On-Chip
Buffer Allocation Methodology,” (with S. Kallakuri, N. Thepayasuwan, and A.
Doboli), __Proceedings of GLSVLSI
(Great Lakes Symposium on VLSI)__,

“Optimal Admission Control for a Markovian Queue under the
Quality of Service Constraint,” (with Xiaofei Fan-Orzechowski), __Proceedings
of the 44 ^{nd} IEEE Conference on Decision and Control, and the
European Control Conference 2005__, Seville,
Spain, December 12-15, 2005, pp. 1729-1734.

“On State Space Reduction in Sequential Decision
Processes,” __Proceedings of the 44 ^{nd} IEEE Conference on Decision
and Control, and the European Control Conference 2005__, Seville, Spain,
December 12-15, 2005, pp. 7314-7319.

“Non-Randomized
Control of Constrained Markov Decision Processes,” (with R.C. Chen), __Proceedings
of the 2006 American Control Conference__,

“Optimality of
Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), __Proceedings of the 2006 NSF Design,
Service, and Manufacturing Grantees and Research Conference__, St. Louis, MO,
July 24-27, 2006 (CD).

“Optimality of
Randomized Trunk Reservation for Multiple-Constraint Problems,” (with X. Fan-Orzechowski), __Proceedings of the 2006 NSF Design,
Service, and Manufacturing Grantees and Research Conference__, St. Louis, MO,
July 24-27, 2006 (CD).

“Quickest
Detection of Drift Change for Brownian Motion in Generalized Bayesian and
Minimax Settings,” (with A.N. Shiryaev), __Proceedings of
the ____2008 NSF Engineering
Research and Innovation Conference__,

“On Polynomial
Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), __Proceedings of the ____2008 NSF Engineering Research and Innovation
Conference__, Knoxville, TN, January 7-10, 2008 (CD).

“Polynomial
Classification Algorithms for Markov Decision Processes,” (with Fenghsu
Yang), __Proceedings of the 47th IEEE Conference on Decision and Control__,
Cancun, Mexico, December 9-11, 2008,
pp.4485 – 4490.

“Adaptive
Computation of Optimal Nonrandomized Policies in Constrained Average-Reward
MDPs,” __2009 IEEE Symposium on
Adaptive Dynamic Programming and Reinforcement Learning (ADPLR 2009)
Proceedings__, March 30 – April 2, 2009, Nashville, TN, pp. 96 – 100

“Efficient Computation of Optimal
Nonrandomized Policies in Constrained Average-Reward Markov Decision
Processes,” __Proceedings of NSF CMMI Research and Innovation Conference 2009__,”

“Optimal
Admission for a Queue with Several Types of Customers and Holding Costs,” (with
F. Yang), In __Modern Trends in Controlled Stochastic Processes: Theory and
Applications__, A.B. Piunovskiy (ed.), pp. 304 – 322, Luniver Press, UK, 2010

“Optimal
Admission to an M/M/k/N Queue with Several Customer Types and Holding Costs,”
(with F. Yang), __Proceedings of the
49th IEEE Conference on Decision and
Control__, Atlanta, GA, December 15-17, 2010,
pp.2791 – 2796.

“Variance
Minimization for Constrained Discounted Continuous-Time MDPs with Exponentially
Distributed Stopping Times,” (with J. Fei), __Proceedings of NSF CMMI Research
and Innovation Conference 2011__,” Atlanta, GA, January 4-7, 2011 (CD).

“Optimal
Admission Policies for M/M/k/N Queue with Multiple Customer Types and Holding
Costs,” (with F. Yang), __Proceedings
of NSF CMMI Research and Innovation Conference 2011__,” Atlanta, GA, January
4-7, 2011 (CD).

“Smart
Grid Software Applications for Distribution Network Forecasting,” (with Jun
Fei, J.T. Hajagos, and R.J. Rossin), __Proceedings of ENERGY 2011: The First
International Conference on Smart Grids, Green Communications and IT
Energy-aware Technologies__, May 21-27, Venice/Mestre, Italy, 2011, pp.
76-80. PDF

“A
Rolling Horizon Approach to Distribution Feeder Reconfiguration with Switching
Costs,” (J. Hu, and K. Huang), __Proceedings
of the Second IEEE International Conference on Smart Grid Communications__,
Brussels, Belgium, 17-20 October 2011, pp. 357-362.

“Switching
on and off the full capacity of an M/M/infinity queue,” (with X. Zhang), __Proceedings of the 50th IEEE Conference on Decision and Control and
European Control Conference__,

“A
Parallel Rolling Horizon Scheme for Large Scale Security Constrained Unit Commitment
Problems with Wind Power Generation,” (with J. Hu and E. Yuan), __Proceedings
of ENERGY 2012: The Second International Conference on Smart Grids, Green
Communications and IT Energy-aware Technologies__, March 25-30, 2012, St.
Maarten, pp. 13-21. PDF

E.A.
Feinberg, and U.G. Rothblum, “Splitting in a finite Markov decision problem.”
(with E.V
Denardo and U.G. Rothblum), __SIGMETRICS
Performance Evaluation Review__, **39**, p. 38, 2012.

“The
multi-armed bandit, with constraints.” (with E.V Denardo and U.G.
Rothblum), __SIGMETRICS
Performance Evaluation Review__, **39**, p. 39, 2012.

“Sufficiency
of Markov policies for continuous-time Markov decision processes and solutions
to Kolmogorov’s forward equation for jump Markov processes,” (with M. Mandava and A.N. Shiryaev), __ Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference__, Florence, Italy, December 10-13, pp.
5716-5721, 2013.

“Optimality
conditions for total-cost partially observable Markov decision processes,”
(with P.O. Kasyanov, M.Z. Zgurovsky), __Proceedings
of the 52th IEEE Conference on Decision
and Control and European Control Conference__,

”Voltage
and reactive power control using approximate stochastic annealing,” (with J. Hu, E. Yuan), __Proceedings of the 52th IEEE Conference on Decision and Control and
European Control Conference__,

“Convergence
of Value Iterations for Total-Cost MDPs and POMDPs with General State and
Action Sets,” (with P.O. Kasyanov and M.Z. Zgurovsky), Proceedings of 2014 IEEE__ Symposium on Adaptive Dynamic
Programming and Reinforcement Learning (ADPRL), IEEE SSCI 2014__, Orlando,
FL, December 9-12, pp. 41-48, 2014.

“Sensor
Placement for Real-Time Power Flow Calculations in Transmission Networks,”
(with M. Li, R. Samulyak, B. Fardanesh, G. Stefopoulos), __ENERGY 2015: The
Fifth International Conference on Smart Grids, Green Communications and IT
Energy-aware Technologies__, Rome, Italy, May 24-29, pp. 12-17, 2015.

“On the Average-Cost
Optimality Equations and Convergence of Discounted-Cost Relative Value
Functions for Inventory Control Problems with Quasiconvex Cost Functions,” (with
Y. Liang), __Proceedings of the 56th IEEE Conference on Decision and Control__,
Malbourne, Australia, December 12-15, pp. 641-646, 2017.

“A Flexible Mixed Additive-Multiplicative
Model for Load Forecasting in a Smart
Grid Setting,” (with J. Fei), to appear in __Proceedings of the Conference Forecasting
and Risk Management for Renewal Energy__, Paris, June 7-9, 2017, Springer. PDF