Eugene A. Feinberg

Department of Applied Mathematics and Statistics
State University of New York at Stony Brook
Stony Brook, NY, 11794-3600
tel: (631) 632-7189
fax: (631) 632-8490
email: Eugene.Feinberg@sunysb.edu

 

Books

Handbook of Markov Decision Processes: Methods and Algorithms (with A. Shwartz, editors), Kluwer, Boston, 2002

Papers in Refereed Journals and Books:

“On Controlled Finite State Markov Processes with Compact Control Sets,” SIAM Theory Probability Appl., 20, pp. 856-862, 1975 PDF

“Control of Queueing Systems” (with M. A. Feinberg), Radioelectronics, no. 3, pp. 3-34, 1975 (in Russian)

“On e-optimal Control of Discrete Time Markov Chains,” Transactions of Moscow Institute of Transport Engineering, 510, pp. 33-36, 1976 (in Russian)

“Railway Transport Radio Communication Models” (with L. M. Zuravliova), Railway Transport, no. 6, pp. 81-83, 1976 (in Russian)

“On Finite Controlled Markov Chains,” Uspehi Math. Nauk, 32, no 3, pp. 181-182, 1977 (in Russian)

“The Existence of a Stationary e-optimal Policy for a Finite Markov Chain,” SIAM Theory Probability Appl., 23, pp. 297-313, 1978 PDF

“On Homogeneous Markov Decision Models with Continuous Time and Finite or Countable State Spaces” (with A. A. Yushkevich), SIAM Theory Probability Appl., 24, pp. 156-161, 1979 PDF PDF

“On Ergodic Conditions for Finite Controlled Markov Chains with Arbitrary Decision Sets,” Transactions of Moscow Institute of Transport Engineering, 640, pp. 51-59, 1979 (in Russian)

“An e-optimal Control of a Finite Markov Chain,” SIAM Theory Probability Appl., 25, pp. 70-81, 1980 PDF

“Optimization Models for Railway Radio Communication Procedures” (with L. M. Zuravliova), Railway Transport, no. 4, pp. 65-67, 1980 (in Russian)

“An Optimal Strategy for Activation of a Service Device in One Class of Systems with Group Service” (with L. B. Boguslavsky), Problems of Information Transmission, no. 3,pp. 243-253, 1980.

“Optimization of Number of Vans in a Group in Railway Operations” (with S. M. Rezer), Transactions of Moscow Institute of Transport Engineering, 653, pp. 55-63, 1981 (in Russian)

“Nonrandomized Markov and Semi-Markov Strategies in Dynamic Programming,” SIAM Theory Probability Appl., 27, pp. 116-126, 1982 PDF

“Controlled Markov Processes with Arbitrary Numerical Criteria,” SIAM Theory Probability Appl., 27, pp. 486-503, 1982. PDF

“Stationary Strategies in Dynamic Programming with a Countable State Space” (with I. M. Sonin), Studies on Mathemat. Econom. and Control Theory, Central Econom.-Math. Institute Press, Moscow, pp. 137-173, 1983 (in Russian)

“Stationary and Markov Policies in Countable State Dynamic Programming” (with I. M. Sonin), Lecture Notes in Math., 1021, pp. 111-129, 1982 PDF

“Sufficient Classes of Strategies in Controllable Markov Chains with Total Criterion” (with I. M. Sonin), Soviet Math. Dokl., 29, no. 2, pp. 308-311, 1984

“Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming” (with I. M. Sonin), Statistics and Control of Stochastic Processes, Steklov Seminar, Optimization Software, NY, pp. 69-101, 1985

“Optimal Specification of Railway Platform Sections,” Transactions of Moscow Institute of Transport Engineering, 758, pp. 58-62, 1984 (in Russian)

“Optimal Control of Parameters of Multiple Access Protocols” (with Ya. A. Kogan and A. N. Smirnov), Automatic Control and Computer Science, no. 6, pp. 48-54, 1984.

“Optimal Control by the Retransmission Probability in Slotted ALOHA Systems” (with Ya. A. Kogan and A. N. Smirnov), Performance Evaluation, 5, pp. 85-96, 1985

“Sufficient Classes of Strategies in Discrete Dynamic Programming. I: Decomposition of Randomized Strategies and Embedded Models,” SIAM Theory Probability Appl., 31, pp. 658-668, 1986 PDF

“The Structure of Persistently Nearly Optimal Strategies in Stochastic Dynamic Programming,” Lecture Notes in Control and Inform. Sci., 81, pp. 22-31, 1986

“Sufficient Classes of Strategies in Discrete Dynamic Programming. II: Locally Stationary Strategies,” SIAM Theory Probability Appl., 32, No. 3, pp. 478-493, 1987 PDF

“Parametric Stochastic Dynamic Programming,” Statistics and Control of Stochastic Processes, Steklov Seminar, 2, Optimization Software, NY, pp. 103-120, 1989

“Nonrandomized Strategies in Stochastic Decision Processes,” Annals of Operations Research, 29, pp. 315-332, 1991

“A Markov Decision Model of a Search Process,” Contemporary Mathematics, 125, pp. 87-96, 1992

“On Stationary Strategies in Borel Dynamic Programming,” Mathematics of Operations Research, 17, pp. 393-397, 1992

Finite State Markov Decision Models with Average Reward Criteria” (with H. Park), Stochastic Processes Appl., 49, pp. 159-177, 1994

“Markov Decision Models with Weighted Discounted Criteria” (with A. Shwartz), Mathematics of Operations Research, 19, pp. 152-168, 1994

“A Generalization of “Expectation Equals Reciprocal of Intensity” to Nonstationary Distributions,” J. Appl. Probability, 31, pp. 262-267, 1994. .   Download PS or PDF 

“Constrained Semi-Markov Decision Processes with Average Rewards,” ZOR - Mathematical Methods of Operations Research, 39, pp. 257-288, 1994

“Optimality of Randomized Trunk Reservation” (with M. Reiman), Probability in Engineering and Informational Sciences, 8, pp. 483-489, 1994

“Constrained Markov Decision Models with Weighted Discounting” (with A. Shwartz), Mathematics of Operations Research, 20, pp. 302-320, 1995

Bicriterion Optimization of an M/G/1 Queue with a Removable Server” (with D.J. Kim), Probability in Engineering and Informational Sciences, 10, pp. 57-73, 1996.  Download PDF

“Letter to the Editor,” Operations Research, 44, p. 526, 1996

“Constrained Discounted Dynamic Programming” (with A. Shwartz), Mathematics of Operations Research, 21, pp. 922-945, 1996. . Download PS or DVI 

“Notes on Equivalent Stationary Policies in Markov Decision Processes with Total Rewards” (with I.M. Sonin), ZOR - Mathematical Methods of Operations Research, 44, pp. 205-221, 1996.

“On Measurability and Representation of Strategic Measures in Markov Decision Processes”, in Statistics, Probability and Game Theory Papers in Honor of David Blackwell (eds. T.S. Ferguson et al.), IMS Lecture Notes - Monograph Series, 30, pp. 29-43, 1996 PDF

“Stochastic Monotonicity for Stationary Recurrence Times of First Passage Heights” (with E.V. Denardo and O. Kella), Annals of Applied Probability, 7, pp. 326-339, 1997.  Download PDF

“On Future Organization of Hybrid Chip Manufacturing” (with S. Luryi), in Future Trends in Microelectronics , J. Wiley & Sons (eds. S. Luryi and A. Zaslavsky), pp. 47-54, 1999.

 

“Constrained Dynamic Programming with Two Discount Factors: Applications and an Algorithm”; (with A. Shwartz), IEEE Transactions on Automatic Control, 44, pp. 628-631, 1999.   Download PS or DVI 

“ Weighted Discounted Stochastic Games with Perfect Information” (with E. Altman and A. Shwartz), Annals of the International Society of Dynamic Games, 5, pp. 303-324, 2000.   Download PDF

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles  Mathematics of Operations Research, 25, pp. 130-140, 2000.  Download PDF

 

“A Note on the Existence of Optimal Policies in Continuous Dynamic Programs with Compact Action Sets” (with R. Cavazos-Cadena and R. Montes-de-Oca), Mathematics of Operations Research, 25, pp. 657-666, 2000.  Download PS or DVI

 

“Multiple Objective Nonatomic Markov Decision Processes with Total Reward Criteria” (with A.B. Piunovskiy) ),  Journal of Mathematical Analysis and Applications, 247, pp. 45-66, 2000.  Download PS or DVI 

 

“Perturbed Zero-Sum Games with Applications to Stochastic and Repeated Games” (with E. Altman, J. Filar, and V. Gaitsgory), Annals of the International Society of Dynamic Games, 6, pp. 165-181, 2001.   Download PS or DVI 

 

“Introduction” (with A. Shwartz),  Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp. 1-17, 2002.

 

“Mixed Criteria” (with A. Shwartz) , Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 209-230, 2002.  Download PS or DVI

 

“Total Reward Criteria,” Handbook of Markov Decision Processes: Methods and Applications, Kluwer, pp 173-207, 2002.

 

“Constrained Discounted Semi-Markov Decision Processes,” Markov Processes and Controlled Markov Chains (Z. How, J.A. Filar, A. Chen, eds), Kluwer, Norwell, MA, pp. 231-242, 2002. Download PS

 

Nonatomic Total Reward Markov Decision Processes with Multiple Criteria” (with A.B. Piunovskiy),  Journal of Mathematical Analysis and Applications, 273, 93-111, 2002. Download  PDF

 

“Optimality of D-policies for an M/G/1 Queue with a Removable Server” (with O. Kella),  Queueing Systems – Theory and Applications (QUESTA) 42, pp. 355-376, 2002.   Download PDF

“Continuous Time Discounted Jump Markov Decision Processes: A Discrete-Event Approach,” Mathematics of Operations Research, 29, pp. 492-524, 2004.  Download PDF 

“Optimality of Four-Threshold Policies in Inventory Systems with Customer Returns and Borrowing/Storage Options,” (with M.E. Lewis),  Probability in Engineering and Informational Sciences, 19, 45-71, 2005.  Download PDF

“Generalized Pinwheel Problem,” (with M.T. Curry),  Mathematical Methods of Operations Research, 62, 99-122, 2005.  Download PDF

“Load Forecasting,” (with D. Genethliou),   Applied Mathematics for Restructured Electric Power Systems: Optimization, Control, and Computational Intelligence (J. H. Chow, F.F. Wu, and J.J. Momoh, eds.), Spinger, pp. 269-285, 2005.   Download PDF

“On Essential Information in Stochastic Decision Processes,” Mathematical Methods of Operations Research, 62, 399-410, 2005.  Download PDF

“On Dvoretzky-Wald-Wolfowitz Theorem on Nonrandomized Statistical Decisions,” (with A.B. Piunovskiy), SIAM Theory Probability Appl., 50, 463-466, 2006.  Download PDF

Optimality of Randomized Trunk Reservation for a Problem with a Single Constraint,” (with X. Fan-Orzechowski),  Advances in Applied Probability, 38, 199-220, 2006.  Download PDF

 

“Quickest Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev), Statistics & Decisions, 24, 445-470, 2006. Download PDF

 Optimality of Randomized Trunk Reservation for a Problem with Multiple Constraints,” (with X. Fan-Orzechowski),  Probability in Engineering and Informational Sciences, 21, 189-200, 2007.   Download PDF

“Optimality Inequalities for Average Cost Markov Decision Processes and the Stochastic Cash Balance Problem,” (with M.E. Lewis), Mathematics of Operations Research, 32, 769-783, 2007.  Download PDF

“Non-Randomized Policies for Constrained Markov Decision Processes,” (with R.C. Chen), Mathematical Methods of Operations Research, 66, 165-179, 2007.  Download PDF

 “Optimality of Deterministic Policies for Certain Stochastic Control Problems with Multiple Criteria and Constraints,” in Mathematical Control Theory and Finance, A. Sarychev et al (eds), Springer, Berlin, pp. 137-148, 2008.  Download PDF

“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), Operations Research Letters, 36, pp. 527-530, 2008.  Download  PDF

“On Asymptotic Optimality of the Second Order in the Minimax Quickest Detection Problem of Drift Change for Brownian Motion,” (with E.V. Burnaev and A.N. Shiryaev),  SIAM Theory Probability Appl., 53, No 3, pp. 519-536, 2009. PDF

 

 “An Inequality for Variances of the Discounted Rewards,” Journal of Applied Probability, (with J. Fei),  46, 1209-1212, 2009.  Download PDF

 

“Compactness of the Space of Non-Randomized Policies in Countable-State Sequential Decision Processes,” (with R.C. Chen),   Mathematical Methods of Operations Research., 71, 307-323, 2010.  Download PDF

 

“On Strongly Equivalent Nonrandomized Transition Probabilities,” (with A.B. Piunovskiy),  SIAM Theory Probab. Appl., 54, 300-307, 2010.  Download  PDF

 

“Total Expected Discounted Reward MDPs: Existence of Optimal Policies,  Wiley Encyclopedia of Operations Research and Management Science, Wiley Online Library, 2011, DOI: 10.1002/9780470400531.eorms0906.  Download PDF

 

“On Maximal Ranges of Vector Measures for Subsets and Purification of Transition Probabilities,” (with P. Dai),  Proceedings of the American Mathematical Society, 139, pp. 4497-4511, 2011.  Download  PDF

“Optimality of Trunk Reservation for an M/M/k/N Queue with Several Customer types and Holding Costs,” (with F. Yang), Probability in Engineering and Informational Sciences, 25, pp. 537-560, 2011. Download  PDF

“Splitting Randomized Stationary Policies in Total-Reward Markov Decision Processes,” (with U.G. Rothblum),  Mathematics of Operations Research, 37, pp. 129-153, 2012.  Download PDF

“Reduction of Discounted Continuous-Time MDPs with Unbounded Jump and Reward Rates to Discrete-Time Total-Reward MDPs,” in Optimization, Control, and Applications of Stochastic Systems, D. Hernandez and A. Minjarez (eds), Birkhauser, Springer, pp. 77-98,  2012.  Download PDF

 “Average-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities,” (with P.O. Kasyanov and N.V. Zadoianchuk),  Mathematics of Operations Research  37, pp.591-607, 2012.   Download PDF

“Berge's Theorem for Noncompact Image Sets,” (with P.O. Kasyanov and N.V. Zadoianchuk),  Journal of Mathematical Analysis and Applications, 397, pp. 255-259, 2013.    Download PDF

“Variance Minimization for Constrained Discounted Continuous-Time MDPs with Exponentially Distributed Stopping Times,” (with J. Fei), Annals of Operations Research,  208, pp. 433-450, 2013.    Download  PDF

“The Multi-Armed Bandit, with Constraints,” (with E.V. Denardo and U.G. Rothblum),  Annals of Operations Research ,  208, pp. 37-62, 2013.   Download PDF

“Strong polynomiality of policy iterations for average-cost MDPs modeling replacement and maintenance problems,” (with J. Huang), Operations Research Letters, 41, pp. 249-251, 2013. Download PDF

 

Fatou’s Lemma for Weakly Converging Probabilities,” (with P.O. Kasyanov, N.V. Zadoianchuk) Theory Probab. Appl., 58, pp. 683-689, 2014.  PDF

 

“Extension of Lyapunov's Convexity Theorem to Subranges,” (with P. Dai),  Proceedings of the American Mathematical Society, 142, pp. 361-467, 2014.  Download PDF

 

“On Solutions of Kolmogorov’s Equations for Jump Markov Processes,” (with M. Mandava, A.N. Shiryaev),  Journal of Mathematical Analysis and Applications, 411, pp. 261-270, 2014. Download PDF

 

“The Value Iteration Algorithm is Not Strongly Polynomial for Discounted Dynamic Programming,” (with J. Huang), Operations Research Letters, 42, pp. 130-131, 2014. Download PDF

 

“Berge’s Maximum Theorem for Noncompact Image Sets,“ (with P.O. Kasyanov and M. Voorneveld),  Journal of Mathematical Analysis and Applications, 413, pp. 1040-1046, 2014. Download  PDF

 

“Examples Concerning Abel and Cesàro Limits,” (with C.J. Bishop and J. Zhang),  Journal of Mathematical Analysis and Applications, 420, pp. 1654-1661, 2014.  PDF

 

“Optimality Conditions for Partially Observable Markov Decision Processes,” (with P.O. Kasyanov and M.Z. Zgurovsky) in Continuous and Distributed Systems: Theory and Applications, M.Z. Zgurovsky, V.A. Sadovnichiy (editors),  Springer,  Cham, Switzeland, pp. 251-264,  2014.

 

“Modified Policy Iteration Algorithms are Not Strongly Polynomial for Discounted Dynamic Programming,” (with J. Huang, B. Scherrer), Operations Research Letters, 42, pp. 429-431, 2014.  PDF

 

“Convergence of Probability Measures and Markov Decision Models with Incomplete Information,” (with P.O. Kasyanov and M.Z. Zgurovsky), Proceedings of the Steklov Institute of Mathematics, 287, pp. 96-117, 2014.  PDF

 

“Optimal Switching on and off the Entire Service Capacity of a Parallel Queue,” (with X. Zhang),  Probability in Engineering and Informational Sciences,  29, pp. 483-506, 2015. PDF

 

“Continuity of Minima: Local Results,” (with P.O. Kasyanov), Set-Valued and Variational Analysis, 23, pp. 485-499, 2015. PDF

 

“Optimal Pricing for a GI/M/k/N Queue with Several Customer Types and Holding Costs,” (with F. Yang), Queueing Systems, 82, pp. 103-120, 2016.  PDF

 

“A Stochastic Search Algorithm for Voltage and Reactive Power Control with Switching Costs and ZIP Load Model,” (with J. Hu and E. Yuan), Electric Power Systems Research, 133, pp. 328-337, 2016.  PDF

 

“Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition Probabilities,” (with P.O. Kasyanov and M.Z. Zgurovsky), Mathematics of Operations Research, 41, pp. 656-681, 2016.  PD F

 “Uniform Fatou’s Lemma,” (with Kasyanov and M.Z. Zgurovsky),    Journal of Mathematical Analysis and Applications,  444, pp. 550-567, 2016.  PD F

“Structure of Optimal Solutions to Periodic-Review Total-Cost Stochastic Inventory Control Problems,  (with Y. Liang), ACM SIGMETRICS Performance Evaluation Review, 44, pp. 21-23, 2016.  PD F

“Optimality Conditions for Inventory Control,” Tutorials in Operations Research, INFORMS, pp. 14-44, 2016.  PD F

 “On the Reduction of Total-Cost and Average-Cost MDPs to Discounted MDPs,” (with J. Huang), Naval Research Logistics, 2017, published online: DOI 10.1002/nav.21743.  PD F

 “On the Convergence of Optimal Actions for Markov Decision Processes and the Optimality of (s,S) Policies for Inventory Control,”  (with M.E. Lewis), Naval Research Logistics, 2017, published online: DOI 10.1002/nav.21750. PD F

 “Kolmogorov’s Equations for Jump Markov Processes with Unbounded Jump Rates,” (with M. Mandava and A.N. Shiryaev), Annals of Operations Research, 2017, published online: DOI 10.1007/s10479-017-2538-8.  PD F

“On the Optimality Equation for Average Cost Markov Decision Processes and its Validity for Inventory Control,” (with Y. Liang),   Annals of Operations Research, 2017, published online: DOI 10.1007/s10479-017-2561-9. PD F

 

“Structure of Optimal Solutions to Periodic-Review Total-Cost Inventory Control Problems,” (with Y. Liang), Annals of Operations Research, 2017, published online: DOI 10.1007/s10479-017-2548-6.      PD F

Papers in Conference Proceedings:

“Quality Evaluation and Optimization of Data Transmission Procedures in Computer Networks” (with L. B. Boguslavsky, E. Gelembe, Ya. A. Kogan), Proceedings of 7-th National Conference on Coding Theory and Data Transmission, Vilnius, 3, pp. 25-30, 1978 (in Russian)

“Markov and Semi-Markov Strategies in Dynamic Programming,” Proceedings of 14-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp. 30-31, 1980 (in Russian)

“Nonrandomized Strategies in Markov Decision Processes,” Proceedings of the Conference on Mathematical Methods of Optimization in Large Economic and Technical Systems,” Baku, pp. 188-190, 1980 (in Russian)

“Discrete Time Stochastic Decision Processes with Arbitrary Real-Valued Criteria,” Abstracts of 3-rd Vilnius Conference on Probability Theory and Mathematical Statistics, 3, pp. 88-89, 1981

“On Markov Decision Models with Measurable Value Functions,” Proceedings of 15-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, pp.12--13, 1981 (in Russian)

“Markov Policies in Infinite Horizon Dynamic Programming Problems with Bounded Value Functions” (with I. M. Sonin), Abstracts of 4-th USSR - Japan Symposium on Probability Theory and Mathematical Statistics, Tbilisi, 1, pp. 209-210, 1982

“Persistently Nearly Optimal Strategies in Countable State Markov Decision Chains with the Total Reward Criterion” (with I. M. Sonin), Abstracts of International Conference on Stochastic Optimization, Kiev, pp. 74-76, 1984

“Uniformly Optimal Strategies in Controlled Random Sequences,” Proceedings of 18-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 44, 1984 (in Russian)

“Optimal Control of Parameters in Random Multivariate Access” (with Ya. A. Kogan and A. N. Smirnov), Proceedings of 9-th Russian Conference on Computer Networks, Puschino, 2.1, pp. 41-44, 1984 (in Russian)

“On the Existence of Good Strategies in Stochastic Dynamic Programming,” Abstracts of 4-th Vilnius Conference on Probability Theory and Mathematical Statistics, 4, pp. 70-72, 1985

“Controlled Markov Models with Decision Set Selection,” Proceedings of 19-th Conference on Probability Theor and Mathematical Statistics, Bakuriani, p. 58, 1985 (in Russian)

“Optimization of a railway traffic” (with S. M. Rezer and L. N. Schennikov), Proceedings of the Conference “Optimization and Control of Transportation Systems”, Gomel, pp. 169-171, 1985 (in Russian).

“Additive Functionals of Jump Processes: a Discrete Time Representation,” Abstracts of I-st World Congress of Bernoulli Society, Tashkent, 1, p. 289, 1986

“Parametric Dynamic Programming,” Proceedings of 20-th Conference on Probability Theory and Mathematical Statistics, Bakuriani, p. 56, 1986 (in Russian)

“Optimality of Pure Strategies in Stochastic Decision Processes,” Proceedings of the 29-th IEEE Conference on Decision and Control, 4, pp. 2149-2154, 1990

“Weighted Discounted Dynamic Programming” (with A. Shwartz), Proceedings of the 30-th IEEE Conference on Decision and Control, 2, pp. 485-486, 1991.

“Optimal Switching Policies for M/G/1 Queues with Two Performance Criteria” (with D.J. Kim), Proceedings of the International Conference on Operations Research, pp. 227-232, 1994, Berlin, Springer-Verlag.

“Multiple Objective Optimal Control of Production Systems with Stochastic Demand” (with D.J. Kim), Proceedings of the First Regional Symposium on Manufacturing Science and Technology, Stony Brook, NY, pp. 37-43, 1995.

“Optimization of Jump Stochastic Systems,” Proceedings of the 1996 NSF Design and Manufacturing Grantees Conference, Albuquerque, NM, pp.497-498, 1996.

“Weighted Discounted Stochastic Games with Perfect Information,” (with E. Altman and A. Shwartz), Proceedings of the Seventh International Symposium on Dynamic Games and Applications, December 16-18, Kanagawa, Japan, 1, 18 - 31, 1996.

“Optimization of Discounted Constrained Jump Stochastic Systems,” Proceedings of the 1997 NSF Design and Manufacturing Grantees Conference, Seattle, WA, pp. 457-458, 1997. 

“Geographic Scheduling System for Customer Service,” (with A. Zhukovsky), Proceedings of the 18th National Conference of the American Society of Engineering Management, Virginia Beach, pp. 561 - 563, 1997.

“Constrained Discounted Markov Decision Processes and Hamiltonian Cycles,” Proceedings of the 36-th IEEE Conference on Decision and Control, 3, pp. 2821 - 2826, 1997. 

“Optimization of Jump Stochastic Systems: Constrained Dynamic Programming and Hamiltonian Cycles,” Proceedings of the 1997 NSF Design and Manufacturing Grantees Conference, NSF, Arlington, VA, pp. 217-218, 1998. 

“Perturbed Zero-Sum Games with Applications to Dynamic Games” (with E. Altman, J. Filar, and V. Gaitsgory),  Proceedings of the Seventh International Symposium on Dynamic Games and Applications,  July 5-8, Maastricht, The Netherlands,  pp. 45 - 51,  1998. 

 

“Optimization of Jump Stochastic Systems,” Proceedings of the 1999 NSF Design and Manufacturing Grantees Conference, Long Beach,  CA, January 5-8, 1999 (CD).

 

“New Approach to Optimization of Discounted Stochastic Continuous-Time Discrete-Event Systems,” Proceedings of the 38‑th IEEE Conference on Decision and Control 1, pp. 937 – 941, 1999. 

 

“Optimality of Pure Strategies in Continuous Dynamic Programming Problems with Multiple Criteria, “ (with A.B. Piunovskiy)  Proceedings of the 2000 NSF Design and Manufacturing Grantees Conference, Vancouver,  Canada, January 2000 (CD).

 

Nonatomic Total Reward Markov Decision Processes with Multiple Criteria,  (with A.B. Piunovskiy), Proceedings of the 39‑th IEEE Conference on Decision and Control, 1, pp. 723 – 728, 2000.

 

Nonatomic Total Reward Markov Decision Processes,” (with A. B. Piunovskiy), Proceedings of the 2001 NSF Design and Manufacturing Research Conference, Tampa, Florida, January 6-9, 2001 (CD).

 

“Optimization of Jump Stochastic Systems,” Proceedings of the 2002 NSF Design and Manufacturing Research Conference, San Juan, Puerto Rico, January 7-10, 2002 (CD).

 

“Sensor Resource Management for an Airborne Early Warning Radar,” (with M. Bender, M. Curry, D. Huang, T. Koutsoudis, J. Bernstein), Signal and Data  Processing of Small Targets, Proceedings of SPIE – The International Society of Optical Engineering, 2-4 April 2002, Orlando,  4728, pp. 145-156, 2002.

 

“Online Scheduling with Hard Constraints: a Semi-Markov Decision Process Approach,” (with M. Curry), Control and Applications, Proceedings of the IASTED International Conference, Cancun, MX, May 20-22, pp. 104-109, 2002.

 

“Load Pocket Forecasting,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Second IASTED International Conference, Crete, Greece, June 25-28, pp. 50-54, 2002.  Download PDF

 

“Constrained Finite Continuous-Time Markov Decision Processes with Average Rewards,” Proceedings of IEEE 2002 Conference on Decisions and Control, Las Vegas,  December 10-13, pp. 3805-3810, 2002. Download PDF

 

“Average reward constrained continuous-time finite Markov decision processes,  Proceedings of the 2002 NSF Design and Manufacturing Research Conference, Birminham, AL, January 6-9, 2003 (CD).

 

“Statistical Load Modeling,” (with D. Genethliou and J.T. Hajagos), Power and Energy Systems, Proceedings of the Seventh IASTED International Multi-Conference, Palm Springs, CA, February 24-26, pp. 88-91, 2003. Download PDF

 

“Control of Finite Continuous-Time Markov Chains with Average Rewards and  Constraints,” Kolmogorov and Contemporary Mathematics, Abstracts, Moscow State University, June 16-21, 2003, pp. 427-428.

 

“Online Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of   the 42nd  IEEE Conference on Decision and Control, Maui, December 9-12, 2003, pp. 4333-4338.

 

“Real Time Scheduling: Generalized Pinwheel Problem,”(with M.T. Curry), Proceedings of the 2004 NSF Design and Manufacturing Research Conference, Dallas, TX, January 4-7, 2004 (CD).

 

“Load Pocket Forecasting Software,” (with D. Genethliou, J.T. Hajagos, B.G. Irrgang, and  R.R. Rossin),

Proceedings of 2004 IEEE Power Systems Conference & Exposition,  New York, October 10-13, 2004,  Download PDF

 

“Optimality Inequalities for Average Cost MDPs and their Inventory Control Applications,” (with M.E. Lewis),  Proceedings of   the 43nd  IEEE Conference on Decision and Control, Bahamas, December 14-17, 2004, pp. 4411-4416, Download PDF

 

“Optimality of Randomized Trunk Reservation for a Problem with a Single Constraint,” (with X. Fan-Orzechowski), Proceedings of the 2004 NSF Design and Manufacturing Research Conference, Scottsdale, AZ, January 3-6, 2005 (CD).

 

“Buffer Insertion for Bridges and Optimal Buffer Sizing for Communication Sub-System of Systems-on-Chip,” (with S.

Kallakuri, A. Doboli), Proceedings of Design Automation and Test in Europe, Munich, Germany, March 6-10, 2005, pp 826-

827.

 

“A Continuous Time Markov Decision Process Based On-Chip Buffer Allocation Methodology,” (with S. Kallakuri, N. Thepayasuwan, and A. Doboli), Proceedings of GLSVLSI (Great Lakes Symposium on VLSI), Chicago, IL,  April 17-19, 2005, pp. 345-348.

 

“Optimal Admission Control for a Markovian Queue under the Quality of Service Constraint,” (with Xiaofei Fan-Orzechowski), Proceedings of the 44nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville,  Spain, December 12-15, 2005, pp. 1729-1734.

 

“On State Space Reduction in Sequential Decision Processes,” Proceedings of the 44nd IEEE Conference on Decision and Control, and the European Control Conference 2005, Seville, Spain, December 12-15, 2005, pp. 7314-7319.

 

“Non-Randomized Control of Constrained Markov Decision Processes,” (with R.C. Chen), Proceedings of the 2006 American Control Conference, Minneapolis, MN, June 14-16, 2006.

 

“Optimality of Randomized Trunk Reservation for Multiple-Constraint Problems,”  (with X. Fan-Orzechowski), Proceedings of the 2006 NSF Design, Service, and Manufacturing Grantees and Research Conference, St. Louis, MO, July 24-27, 2006 (CD).

 

“Optimality of Randomized Trunk Reservation for Multiple-Constraint Problems,”  (with X. Fan-Orzechowski), Proceedings of the 2006 NSF Design, Service, and Manufacturing Grantees and Research Conference, St. Louis, MO, July 24-27, 2006 (CD).

 

“Quickest Detection of Drift Change for Brownian Motion in Generalized Bayesian and Minimax Settings,” (with A.N. Shiryaev),  Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, TN, January 7-10, 2008 (CD).

 

“On Polynomial Classification Problems for Markov Decision Processes,” (with Fenghsu Yang), Proceedings of the 2008 NSF Engineering Research and Innovation Conference, Knoxville, TN, January 7-10, 2008 (CD).

 

“Polynomial Classification Algorithms for Markov Decision Processes,” (with Fenghsu Yang),  Proceedings of the 47th  IEEE Conference on Decision and Control, Cancun, Mexico, December 9-11, 2008,  pp.4485 – 4490.

 

“Adaptive Computation of Optimal Nonrandomized Policies in Constrained Average-Reward MDPs,”  2009 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPLR 2009) Proceedings, March 30 – April 2, 2009, Nashville, TN, pp. 96 – 100

 

 “Efficient Computation of Optimal Nonrandomized Policies in Constrained Average-Reward Markov Decision Processes,” Proceedings of NSF CMMI Research and Innovation Conference 2009,” Honolulu, Hawaii, June 22-25, 2009 (CD)

 

“Optimal Admission for a Queue with Several Types of Customers and Holding Costs,” (with F. Yang), In Modern Trends in Controlled Stochastic Processes: Theory and Applications, A.B. Piunovskiy (ed.), pp. 304 – 322, Luniver Press, UK, 2010

 

“Optimal Admission to an M/M/k/N Queue with Several Customer Types and Holding Costs,” (with F. Yang),  Proceedings of the 49th  IEEE Conference on Decision and Control, Atlanta, GA, December 15-17, 2010,  pp.2791 – 2796.

 

“Variance Minimization for Constrained Discounted Continuous-Time MDPs with Exponentially Distributed Stopping Times,” (with J. Fei), Proceedings of NSF CMMI Research and Innovation Conference 2011,” Atlanta, GA, January 4-7, 2011 (CD).

 

“Optimal Admission Policies for M/M/k/N Queue with Multiple Customer Types and Holding Costs,” (with F. Yang),   Proceedings of NSF CMMI Research and Innovation Conference 2011,” Atlanta, GA, January 4-7, 2011 (CD).

 

“Smart Grid Software Applications for Distribution Network Forecasting,” (with Jun Fei, J.T. Hajagos, and R.J. Rossin), Proceedings of ENERGY 2011: The First International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, May 21-27, Venice/Mestre, Italy, 2011, pp. 76-80.  PDF

 

“A Rolling Horizon Approach to Distribution Feeder Reconfiguration with Switching Costs,” (J. Hu, and K. Huang),  Proceedings of the Second IEEE International Conference on Smart Grid Communications, Brussels, Belgium, 17-20 October 2011, pp. 357-362.

 

“Switching on and off the full capacity of an M/M/infinity queue,” (with X. Zhang),  Proceedings of the 50th  IEEE Conference on Decision and Control and European Control Conference, Orlando, Fl, December 12-15, 2011, pp. 7678-7683.

 

“A Parallel Rolling Horizon Scheme for Large Scale Security Constrained Unit Commitment Problems with Wind Power Generation,” (with J. Hu and E. Yuan), Proceedings of ENERGY 2012: The Second International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, March 25-30, 2012, St. Maarten, pp. 13-21.  Downloiad PDF

 

 

E.A. Feinberg, and U.G. Rothblum, “Splitting in a finite Markov decision problem.” (with E.V Denardo and U.G. Rothblum),  SIGMETRICS Performance Evaluation Review, 39, p. 38, 2012.

 

“The multi-armed bandit, with constraints.” (with E.V Denardo and U.G. Rothblum),   SIGMETRICS Performance Evaluation Review, 39, p. 39, 2012.

 

“Sufficiency of Markov policies for continuous-time Markov decision processes and solutions to Kolmogorov’s forward equation for jump Markov processes,”  (with M. Mandava and A.N. Shiryaev),  Proceedings of the 52th  IEEE Conference on Decision and Control and European Control Conference, Florence, Italy, December 10-13, pp. 5716-5721, 2013.

 

“Optimality conditions for total-cost partially observable Markov decision processes,” (with P.O. Kasyanov, M.Z. Zgurovsky),   Proceedings of the 52th  IEEE Conference on Decision and Control and European Control Conference, Florence, Italy, December 10-13, pp. 5728-5732, 2013.

 

”Voltage and reactive power control using approximate stochastic annealing,”  (with J. Hu, E. Yuan),   Proceedings of the 52th  IEEE Conference on Decision and Control and European Control Conference, Florence, Italy, December 10-13, pp. 6954-6959, 2013.

 

“Convergence of Value Iterations for Total-Cost MDPs and POMDPs with General State and Action Sets,” (with P.O. Kasyanov and M.Z. Zgurovsky), Proceedings  of 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), IEEE SSCI 2014, Orlando, FL, December 9-12, pp. 41-48, 2014.

 

“Sensor Placement for Real-Time Power Flow Calculations in Transmission Networks,” (with M. Li, R. Samulyak, B. Fardanesh, G. Stefopoulos), ENERGY 2015: The Fifth International Conference on Smart Grids, Green Communications and IT Energy-aware Technologies, Rome, Italy, May 24-29, pp. 12-17, 2015.