{"id":1261,"date":"2022-04-05T19:42:18","date_gmt":"2022-04-05T19:42:18","guid":{"rendered":"https:\/\/meyn.ece.ufl.edu\/?page_id=1261"},"modified":"2026-04-14T09:50:09","modified_gmt":"2026-04-14T14:50:09","slug":"stochastic-control","status":"publish","type":"page","link":"https:\/\/faculty.eng.ufl.edu\/meyn\/courses\/stochastic-control\/","title":{"rendered":"Stochastic Control"},"content":{"rendered":"

The first goal is to learn how to formulate models for the purposes of control, in applications ranging from finance to power systems to medicine. Linear and Markov models are chosen to capture essential dynamics and uncertainty.<\/p>\n

The course provides several approaches to design control laws based on these models, and methods to approximate the performance of the controlled system. In parallel with these algorithmic objectives, students will be provided with an introduction to basic dynamic programming theory, closely related stability theory for Markovian and linear systems.<\/p>\n

What about model free?\u00a0 \u00a0The course also provides an introduction to reinforcement learning (based on chapters 8 and 9 of the new monograph<\/a>).\u00a0 Understanding RL is not difficult after you have a good understanding of optimal control fundamentals.<\/p>\n\n\n\n
Course information from Spring 2020<\/a>Topics include:<\/p>\n
    \n
  1. Control and Stability Theory<\/li>\n
  2. Optimal Control<\/li>\n
  3. Monte-Carlo Methods<\/li>\n
  4. ODE Methods for Algorithm Design<\/li>\n
  5. TD and Q-learning\u00a0 (and Actor-Critic methods if time permits)<\/li>\n<\/ol>\n<\/td>\n
\"Book\"<\/a>CS&RL<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Essential background<\/strong>: stochastic processes and prior exposure to control concepts.\u00a0 Experience with Matlab or Python is also essential.<\/p>\n

More Resources<\/strong><\/h4>\n