{"id":1265,"date":"2022-04-05T19:42:37","date_gmt":"2022-04-05T19:42:37","guid":{"rendered":"https:\/\/meyn.ece.ufl.edu\/?page_id=1265"},"modified":"2026-04-14T09:51:19","modified_gmt":"2026-04-14T14:51:19","slug":"stochastic-processes","status":"publish","type":"page","link":"https:\/\/faculty.eng.ufl.edu\/meyn\/courses\/stochastic-processes\/","title":{"rendered":"Stochastic Methods for Engineers II"},"content":{"rendered":"

An introduction to stochastic process theory with emphasis on applications to communications, control, signal processing and machine learning.\u00a0\u00a0 The course covers basic models, including Markov processes, and how they lead to algorithms for classification prediction, inference and model selection.<\/p>\n

The course mainly follow’s Hajek’s excellent textbook.\u00a0 Markov chains and Monte-Carlo methods are based on chapters 7 and 8 of the new monograph<\/a><\/p>\n\n\n\n
\n
Outline<\/h5>\n
    \n
  1. Review of stochastic processes<\/li>\n
  2. Convergence theory & asymptotic statistics<\/li>\n
  3. Random processes & applications<\/li>\n
  4. Inference for stochastic processes<\/li>\n
  5. Monte-Carlo methods (MCMC, stochastic approximation, particle filter, … as time permits)<\/li>\n<\/ol>\n<\/td>\n
\"HajekBook\"<\/a><\/td>\n\"Book\"<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

Most of the topics in this course are basic to any machine learning \/ statistical learning course.\u00a0 <\/span>Applications to stochastic optimization and reinforcement learning are touched on throughout;\u00a0 <\/span>topics such as relative-entropy, mutual information, and hidden Markov models are relevant in every corner of communications, control and signal processing. Wikipedia will list applications ranging from \u2000nance to natural language processing.<\/p>\n

Essential background<\/strong>: Probability theory and appreciation for mathematical foundations.\u00a0 Experience with Matlab or Python is also essential.<\/p>\n

Notes for 2023:<\/strong><\/p>\n