Adjunct Prof. Dr.-Ing. Mahdi Bohlouli

Personal Academic Page

Reinforcement Learning Course

Course Description:

Reinforcement Learning (RL) is a Machine Learning (ML) paradigm that focuses on goal-directed learning from interactions. In particular, it is learning how to map situations to actions by maximizing a scalar reward signal. RL is studied in other disciplines such as game theory, robotics, operations research, and multi-agent systems. It has roots in psychology and the advancements in psychology contributed to the advancements of RL and vise versa. It has been existing for some years, has been highlighted, and gained again the attention of ML researchers in recent years, especially in terms of Deep Reinforcement Learning (DRL). In this course, we will cover the difference between RL and other ML paradigms such as supervised learning and unsupervised learning, exploration and exploitation dilemma, main elements of RL systems, model-free, and model-based methods, planning, control and how to design RL algorithms to RL problems.

Learning Outcomes:

At the end of the course you will:

  • know the fundamentals of reinforcement learning
  • know different RL problems and solutions
  • implement RL methods

Course Outline:

Week # When What Who Topic Slides Recordings
Week 1 26.04.2020 Lecture Bohlouli Introduction to the course and logistics Course_Intro.pdf Rec_wk1.1
28.04.2020 Lecture Bohlouli An Introduction to Reinforcement Learning RL_Intro.pdf Rec_wk1.2
Week 2 03.05.2020 Tutorial Bohlouli Markov Decision Process MDP.pdf Rec_wk2.1
05.05.2020 Lecture Nazeri Elements of RL Systems: Environments Blackboard Rec_wk2.2
Week 3 11.05.2020 Lecture Bohlouli Planning by Dynamic Programming Planning_DP.pdf Rec_wk3.1
12.05.2020 Lecture Bohlouli Components of RL Agent RL_Comp.pdf Rec_wk3.2
Week 4 17.05.2020 Lecture Bohlouli Monte-Carlo and Temporal Difference Learning MC_TD.pdf Rec_wk4.1
19.05.2020 Lecture Nazeri Elements of RL Systems: Action Values Blackboard Rec_wk4.2
Week 5 24.05.2020 Lecture Bohlouli Public Holiday Public Holiday
31.05.2020 Lecture Bohlouli Model Free Control MFC.pdf Rec_wk5.2
Week 6 01.06.2020 Lecture Bohlouli Off-Policy Learning, Q-Learning off-Policy.pdf Rec_wk6.1
02.06.2020 Lecture Nazeri Blackboard Blackboard Rec_wk6.2
Week 7 07.06.2020 Lecture Bohlouli Value Function Approximation value-approx.pdf Rec_wk7.1
09.06.2020 Lecture Bohlouli Review Lecture Review Lecture Rec_wk7.2
Week 8 14.06.2020 Lecture Bohlouli Value Function Approximation, Incremental and Batch Methods batch.pdf Rec_wk8.1
16.06.2020 Tutorial Nazeri Blackboard Blackboard Rec_wk8.2
Week 9 21.06.2020 Lecture Bohlouli Model-Based RL mode-based.pdf Rec_wk9.1
23.06.2020 Lecture Bohlouli Model-Based RL (2nd part), Integrated Architectures model-based2.pdf Rec_wk9.2
Week 10 28.06.2020 Lecture Bohlouli Review and Students Presentation Round Discussion
30.06.2020 Tutorial Nazeri

Assignments:

Assignment # Release Date Description Submission Deadline Source Files
Assignment 1 05.05.2020 Soccer environment 19.05.2020, 16:59 CEST soccer_env.zip
Assignment 2 19.05.2020 Action Values 01.06.2020, 23:59 CEST Assignment 2, Supp_materials
Assignment 3 02.06.2020 OpenAI Gym 15.06.2020, 23:59 CEST Assignment 3
Assignment 4 20.06.2020 Value Iteration 29.06.2020, 23:59 CEST Assignment 4
Assignment 5 30.07.2020 13.07.2020, 23:59 CEST

Course Student Presentations:

Date Topic Student Name Slides
19.05.2020 A Natural Policy Gradient Amirreza Mohammadi
26.05.2020 Approximately Optimal Approximate Reinforcement Learning Ehsan Rassekh
23.06.2020 Robot learning from demonstrations Sarina Danaei
30.06.2020 The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems Fatemeh Merhara
07.07.2020 Hysteretic Q-Learning : an algorithm for decentralized reinforcement learning in cooperative multi-agent teams Reza Khaleghi

Alternative Presentation Topics:

Date Topic Student Name Slides
Policy Gradient Methods for Reinforcement Learning with Function Approximation
Algorithms for inverse reinforcement learning
Kernel-Based Reinforcement Learning
An Analysis of Temporal-Difference Learning with Function Approximation
Reinforcement Learning of Motor Skills with Policy Gradients

Final Project:

Title Release Date Description Submission Deadline Source Files
Assignment 1 05.08.2020 21.08.2020, 16:59 CEST capstone.pdf

Prerequisites:

Before commencing this course, you should:
  • have experiences and good knowledge of machine learning
  • be familiar with linear Algebra
  • have solid programming skills in Python
  • be familiar with working on a Unix-style operating systems

References:

Class Time and Location:

  • Sundays, 14:30 – 16:00 CET.
  • Tuesdays, 14:30 – 16:00 CET.
  • Given the current Corona Situation, this semester, the class will be completely online.
  • You should first apply for approval through the following link. This link will be also the online course sessions every week.
  • Course Videos Link: [https://vuniv.iasbs.ac.ir/b/dr–xea-944]

Final Exam:

  • The final exam will be held on 15.07.2020, at 11:30 CET, in a written form and online.

Course Links:

Piazza Course Page: [piazza.com/iasbs.ac.ir/spring2020/rl101]
Course Videos Link: [https://vuniv.iasbs.ac.ir/b/dr–xea-944]

Instructors:

Social media & sharing icons powered by UltimatelySocial