Course Description:
Reinforcement Learning (RL) is a Machine Learning (ML) paradigm that focuses on goal-directed learning from interactions. In particular, it is learning how to map situations to actions by maximizing a scalar reward signal. RL is studied in other disciplines such as game theory, robotics, operations research, and multi-agent systems. It has roots in psychology and the advancements in psychology contributed to the advancements of RL and vise versa. It has been existing for some years, has been highlighted, and gained again the attention of ML researchers in recent years, especially in terms of Deep Reinforcement Learning (DRL). In this course, we will cover the difference between RL and other ML paradigms such as supervised learning and unsupervised learning, exploration and exploitation dilemma, main elements of RL systems, model-free, and model-based methods, planning, control and how to design RL algorithms to RL problems.
Learning Outcomes:
At the end of the course you will:
know the fundamentals of reinforcement learning
know different RL problems and solutions
implement RL methods
Course Outline:
Week #
When
What
Who
Topic
Slides
Recordings
Week 1
26.04.2020
Lecture
Bohlouli
Introduction to the course and logistics
Course_Intro.pdf
Rec_wk1.1
28.04.2020
Lecture
Bohlouli
An Introduction to Reinforcement Learning
RL_Intro.pdf
Rec_wk1.2
Week 2
03.05.2020
Tutorial
Bohlouli
Markov Decision Process
MDP.pdf
Rec_wk2.1
05.05.2020
Lecture
Nazeri
Elements of RL Systems: Environments
Blackboard
Rec_wk2.2
Week 3
11.05.2020
Lecture
Bohlouli
Planning by Dynamic Programming
Planning_DP.pdf
Rec_wk3.1
12.05.2020
Lecture
Bohlouli
Components of RL Agent
RL_Comp.pdf
Rec_wk3.2
Week 4
17.05.2020
Lecture
Bohlouli
Monte-Carlo and Temporal Difference Learning
MC_TD.pdf
Rec_wk4.1
19.05.2020
Lecture
Nazeri
Elements of RL Systems: Action Values
Blackboard
Rec_wk4.2
Week 5
24.05.2020
Lecture
Bohlouli
Public Holiday
Public Holiday
—
31.05.2020
Lecture
Bohlouli
Model Free Control
MFC.pdf
Rec_wk5.2
Week 6
01.06.2020
Lecture
Bohlouli
Off-Policy Learning, Q-Learning
off-Policy.pdf
Rec_wk6.1
02.06.2020
Lecture
Nazeri
Blackboard
Blackboard
Rec_wk6.2
Week 7
07.06.2020
Lecture
Bohlouli
Value Function Approximation
value-appro x.pdf
Rec_wk7.1
09.06.2020
Lecture
Bohlouli
Review Lecture
Review Lecture
Rec_wk7.2
Week 8
14.06.2020
Lecture
Bohlouli
Value Function Approximation, Incremental and Batch Methods
batch.pdf
Rec_wk8.1
16.06.2020
Tutorial
Nazeri
Blackboard
Blackboard
Rec_wk8.2
Week 9
21.06.2020
Lecture
Bohlouli
Model-Based RL
mode-based.pdf
Rec_wk9.1
23.06.2020
Lecture
Bohlouli
Model-Based RL (2nd part), Integrated Architectures
model-based2.pdf
Rec_wk9.2
Week 10
28.06.2020
Lecture
Bohlouli
Review and Students Presentation
Round Discussion
—
30.06.2020
Tutorial
Nazeri
Assignments:
Assignment #
Release Date
Description
Submission Deadline
Source Files
Assignment 1
05.05.2020
Soccer environment
19.05.2020, 16:59 CEST
soccer_env.zip
Assignment 2
19.05.2020
Action Values
01.06.2020, 23:59 CEST
Assignment 2 , Supp_materials
Assignment 3
02.06.2020
OpenAI Gym
15.06.2020, 23:59 CEST
Assignment 3
Assignment 4
20.06.2020
Value Iteration
29.06.2020, 23:59 CEST
Assignment 4
Assignment 5
30.07.2020
13.07.2020, 23:59 CEST
Course Student Presentations:
Alternative Presentation Topics:
Final Project:
Title
Release Date
Description
Submission Deadline
Source Files
Assignment 1
05.08.2020
—
21.08.2020, 16:59 CEST
capstone.pdf
Prerequisites:
Before commencing this course, you should:
have experiences and good knowledge of machine learning
be familiar with linear Algebra
have solid programming skills in Python
be familiar with working on a Unix-style operating systems
References:
Class Time and Location:
Sundays, 14:30 – 16:00 CET.
Tuesdays, 14:30 – 16:00 CET.
Given the current Corona Situation, this semester, the class will be completely online.
You should first apply for approval through the following link. This link will be also the online course sessions every week.
Course Videos Link: [ https://vuniv.iasbs.ac.ir/b/dr–xea-944 ]
Final Exam:
The final exam will be held on 15.07.2020, at 11:30 CET, in a written form and online.
Course Links:
Piazza Course Page: [piazza.com/iasbs.ac.ir/spring2020/rl101 ]
Course Videos Link: [https://vuniv.iasbs.ac.ir/b/dr–xea-944 ]
Instructors:
Asst. Prof. Dr. Mahdi Bohlouli
Mohammad H. Nazeri