Postgraduate Course: Reinforcement Learning (INFR11010)
Course Outline
| School | School of Informatics |
College | College of Science and Engineering |
| Credit level (Normal year taken) | SCQF Level 11 (Postgraduate) |
Availability | Available to all students |
| SCQF Credits | 10 |
ECTS Credits | 5 |
| Summary | This module covers a range of adaptive learning systems, in particular reinforcement learning and unsupervised methods, particularly as used in RL systems. By the end of the module the student should have a grasp of modern learning techniques and the issues involved in dealing with real-world data. The main techniques covered in the course are basic reinforcement learning, dynamic programming, Monte Carlo methods, Q-learning, function approximation, unsupervised and constructive methods, radial basis and other local functions, classifier systems as compared to RL systems. |
| Course description |
The main topics to be covered are some or all of the following (there are some changes from year to year)
* Reinforcement learning framework
* Bandit problems and action selection
* Dynamic programming methods
* Monte-Carlo methods
* Temporal difference methods
* Q-learning and eligibility traces
* Environment modelling
* Function approximation for generalisation
* Actor-critic, applications
* Planning in the RL context
* Unsupervised, self-organising networks and RL
* Constructive methods - nets that grow
* Evaluating performance
Relevant QAA Computing Curriculum Sections: Artificial Intelligence, Data Structures and Algorithms, Intelligent Information Systems Technologies, Simulation and Modelling
|
Entry Requirements (not applicable to Visiting Students)
| Pre-requisites |
|
Co-requisites | |
| Prohibited Combinations | |
Other requirements | This course is open to all Informatics students including those on joint degrees. For external students where this course is not listed in your DPT, please seek special permission from the course organiser (lecturer).
Mathematical background, at the level of undergraduate informatics, particularly linear algebra, multivariate calculus and statistics. Some programming (e.g. in Matlab) will be required. |
Information for Visiting Students
| Pre-requisites | None |
| High Demand Course? |
Yes |
Course Delivery Information
|
| Academic year 2017/18, Available to all students (SV1)
|
Quota: None |
| Course Start |
Semester 2 |
Timetable |
Timetable |
| Learning and Teaching activities (Further Info) |
Total Hours:
100
(
Lecture Hours 20,
Seminar/Tutorial Hours 8,
Summative Assessment Hours 2,
Programme Level Learning and Teaching Hours 2,
Directed Learning and Independent Learning Hours
68 )
|
| Assessment (Further Info) |
Written Exam
80 %,
Coursework
20 %,
Practical Exam
0 %
|
| Additional Information (Assessment) |
Two assignments are set, each accounting for 10% of the overall mark. Typically they require programming a learning system or experimenting with an existing system, using MATLAB.
You should expect to spend approximately 25 hours on the coursework for this course.
If delivered in semester 1, this course will have an option for semester 1 only visiting undergraduate students, providing assessment prior to the end of the calendar year. |
| Feedback |
Not entered |
| Exam Information |
| Exam Diet |
Paper Name |
Hours & Minutes |
|
| Main Exam Diet S2 (April/May) | | 2:00 | |
Learning Outcomes
On completion of this course, the student will be able to:
- Knowledge of basic and advanced reinforcement learning techniques
- Insight into the problems involved in applying these techniques to deal with real world data, and how to overcome those problems
- Appreciation and identification of suitable learning tasks to which these learning techniques can be applied , and the ability to evaluate how effective a particular learning procedure has been -- internal indicators of learning success vs. actual behaviour of the learner
- Use and writing of Matlab programs, ability to set up and run computational experiments to produce statistically sound results
- Formulation of problems, evaluation of results from the student's own experiments and those presented in some cases in the research literature
|
Reading List
# Reinforcement Learning. An Introduction. Richard S. Sutton and Andrew G. Barto. MIT Press, Cambridge MA, 1998.
Probablistic Robotics, MIT Press 2006, Chapter IV, S.Thrun, W.Burgard, D.Fox
Algorithms for Reinforcement Learning, Morgan and Claypool Publishers 2010, C.Szepesvari
Reinforcement Learning: State-of-the-art, Volume 12, Springer Science & Business Media 2012. M.Wiering and M.V.Otterlo |
Contacts
| Course organiser | Dr Subramanian Ramamoorthy
Tel: (0131 6)50 9969
Email: |
Course secretary | Ms Alexandra Welsh
Tel: (0131 6)50 2701
Email: |
|
© Copyright 2017 The University of Edinburgh - 6 February 2017 8:08 pm
|
University Homepage