-
Presentation
Presentation
The course delves into the fields of modern artificial intelligence and machine learning, focusing on advanced automatic learning algorithms that interact with environments to maximize rewards. It encompasses areas such as automatic decision-making, optimization, robotics, and game theory. Its relevance lies in providing students with tools to tackle complex learning and control problems in various sectors, ensuring a comprehensive understanding of intelligent systems.
-
Class from course
Class from course
-
Degree | Semesters | ECTS
Degree | Semesters | ECTS
Doctorate | Semestral | 5
-
Year | Nature | Language
Year | Nature | Language
1 | Optional | Português
-
Code
Code
ULHT1504-25629
-
Prerequisites and corequisites
Prerequisites and corequisites
Not applicable
-
Professional Internship
Professional Internship
Não
-
Syllabus
Syllabus
Part 1: Introduction to Reinforcement Learning Overview of reinforcement learning and its applications Exploration-exploitation tradeoffs Types of reinforcement learning algorithms Part 2: Markov Decision Processes Introduction to Markov decision processes The Bellman equation Value and policy iteration; dynamic programming Approximate dynamic programming Part 3: Monte Carlo Methods Monte Carlo prediction Monte Carlo control On-policy and off-policy learning Part 4: Temporal-Difference Learning TD prediction Sarsa Q-learning Part 5: Function Approximation Linear function approximation Non-linear function approximation Deep Q-Networks Part 6: Applications of Reinforcement Learning Robotics Game playing Autonomous systems Part 7: Advanced Topics in Reinforcement Learning Exploration in deep reinforcement learning Multi-agent and game-theoretic reinforcement learning Consensus-based distributed reinforcement learning
-
Objectives
Objectives
The course objectives are: to introduce the fundamental concepts and principles of reinforcement learning and optimal control of Markov Decision Processes (MDPs); to provide a comprehensive understanding of the different approaches to solving MDPs, including value iteration, policy iteration, and dynamic programming; to teach different approaches and concepts in reinforcement learning, including exploration-exploitation tradeoffs, reward functions, temporal-difference, Monte Carlo, policy gradient and actor-critic methods; to introduce advanced topics such as deep reinforcement learning and multi-agent reinforcement learning including decentralized and distributed learning approaches based on the consensus algorithm; to equip students with the practical skills to implement and apply the algorithms to real-world problems and to encourage students to critically evaluate the strengths and limitations of the learned approaches, and to identify future research directions in the field.
-
Teaching methodologies and assessment
Teaching methodologies and assessment
Lectures: Lectures can be delivered through traditional classroom teaching, online videos, or pre-recorded lectures that students can access anytime. Group Discussions: Students can discuss their views and opinions on specific topics related to the course. This approach can help students to develop their analytical and critical thinking skills. Problem-Solving Exercises: Problem-solving exercises can be used to develop practical skills for implementing and applying reinforcement learning algorithms. These exercises can be done in groups or individually, and they can be based on real-world problems related to robotics, game playing, and autonomous systems. Assignments: These assignments can include programming assignments that require students to implement and apply reinforcement learning algorithms to real-world problems. Projects: Projects can be assigned to allow students to work on real-world problems related to robotics, game playing, and autonomous systems.
-
References
References
Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction", 2015. Yuxi Li, Deep Reinforcement Learning: An Overview, 2018. M.S. Stankovic, N. Ilic and S.S. Stankovic, “Decentralized Consensus-Based Estimation and Target Tracking“, Academic Mind, Belgrade, 2021 Vamvoudakis, K.G., Wan, Y., Lewis, F.L., Cansever, D. (eds), Handbook of Reinforcement Learning and Control. Studies in Systems, Decision and Control, vol 325. Springer, 2021. Bertsekas, Dimitri. Reinforcement learning and optimal control. Athena Scientific, 2019. Bertsekas, Dimitri P. "Dynamic programming and optimal control 4th edition, volume ii." Athena Scientific, 2015. Meyn, Sean. Control systems and reinforcement learning. Cambridge University Press, 2022. Ba¿ar, Tamer, and Geert Jan Olsder. Dynamic noncooperative game theory. Society for Industrial and Applied Mathematics, 1998. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
-
Office Hours
Office Hours
-
Mobility
Mobility
No
