Class Autonomous Agent Systems

The concept of an agent or autonomous entity is a central focus in video game development, especially in an Artificial Intelligence (AI) context. This crucial concept is introduced in this course in an AI context, firstly using ad-hoc authoring (e.g., through state machines and decision trees), which give an illusion of intelligence, albeit limited and reactive, moving on later to search techniques, which can already lead to surprising and unexpected decisions by agents. The next part of the course is devoted to three techniques widely used in video games, in particular reinforcement learning, pathfinding and goal-oriented planning systems, which can take the illusion of intelligent agent to a whole other level. The course ends by discussing the combination of several techniques (including some lectured in the Machine Learning Foundations course).

Not applicable

Representation, utility and learning. Agent behaviors with ad-hoc authoring: finite state machines, decision trees, behavior trees. Tree search algorithms: concepts of graphs and trees; pathfinding: Dijkstra and A*  algorithms, norms/distances: Euclidean, Manhattan; board game search: minimax, alpha-beta pruning, move ordering, iterative deepening, transposition tables; Monte Carlo Tree Search (MCTS). Reinforcement Learning (TD-Learning) and Q-Learning

The course is theoretical and practical, alternating between methods: TM1. Expository, in the presentation of the concepts. TM2. Demonstrative, in the demonstration of the concepts through examples. TM3. Participatory, in the resolution of problems with reference to the examples presented, and the use of exercise sheets to be solved autonomously by the students in class. TM4. By research, in the development of group projects.   Continuous assessment consists of completing projects throughout the semester, with the final component making up 90% of the grade, of which the last 10% consists of the student’s participation and attendance in classes. The assessment includes three practical projects during the semester, each worth 30% of the final grade.

Millington, I. (2019). AI for Games (3rd ed.). CRC Press. Yannakakis, G. N., & Togelius, J. (2018). Artificial intelligence and games. Springer. Russel, S. and Norvig, P. (2021). Artificial Intelligence: A Modern Approach. Pearson Education Limited. Togelius, J. (2019). Playing smart: On games, intelligence, and artificial intelligence. MIT Press.