General Information:

Title                  Multi-Agent Learning
Type                 Research Project
Lecturer           Jun.-Prof. Dr.-Ing. Setareh Maghsudi
ECTS Points    9
Duration          One semester (Including Summer time)
Begin                April (for summer semester) and October (for winter semester)
Location           Online via Zoom
Level                 M.Sc. (Semesters 1-3)
Frequency        Every Semester

Content:

The students can work individually or in a team of two. Each student (or each team) selects a topic in the area of multi-agent learning, as guided by the lecturer. The project includes the following steps: State-of-the-art literature research; Problem formulation; Solution development; Theoretical or numerical analysis; Writing a term paper.

Examples of potential Topics:

• Multi-Agent Teacher-Student Framework: In a teacher-student framework of reinforcement learning, the teacher assists the learner to learn the best action while having a budget; For example, it can answer a specific number of questions. There are several algorithms for this scenario. What would be the model and solution if multiple teachers (with different knowledge area/level) or students (with different interest areas/performance level) exist?
• Multi-Agent Inverse Reinforcement Learning: In inverse reinforcement learning, the goal is to learn the utility function of the agent rather than the optimal decision-making policy. In a multi-agent scenario, however, the agents affect each other that might complicates the problem. The question to answer is to investigate such complications and proposing solutions.
• Cooperative Learning in Networks: Consider a scenario where several non-identical learners would like to cooperate with each other to learn the best action among many, or to develop a model. What are the strategies of coalition formation? How these are affected by the similarity of agents, their distance in network, etc.?
• Multi-Agent Off-Policy Evaluation and Learning: How to learn from logged data and counterfactual reasoning to improve the learning policy, especially when some features of the system changes over time? Which factors change if there are several agents, which affect each other’s utility, involve in the environment? How to incentivize the agents to share the logged data or the learning’s outcomes?
• Communication-Efficient Distributed Learning: Assume that several non-identical learners would like to cooperate with each other to learn the best action among many via reinforcement learning or to develop a model, etc. How this can be done if sharing the feedback between learners is costly and therefore sharing shall be done as scarce and as fast as possible?

 

Qualification Objectives:

After this project, the students can identify the problems in both theoretical applied domains that can be modeled and solved by combining the methods and concepts from game theory and machine learning. The students also possess the ability to distinguish the problems based on the required effort to solve, the most appropriate research methodology, and the like. Besides, they have a high level of proficiency in selecting, reading, analyzing, and criticizing scientific results efficiently. They can do independent research, prepare technical report and presentation slides, hold talks, and participate in discussions.

Participation Requirements:

Participation in the following lectures: “Game Theory with Application to Multi-Agent Systems” and “Mathematics for Machine Learning“, or provable good knowledge of game theory and machine learning.

Evaluation:

The evaluation is based on the results and quality of the term paper. Each student receives an individual grade based on her/his contribution to the term paper.

Registration:

If you are interested, please send an email to <setareh.maghsudispam prevention@uni-tuebingen.de> with “Research Project Multi-Agent Learning“ in the subject line. Please include your transcripts and half a page about the topics that interests you. You do not have to select from the topics listed in “Description”. Those are some example. If you have some other ideas and interests, feel free to share.

Literature:

A good tutorial, including some references, can be found here [Accessed 30.01.2021]: www.cs.utexas.edu/~larg/ijcai17_tutorial/multiagent_learning.pdf