Doctoral Thesis Proposal - Carlos G. Martin

May 5, 2026  10:00AM—11:30AM

Location:
4405 & Zoom - Gates and Hillman Centers

Speaker:
CARLOS G. MARTIN, Ph.D. Student, Computer Science Department, Carnegie Mellon University
https://carlosgmartin.com/

Solving infinite games with deep multiagent reinforcement learning

In this thesis, we study the problem of solving infinite games. Such games can have infinitely many states, actions, players, and steps. Unlike mean‑field games, our players need not be symmetric or exchangeable. Furthermore, we allow such games to have partial observability, hidden information, imperfect recall, stochastic state transitions, discontinuous utility functions, and interdependent social preferences (i.e., matrix-valued discount factors). Together, these properties can model a wide range of highly complex, real-world scenarios that defy traditional game-theoretic solvers.

To tackle this problem, we propose a unified framework grounded in deep multiagent reinforcement learning. It includes five core components.

  • First, it introduces randomized policy networks (RPNs) to model observation-dependent mixed strategies over infinite action spaces.
  • Second, it represents complex strategy profiles across an infinite continuum of players using player-to-strategy networks (P2SNs).
  • Third, it evolves these representations through a shared-parameter simultaneous gradient (SPSG), which extends the standard simultaneous gradient to this shared-parameter regime.
  • Fourth, to ensure computational efficiency, it estimates this gradient using randomized parameter perturbations via a joint-perturbation simultaneous pseudo-gradient (JPSPG).
  • Fifth, it employs approximate exploitability descent (ApproxED) with learned best-response functions (BRFs).

We propose to benchmark our approach on a diverse suite of real-world domains. These include financial markets, traffic flow, epidemiological contagion, energy grids, and evolutionary ecology.

Thesis Committee:
Tuomas Sandholm (Chair)
Vincent Conitzer
Fei Fang
Ian Gemp (Google)

Additional Information

In-person and Zoom

Contact
Matt Stewart

Add event to Google
Add event to iCal