Communications in Mathematical Sciences

Volume 19 (2021)

Number 2

Deep fictitious play for stochastic differential games

Pages: 325 – 353

DOI: https://dx.doi.org/10.4310/CMS.2021.v19.n2.a2

Author

Ruimeng Hu (Department of Statistics, Columbia University, New York, N.Y., U.S.A.; and Department of Mathematics and Department of Statistics and Applied Probability, University of California, Santa Barbara, Calif., U.S.A.)

Abstract

In this paper, we apply the idea of fictitious play to design deep neural networks (DNNs), and develop deep learning theory and algorithms for computing the Nash equilibrium of asymmetric $N$-player non-zero-sum stochastic differential games, for which we refer as deep fictitious play, a multi-stage learning process. Specifically at each stage, we propose the strategy of letting individual player optimize her own payoff subject to the other players’ previous actions, equivalent to solving $N$ decoupled stochastic control optimization problems, which are approximated by DNNs. Therefore, the fictitious play strategy leads to a structure consisting of $N$ DNNs, which only communicate at the end of each stage. The resulting deep learning algorithm based on fictitious play is scalable, parallel and model-free, i.e., using GPU parallelization, it can be applied to any $N$-player stochastic differential game with different symmetries and heterogeneities (e.g., existence of major players). We illustrate the performance of the deep learning algorithm by comparing to the closed-form solution of the linear quadratic game. Moreover, we prove the convergence of fictitious play under appropriate assumptions, and verify that the convergent limit forms an open-loop Nash equilibrium. We also discuss the extensions to other strategies designed upon fictitious play and closed-loop Nash equilibrium in the end.

Keywords

stochastic differential game, fictitious play, deep learning, Nash equilibrium

2010 Mathematics Subject Classification

60G99, 68T20, 91A15, 91A26, 91B50

Received 10 January 2020

Accepted 30 August 2020

Published 12 April 2021