Size-Generalizable Reinforcement Learning for m,n,k-Games Using Fully Convolutional Networks

This study addresses the problem of zero-shot generalization (ZSG) in deep reinforcement learning by proposing an MNK game strategy learning method based on a Fully Convolutional Deep Q-Network (FCN-DQN). Research in deep reinforcement learning aims to develop algorithms that can generalize well to unseen environments at deployment time, thereby avoiding overfitting to the training environment. Solving this problem is crucial for real-world applications, where environments are diverse, dynamic, and inherently unpredictable. By constructing a fully convolutional reinforcement learning policy network composed entirely of convolutional layers with padding to preserve feature map dimensions, the proposed model is able to handle input boards of varying spatial sizes. The model effectively learns local pattern-based strategies and approximations of the k-in-a-row evaluation function rather than performing global search. Furthermore, due to parameter sharing, the network has a relatively small number of parameters and is able to share policy representations across different board scales, thereby improving both sample efficiency and inference efficiency. Experimental results demonstrate that, after being trained on a 3×3 board, the proposed model is able to achieve a certain degree of zero-shot generalization performance in larger, unseen board environments.