Reinforcement Learning-Based Strategic Level Generation for Turn-Based Grid Games

Abstract

Procedural content generation for turn based strategic games is challenging due to the need to satisfy multiple competing design objectives such as strategic depth balance and playability. Traditional manual and search based generation methods are time consuming and often fail to scale effectively under complex design constraints. This thesis presents a reinforcement learning based procedural content generation framework for automated level generation in turn based strategic games. In this approach, Proximal Policy Optimization is used to develop agents within a Gymnasium-compatible grid environment. There are a combination of strategies and measuresemployed in the objective for this approach to aid in developing agents. These include developing agents with diverse paths and complex decisionmaking points while also taking into account the layout of the main elements of a game (e.g., start point, end point, obstacles, objectives). The model is trained for five hundred thousand timesteps and evaluated on three hundred generated levels using three different random seeds. Experimental results show that the proposed reinforcement learning based generator consistently outperforms random baseline generation achieving substantially higher average quality scores with significantly lower variance. Statistical analysis confirms that the observed improvements are robust across all evaluation settings. Through additional evaluative efforts, qualitative evaluation of the methodology shows that the trained agent learns to exhibit meaningful behaviors related to design, including creating effective prioritized paths, building strategic decision points, and designing balanced rulebased gameplay through the use of re-enforcement learning methodology without the need for explicit constraint- based methods. The proposed framework expands the procedural content generation by means of reinforcement learning methodology to include turn-based strategy games and illustrates how practical applications can be used in the development of future games.