Effect of Hybrid Model Structure on Reinforcement Learning Performance

Hybrid quantum-classical neural networks have emerged as promising tools within reinforcement learning (RL), potentially offering enhanced performance and reduced model complexity compared to classical neural networks alone. However, the influence of variational quantum circuit (VQC) structure, particularly with regard to entanglement topology, remains poorly understood. In this work, we explore how different entanglement configurations (none, linear, cyclic, and fully entangled) affect learning in hybrid quantum-classical models applied to the classic CartPole environment. Using a mean reward of 160 as a benchmark for successful learning in the CartPole environment, our hybrid model achieved this performance with 50 parameters, compared to the 86 required by the classical model. Our focus is on settings where the quantum component contains more trainable parameters than the classical counterpart, allowing us to assess performance under a quantum dominant hybrid model. We find that linear and cyclic entanglement structures consistently enable stable learning, while fully entangled and unentangled circuits did not consistently achieve stable learning. These results suggest that quantum reinforcement learning is sensitive to circuit structure, and they provide practical guidance for the design of effective hybrid models.