Sparse Communication for Policy Shaping in Multi-Agent Reinforcement Learning.

Efficient coordination under limited communication is a central challenge in multi-agent reinforcement learning (MARL). Existing approaches often focus on message exchange without explicitly modeling how communication affects policy learning, leading to redundant interactions and limited coordination gains. In this paper, we propose a threshold-gated sparse communication framework built upon QMIX, a monotonic value-decomposition method that mixes individual agent action values into a global team action value. In the proposed framework, communication is integrated into the agent utility function to directly influence policy learning. Each agent encodes local observations into structured representations and activates communication through a learned trigger mechanism. Messages are aggregated via neighbor-constrained attention and incorporated into utility estimation for decentralized decision-making. Experimental results on the StarCraft Multi-Agent Challenge (SMAC) benchmark show that the proposed method improves coordination quality and training stability while significantly reducing communication frequency. On MMM, the Marine-Marauder-Medivac heterogeneous scenario, the communication rate is reduced to approximately 30-38% while achieving up to 96.6% win rate, compared to 92.1% for QMIX. On 10m_vs_11m, a homogeneous scenario where ten allied Marines fight against eleven enemy Marines, communication remains within 28-37% while reaching 88.4% win rate, compared to 85.6% for QMIX. Moreover, on the same task, varying communication thresholds induce clearly differentiated policy behaviors, indicating that sparse communication not only reduces overhead but also plays a critical role in shaping coordination policies. These results demonstrate that selective communication enables efficient coordination while explicitly regulating policy formation.