Action-factorized Rainbow deep Q-network with token Transformer for computation offloading in edge computing-enabled Internet of Ships.

Edge computing (EC) in the Internet of Ships (IoS) reduces the latency and energy burdens of cloud-centric architectures, but fully realizing its benefits requires effective computation offloading strategies. Designing such strategies in dynamic maritime environments remains challenging due to the high-dimensional, combinatorial decision space, strict system constraints, and rapidly varying maritime wireless channels. This study proposes the action-factorized Rainbow deep Q-network (DQN) with token Transformer, a deep reinforcement learning (DRL) algorithm for discovering effective computation offloading strategies in EC-enabled IoS (EC-IoS). The core innovation of the algorithm lies in a novel action factorization mechanism coupled with our custom token Transformer-based state and action encoders, which effectively handle the complex decision space. Built upon Rainbow DQN and further accelerated with a parallel training architecture, the algorithm improves learning efficiency and stability. Experimental results illustrate that the computation offloading strategies learned by our algorithm significantly outperform multiple baselines on the weighted latency-energy objective. More importantly, these strategies achieve a zero rate of invalid actions, satisfy all system constraints, and ensure practical feasibility. Overall, the study demonstrates that the algorithm provides a robust method for computation offloading, effectively balancing latency and energy consumption in EC-IoS, thereby supporting maritime digitalization and automation.