FedHeRM: Federated Heterogeneity-Aware Reinforcement Learning for Secure NOMA Resource Management in Vehicular Edge Networks
The rapid expansion of 5G/6G technologies and the burgeoning demand for low-latency, high-reliability services at the network edge challenge traditional centralized cloud architectures. Edge computing offers a promising solution, yet its dynamic, resource-constrained, and heterogeneous nature necessitates decentralized intelligence for effective resource management. This paper addresses these challenges by synergistically integrating Federated Learning (FL), Multi-Agent Reinforcement Learning (MARL), and Non-Orthogonal Multiple Access (NOMA) resource management. While Federated Multi-Agent Reinforcement Learning (FMARL) is a critical direction, existing methods struggle with extreme heterogeneity and privacy concerns. To overcome these limitations, we propose FedHeRM: Federated Heterogeneity-aware Reinforcement Learning for Secure NOMA Resource Management in Vehicular Edge Networks. FedHeRM models the vehicular edge environment as a Partially Observable Stochastic Game, where RSU agents learn optimal resource allocation policies. Our core innovation lies in a novel Heterogeneity-aware Federated Multi-Agent Reinforcement Learning (HeRA-FMARL) framework, which introduces a Dynamic Heterogeneity Measure (DHM) for adaptive weighted aggregation of model updates, significantly accelerating convergence and enhancing generalization across diverse agents. Furthermore, FedHeRM integrates robust privacy-preserving mechanisms, including Differential Privacy for local updates and Secure Aggregation protocols. Comprehensive experiments in a simulated vehicular edge environment demonstrate that FedHeRM significantly outperforms state-of-the-art baselines across critical metrics, achieving superior system throughput, lower task latency, reduced energy consumption, and enhanced user fairness, while maintaining excellent scalability and strong privacy guarantees. An ablation study confirms the crucial roles of its key components, further validating FedHeRM’s efficacy in highly dynamic and heterogeneous edge networks.
