Abstract
Millimeter-wave (mmWave) communication, a cor-nerstone in the evolution of next-generation wireless networks, offers substantial bandwidth and plays a crucial role in advancing wireless connectivity capabilities. Nevertheless, the inherent directionality and susceptibility to blockages pose significant challenges for a cost-effective beam management in densely deployed networks. This paper presents a Contextual Combina-torial Beam Management (CCBM) framework, leveraging both location-aware link qualities and beam correlation to tackle the joint access point (AP) and beam selection problem in mmWave networks, with a specific focus on mitigating coordination overhead and balancing the load across APs. Built upon a formulated multi-armed bandit problem, CCBM significantly reduces the uncertainty during online probing process by employing early stopping and attention-based selection mechanisms. Theoretical analysis establishes the asymptotically optimality of the proposed approach, complemented by extensive evaluation results showcasing the superiority of our framework over other state-of-the-art schemes in multiple dimensions.