Unmanned aerial vehicles (UAVs) are envisioned to complement the 5G communication infrastructure in future smart cities. Hot spots easily appear in road intersections, where effective communication among vehicles is challenging. UAVs may serve as relays with the advantages of low price, easy deployment, line-of-sight links, and flexible mobility. In this paper, we study a UAV-assisted vehicular network where the UAV jointly adjusts its transmission power and bandwidth allocation under 3D flight to maximize the total throughput. First, we formulate a Markov Decision Process (MDP) problem by modeling the mobility of the UAV/vehicles and the state transitions. Secondly, we solve the target problem using a deep reinforcement learning method, namely, the deep deterministic policy gradient, and propose three solutions with different control objectives. Then we extend the proposed solutions by considering the energy consumption of 3D flight. Thirdly, in a simplified model with small state space and action space, we verify the optimality of proposed algorithms. Comparing with two baseline schemes, we demonstrate the effectiveness of proposed algorithms in a realistic model.
翻译:根据设想,无人驾驶飞行器(UAVs)将在未来智能城市补充5G通信基础设施。热点很容易出现在道路交叉点中,车辆之间的有效通信具有挑战性。无人驾驶飞行器可以作为中继器,其优点是价格低、部署容易、视线连接和灵活的机动性。在本文中,我们研究无人驾驶飞行器协助的车辆网络,无人驾驶飞行器在3D飞行下联合调整其传输动力和带宽分配,以最大限度地实现总吞吐量。首先,我们通过模拟无人驾驶飞行器/车辆的流动和状态过渡,制定Markov决定程序(MDP)问题。第二,我们采用深度强化学习方法(即深度确定性政策梯度)解决目标问题,并提出三种解决办法,实现不同的控制目标。然后,我们扩大拟议的解决办法,考虑3D飞行的能源消耗量。第三,在使用小型国家空间和行动空间的简化模型中,我们核查拟议算法的最佳性。与两个基线方案相匹配,我们以现实模式展示拟议算法的有效性。