Q-Learning for Sum-Throughput Optimization in Wireless Visible-Light UAV Networks

Unmanned Aerial Vehicles (UAVs) Have Been Adopted as Aerial Base Stations (ABSs) to Provide Wireless Connectivity to Ground Users in Events of Increased Network Demand, and Points-Of-Failure Infrastructure (Such as in Disasters). However, with the Existing Crowded Radio Frequency (RF) Spectrum, UAV ABSs Cannot Provide High-Data-Rate Communication Required in 5G and beyond. to Address This Challenge, Visible Light Communication (VLC) is Proposed to Be Equipped on UAVs to Take Advantage of the Flexible and On-Demand Deployment Feature of the UAV, and the High-Data-Rate Communication of the VLC. However, VLC Has Strong Alignment Requirements between Transceivers, Therefore, How to Determine the Position and Orientation of the UAV is Critically Important for Sum-Throughput Improvement. in This Paper, We Propose Two Q-Learning based Methods to Maximize the Sum throughput of the Wireless Visible-Light UAV Network by Jointly Controlling the Position and Orientation of the UAV. the Results Show that the Proposed Approaches Can Achieve a Network-Wide Data Rate Very Close to the Optimal Solution Obtained by Exhaustive Search and Outperform Up to 18% Compared with the Intuitive Centroid-Based Method. Computation Complexity is Also Evaluated, Where Results Showing that the Proposed Two Q-Learning based Methods Can Both Consume Less Computational Time, I.e., Approximately 9 Times and 210 Times Less on Average Than that of the Exhaustive Search Approach.