Prerpints.org logo
Preprint
Article

A Survey on Artificial Intelligence based Internet of Vehicles utilizing Unmanned Aerial Vehicles

Altmetrics

Downloads

329

Views

182

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

13 December 2023

Posted:

14 December 2023

You are already at the latest version

Alerts
Abstract
The vehicular ad hoc network (VANET) when augmented with internet is called internet of vehicles (IoV). In the IoV, vehicles have on board units (OBUs), use communication technologies and customized software to communicate with infrastructure around them. The vehicles can have permanent connectivity and global network awareness based on cloud computing. IoV technologies are still being assessed to ensure that the network is protected from attacks, supports a dynamic vehicular mobility scenarios and meets the low latency communication requirements between the vehicles and the surrounding infrastructure. To meet these strict Quality of Service (QoS) demands, Unmanned Aerial Vehicles (UAVs) play a vital role in IoV ecosystems. An UAV can serve as a mobile roadside unit (mRSU) to transmit data between vehicles, stationery RSUs and other UAVs. The UAVs have better line of sight (LOS) connections which result in better channel states and lower path loss than fixed RSUs. Although the inclusion of UAVs in vehicular networks has improved the system performance by providing better resource management and routing solutions, the traditional optimization techniques with their limitations are often not applicable. Therefore, in recent years, research community is integrating Artificial Intelligence (AI) and Machine Learning (ML) into UAV based IoV ecosystem to better manage resource allocation, routing and mobility management issues to optimize the overall network performance. In this survey paper we review the existing research done in AI/ML based UAV-IoV networks and communications with focus on resource management and routing from year 2019 to 2023. We have studied different AI techniques, their training attributes and architectures in various research works. The limitations of AI-based approaches in terms of required computational resources, availability of real world data and AI models’ complexity for the UAV-IoV environment are also discussed. Finally, the future directions for UAV-IoV research opportunities that can leverage the full potential of AI are presented.
Keywords: 
Subject: Engineering  -   Telecommunications

1. Introduction

The Internet of Vehicles (IoV) is one of the subsystems of Internet of Things (IoT) that consists of mobile vehicles installed with sensors, processing units and software to communicate with one another over the internet or other communication networks [1,2]. IoV is a decentralised system which promises the security and privacy of vehicular and users’ data. It integrates different technologies by providing reliable communication tools [3,4].
In an IoV, vehicle-to-everything (V2X) communication combines the data sharing from vehicles to infrastructure (V2I), between vehicles (V2V), pedestrian (V2P), roadside units (V2R) and unmanned aerial vehicles (V2U). This would play a key role in intelligently managing vehicular traffic as well as network data traffic, providing safer roads and increasing vehicular energy efficiency. However, V2X communication has it’s own limitations. In an IoV, the vehicles are highly mobile with varying densities and need permanent communication links for reliable data exchanges. In most cases, the fixed infrastructures such as fixed RSUs and BSs are unable to provide satisfactory communication and computational services to the vehicles and as a result of this, the QoS deteriorates.
Incorporation of unmanned aerial vehicle (UAV) with IoV can be very advantageous to guarantee a reliable communication infrastructure as it provides better line-of-sight connectivity that would greatly help in load balancing, mobility management, routing solutions and cost effective communications. An UAV can operate independently and has the ability to move and carry data from one place to another. Installed with sensors, computer units, cameras, GPS, wireless transceivers to communicate, an UAV can have a predetermined flight path, interact with the environment around it and if needed dynamically adapt their paths during the flight. Therefore, they are great assets to IoV systems [5]. In particular, UAVs can overcome the limitations of fixed RSUs because they can dynamically change their speed and position, and they can gather and transfer data from one area to another area [6].
Table 1. List of Abbreviations and Notations.
Table 1. List of Abbreviations and Notations.
Abbreviation Description Abbreviation Description
3DQN Double Dueling Deep Q Network LTE Long Term Evolution
3GPP 3rd Generation Partnership Program MACEL Multi-Agent Collaborative Environment Learning
5G 5th Generation MAC Medium Access Control
A3C Asynchronous Actor Citric MARL Multi-Agent Reinforcement Learning
AEC Average Energy Constraint MDP Markov Decision Process
ASR-CUMS Automated Slice Resource Control and Update Management System MJDDPG Multi-objective Joint Optimization-Oriented DDPG Algorithm
ARdeep Adoptive Reliable Deep MLP Multi-Layer Perceptron
BT-MP-DQN Beam-forming Control and Trajectory-Multi-Pass Deep Q Network mmWAVE Millimeter Wave
CA-MOEA Clustering based Adoptive Multi Objective Evolutionary Algorithm MOEA/D A Multi-objective Evolutionary Algorithm Based on Decomposition
CCSRL Cluster-enabled Cooperative Scheduling based on Reinforcement Learning MSA-LS Mobile Service Amount based Link scheduling
CEPF Context Aware Packet Forwarding NGSIM Next generation Simulation
CKF Constant Kalman Filter QAGR Geographic Routing with Q-Learning
CSMA Carrier Sense Multiple Access QFHR Q-learning and Fuzzy-based Hierarchical Routing Solution
DDPG Deep Deterministic Policy Gradient QLBR Q-Learning based Load Balancing Routing
DFS Depth First Search QLFMOR Q-Learning based Fuzzy Logic for Multi Objective Routing Algorithm
DGCIM Dual Graph Coloring based Interference Management QTAR Q-Learning based Traffic Aware Routing
DQL Deep Q-Learning RRB Radio Resource Block
DQN Deep Q-Network RRPV Reinforcement Learning Routing Protocol for Vehicles
DRL Deep Reinforcement Learning SGD Stochastic Gradient Descent
DRQN Deep Recurrent Object Networks SUMO Simulation of Urban Mobility
EED End-to-End Delay SWIPT Simultaneous Wireless and power Transfer
FANET Flying Ad-hoc Network UAS Unmanned Aerial System
FLRLR Fuzzy Logic Reinforcement Learning based Routing UE User Equipment
GCS Ground Control Station UCPA UAV based Clustering and Positioning Protocol
GMM Gaussian Mixture Model URLLC Ultra-Reliable Low-Latency Communications
GPGC-RLF Grouping Graph Coloring with Recursive Largest First V2N Vehicle-to-Network
GYGC Greedy Graph Coloring V2P Vehicle-to-Pedestrian
IMU Inertial Measurement Unit V2R Vehicle-to-Roadside Infrastructure
iProPHET Improved Probability Routing Protocol using History of Encounters and Transitivity VANET Vehicular Ad-Hoc Network
ITS Intelligent Transport Systems VEC Vehicular Edge Computing
JTSM Joint Time Series Modeling VUE Vehicle User Equipment
LBTO Load Balancing and Task Offloading WPT Wireless Power Transfer
LIDAR Light Detection and Ranging WMMSE Weighted Minimum Mean Square Error
LPA Long Prediction Algorithm WSN Wireless Sensor Network

2. Related Work and Survey Contribution

Since last decade, AI has been integrated in IoV as a viable solution to assist in many communication and traffic application related problems. AI combined with V2X can support complex vehicular applications such as traffic control, self governing vehicular mobility, and data storage. ML and Deep Learning (DL), one of the frequently used sub-classes of AI approaches, are thoroughly implemented in IoV to solve complex problems with the help of extensive amounts of available data [7].
Most DL based models require large datasets with various traffic features to train and test the models [8]. However, in vehicular communication, real-time historical data related to routing, channel condition, vehicle’s mobility and resources is not readily available. In such a scenario, it is not possible to use supervised DL techniques. In the absence of historical data, reinforcement learning (RL) has appeared as a very powerful solution in IoV which allows a vehicle to take the decisions by itself for various networking tasks [9,10,11]. The agents which are mostly the vehicles, gather the information about the dynamic environment and then takes an appropriate action to achieve a certain purpose such as resource and mobility management, routing decision and predicting network traffic.
The applications of AI/ML are extensively studied in wireless networks and VANET communications in the existing literature. In [12], mobility and traffic patterns are discussed using the AI/ML techniques in a dynamic vehicular network. The authors presented ML techniques to optimize network performance related to security, handover, resource management and congestion control. The survey in [13] classifies the vehicular research in transportation and network related areas. A detailed AI/ML based vehicular network scenario such as data offloading, mobile edge computing (MEC), network security and transportation cases, related to platooning, autonomous navigation and safety are discussed in detail [13]. The authors list all the software used to implement AI and also reviewed the relevant research problems in this field. In [14], a detailed review of ML techniques is presented in vehicular network related to resource and network traffic management, and reliability. The authors expanded their work to include cognitive radio (CR), beamforming, routing, orthogonal frequency-division multiple access (OFDMA) and Non-orthogonal multiple access tasks in [15].
An overview of ML, CR, VANET, and CR-VANET architectures including functions, open issues and future challenges is presented in [16]. Moreover, the applications of AI/ML in CR-VANET in autonomous vehicular network and their union is also reviewed in this paper. In [17], the authors first discussed the Federated Learning (FL) and its use in wireless IoT. Then, this survey paper points out and discusses the technical challenges for FL based vehicular IoT with future research directions. The survey paper [18], critically review the ML and Deep Reinforcement Learning (DRL) models for MEC decision based offloading in IoV. Main focus of the paper is on buffer and energy aware ML enabled Quality of Experience (QoE) optimization, and it summarizes the recent related research and methods and presents their comparison. In [19], the authors surveyed and analyzed the resource allocation scenarios. In addition to this, the design challenges for resource management in VANETs using ML are presented as well. In [20], a detailed overview of the RL and DRL techniques in IoV network such as joint user association and beam forming, caching, data offloading decisions, energy efficient management of resources and vehicular infrastructure management is presented. Then, future trends, challenges and open issue in 6G based IoV are discussed.
In [21], primary ML concepts based on ML for Wireless Sensor Networks (WSN) and VANETs are summarized briefly. Moreover, ML based WSN and VANETs applications, open issues and challenges are discussed. In [22] a comprehensive survey of AI/ML techniques is presented and then the strengths and weaknesses of these AI models for the VANET environment including safety, traffic, infotainment applications, security, routing, resource and mobility management are provided. In [23] authors surveyed resource allocation techniques on DSRC, Cellular-V2X (C-V2X) and heterogeneous VANET. The AI/ML techniques are reviewed with respect to their integration in VANETs and utilization in designing the several resource allocation tasks related to user association, handover and virtual resource management for V2V and V2I communications. However, AI/ML on V2X is not the main focus of the paper.
In [24], the RL-based routing schemes are classified depending upon the centralized and distributed learning process. Moreover, they surveyed position-based, cluster-based and topology-based routing protocol. The survey in [25] summarizes the vehicular network and Smart Transport Infrastructure (STI) in detail. The paper deals with FL and it’s application in vehicular networks. It elaborates vehicular-IoT (VIoTs), blockchain, FL, and intelligent transportation infrastructure. Then, the FL and blockchain based security and privacy applications in VANET environment are discussed in detail. The challenges arise from integration of FL and blockchain are pointed out in the survey with indication of future research direction. In [26], the survey presents a compilation of network-controlled functions that have been optimized through data-driven approaches in vehicular environments. The research related to integration of AI/ML and V2X communications in the areas such as handover and resource management or user association, caching, routing, beam-forming optimization and QoS prediction are extensively reviewed. This survey classifies the training architecture into centralized, distributed or federated model for each ML technique. The time complexity of supervised, unsupervised and RL models used in the literature are discussed.
In [27], the authors cover the applications of UAVs based IoV network. This work does not include the detailed implementation of AI/ML in UAV based IoV and only mentions a few papers related to software defined network (SDN) based fog computing and AI/ML networks. However, it covers the areas such as privacy, security, congestion and network delays, and communication protocols. In [28], the authors review the Internet of Drones (IoD) and classify the IoD-UAV according to its applications in the areas of resource allocation, aerial surveillance and security and mobility in all the possible IoT based fields. This survey concludes that the most used AI technique in IoD is Convolution Neural Networks (CNN) and the most common areas of research are resource and mobility management. However, this survey completely ignores the IoD based IoV networks. The summary of the survey papers with AI/ML application in IoV networks is provided in Table 2
In our study of the previous literature on UAV based IoV, it is noticed that the application of ML in resource management and routing for UAV or IoD based IoV are not explored yet. At the time of writing of this survey paper, there is no comprehensive survey conducted on this topic. We extensively reviewed the inclusion of autonomous vehicles (AVs) with UAVs and the integration of AI/ML applications in UAV based IoV communications for physical and computation resource allocation and routing algorithms. The paper presents the existing AI/ML based solutions in UAV-IoV, lists the challenges and issues in UAV based IoV with future directions.
The rest of the paper is organized as follows: A basic discussion on the nuts and bolts of IoV and UAV is presented in Section III. In Section IV, AI/ML based resource management in UAV-IoV, V2X and UAV networks is covered. Section V covers AI/ML application in UAV-IoV and V2X and UAV based routing protocols and Section VI presents the challenges and future work and finally we conclude the paper in Section VII.

3. Overview of IoV and UAV Networks

A UAV-based IoV network provides users such as vehicles, passengers and surrounding infrastructure fast data transmission services. It is the result of amalgamation of heterogeneous networks at different computing and communication layers [49,50]. To overcome the limitations of the IoV network related to routing, mobility management, resource allocation, network monitoring and traffic applications, UAVs are proving to be a viable technology. UAV-based IoV networks use cellular network and communication protocols to ensure uninterrupted network connectivity to meet QoS requirements [14]. A UAV assisted IoV network architecture is shown in the Figure where UAVs are flying in the urban area, facilitating the underneath vehicles and infrastructure by providing communication facilities. The UAVs also do communicate with one another.
This section provides a detailed overview of IoV and VANET communication technologies and UAV components and network architecture. The taxonomies and advantages of these technologies are also provided.

3.1. IoV Communication Technologies

IoV is the VANET with internet which provides real-time communication among the vehicles and infrastructure (V2I), vehicles (V2V), pedestrians (V2P), roadside units (V2R) and UAVs (V2U). VANET does not have the access to internet and vehicles communicate with infrastructure using Dedicated Short Range Communication (DSRC) [29]. DSRC is built on IEEE 802.11p [30] and in 2012 became a standard with allocation of 75 MHz bandwidth within the frequency range of 5.85-5.925 GHz by the Federal Communications Commission (FCC). It can achieve the latency of as low as 100 ms. However, DSRC is not good enough to support latency requirements of the future communication based applications as DSRC is based on Carrier-Sense Multiple Access with Collision Avoidance (CSMA/CA) Medium Access Control (MAC) protocol which causes unbounded latency and reliability issues in vehicular communications [31].
Visible Light Communications (VLC) have the potential to solve the spectrum scarcity issue. Its spectrum ranges from 430 to 790 THz [32]. As compared to DSRC, VLC is not impacted by electromagnetic interference. It has low latency, and is less vulnerable to security related attacks than radio base systems [33].
However, the aforementioned standards are unable to assure steady network connectivity to highly mobile vehicles. The long term evolution (LTE) or device to device (D2D) and DSRC can only provide up to 100 Mbps or 3-27 Mbps respectively [34]. The milimeter wave (mmWave) can provide more than 1 Gbps for V2V communication [35,36] and it appeared to be very promising. Recently, mmWave based Giga-V2V (GiV2V) has got a lot of attention in VANET communication. The mmWave is a good fit for applications with rich data and high definition requirements such as camera and LiDAR sensors. But mmWave has its own drawbacks. For instance, its range is limited, it has high penetration loss, LoS requirements, poor diffraction characteristics and mainly it is more costly.
In 2017, Cellular-V2X (C-V2X) was introduced by the 3rd Generation Partnership Project (3GPP) which relies on the capabilities of 4G, 5G and future 6G cellular networks [37]. C-V2X provides high level of safety by ensuring the elevated system performance, communication range and security. In C-V2X, as soon as the wireless link breaks between the cellular BS and vehicle, PC5 [38] interface creates a direct communication channel to provide uninterrupted communication link between two vehicles. The Uu [39] is another logical interface used in C-V2X. IoV-6G could prove a ground breaking technology as it overcomes the shortcomings of 5G by fulfilling the strict key performance indicators (KPIs) [40]

3.2. UAV

Unmanned Aerial System (UAS) is consisted of an UAV and a remote control system to operate the UAV [41]. The basic components and the communication architecture of UAV are discussed in this section.

3.2.1. Components of UAV

A UAV used in IoV environment is often made up of single board computer with CPU, memory and sensors. The sensors play pivotal role in sensing the environment including the global positioning system (GPS), accelerometers, camera and gyroscope for navigation [42], a battery to provide power, a transceiver to exchange data between UAVs and Ground Control Station (GCS) [43], UAV flight status indicator devices [44], flight controller to control UAV take off and landing etc. [45], and finally an inertial measurement unit (IMU) to manage UAV altitude [46].
In UAV based IoV, an UAV and a vehicle can communicate in real-time using a data link, namely Line-of-Sight (LoS), Visual Line of Sight (VLoS) and Beyond Line-of-Sight (BLoS) link. This enables the UAVs to work as mobile base stations (mBSs), to transmit and receive data to and from vehicles, fixed infrastructure, and other UAVs [6,47].

3.2.2. UAV Communication Architecture

UAV communication is divided into two categories namely, centralised communication and decentralised communication.
In centralised communication as shown in Figure 2, UAV communicates with a central controller. There are three types of centralised communications. First is UAV-GCS in which UAV fetches data from GCS using a communication link. This setup does not perform well in adverse weather conditions. Second is UAV-satellite or UAV-High Altitude Platform (HAP) communication which is preferred when there is large distance between UAV and GCS. Third one is cellular technology based UAV-cellular communication. It uses BSs to facilitate the routing technology between nodes [48,49].
In decentralised communication architecture as shown in Figure 3, UAVs build a direct or indirect communication link with the GCS. The gateway UAVs can relay data between GCS and other UAVs in the network. The UAVs, in ad-hoc networks, communicate with each other via wireless devices without relying on GCS [50,51]. There are three types of decentralized UAV communication networks namely simple UAV, multi-group and multi-layer ad-hoc network. In UAV ad-hoc network, the backbone UAV communicates with GCS using high power over long ranges. Then, the backbone UAV becomes a gateway and communicates with other UAVs at short ranges using low power. In multi-group UAV ad-hoc network, each UAV group behaves as FANET with one UAV selected as a backbone UAV to communicate with the GCS. Lastly, in multi-layer UAV Ad-hoc network, the lower layer is used for intra-group communication and upper layer is made up of backbone UAVs to ensure inter UAV and GCS communication. In UAV based communication, multiple links formation help covering the large communication area [48].

4. AI based Resource Allocation in UAV and IoV Networks

In this section, we cover the research contribution in the area of resource allocation for IoV, UAV as well as UAV based IoV networks using AI.

4.1. AI for Resource Allocation in IoV Network

Different objectives have been considered by the authors for AI based resource allocation ranging from load balancing, better QoS or QoE to energy and latency minimization. In this section we focus on reviewing the contributions and research progress in AI based resource allocation for IoV networks.
In [52], the authors combine the network function virtualization (NFV) and QoS guaranteed network slicing (NS) with RL in beyond 5G (B5G) architecture. The slice acceptance advantage of 5% is achieved as compared to the best existing work. One impediment of the research is that the authors assumed the RRB resource allocation in the RAN over direct data rate in RAN. Secondly, there is room for improving the slice allocation capability of the proposed algorithm.
In [53], the authors devised a method for multiple V2V links to reuse the V2I spectrum by using the multi-agent RL. In the training stage proposed model is centralized and in implementation stage it is decentralized. The MARL and SARL algorithms are used for the comparison purpose. The proposed model considerably improves the overall system level performance.
In [54], the authors proposed an AI based task offloading and resource management model. There are two offloading layers proposed. First layer selects between cloud computing (CC) and MEC server using random forest (RF) for task offloading and computing platform is selected in second layer by using DDPG algorithm. The proposed model is compared with KNN, MLP and SVM. T-Drive trajectory dataset, from Microsoft website, is used for the model training and testing. This dataset includes the GPS trajectories of 10,357 taxis during one week in Beijing. The RF model achieves the highest classification accuracy score of 99.83% for task offloading decision as compared to the KNN, MLP and SVM achieved 98%, 94.81% and 90.94%, respectively. Moreover, it is reported that the DDPG significantly minimized the latency cost by 85%.
In [55], the authors proposed the algorithm to build an energy efficient communication system using capabilities of AI and intelligent reflecting surface (IRS). The IRS reflecting signal propagation is used to produce efficient beam-forming for directional transmission in vehicles. Then, DRL model is used for network resource control and allocation. An energy efficiency-maximizing model is formulated and joint optimization of the settings of all vehicles is done for effective and latency-efficient communication. The proposed model is compared with model without IRS and random resource allocation model and the proposed model outperforms both the models.
In [56], a model named ARTNet is proposed to make an AI enabled V2X framework for dividing resources effectively and improvised communication. To achieve their target the authors implemented the proposed algorithm in the software defined vehicular based FC (SDV-F) architecture. In ARTNet, the controllers are responsible to achieve optimal resource utilization at the fog layer, and maximum reduction in the average delay of time critical IoV applications. The proposed model is compared with NRPO and AEC in terms of energy consumption, average latency and energy shortfall.
In [57], an integrated fuzzy based approach is taken to coordinate and manage the abundant resources available in VANETs. In this approach, the system decides the resources that vehicles should use when set under different circumstances including the condition of the network created among vehicles, which is represented by the QoS in the network, its time duration, its size, and the available resource, together with the application requirements. The proposed method achieves higher QoS.
In [58], to tackle the problem of scarcity of computational resources, a selection criteria is proposed to select volunteers’ vehicles capable of executing the computationally intensive task. For the volunteer vehicle identification, the authors used various machine learning based regression techniques including LR, SVR, KNN, DT, RF, GB, XGBoosting, AdaBoost, and ridge regression. For the training and testing of the models vehicular onboard unit computing capability dataset is collected. It contains three different datasets. All three datasets have seven features but different number of samples or sizes. The dataset1 has 17800 samples, dataset2 is made up of 4600 samples and finally dataset3 has only 250 samples. Among all the models implemented, SVR and ridge regression have lowest error value and maximum accuracy.
In [59], the authors predicted leading vehicle trajectory with the speed and acceleration information of the vehicle at front using the proposed method based on joint time-series modeling approach. Moreover, before making any prediction the gaussian mixture model (GMM) is applied on the leading vehicle to predict the driving styles. After style prediction, a LSTM and RNN based proposed joint time series modeling (JTSM) method is implemented to predict the front vehicle trajectories. The Next Generation Simulation (NGSIM) dataset on the US101, and I-80 is used to train and test the proposed model. The proposed model is compared with constant kalman filter (CKF), LSTM, multiple LSTM (MLLSTM). The proposed model shows significant improvement in terms of RMSE.
In [60], authors predict vehicles’ trajectory by using LSTM algorithm. Then, the predicted value is provided to the QL algorithm to figure out the optimal resource allocation policy for the nodes. The real world vehicle trajectory data is used in this research provided by Didi Chuxing, a ride sharing company. The dataset contains 600,000 traffic trajectory samples. The ultimate goal is to enhance the QoS for non-safety related services in MEC-based vehicular networks and the proposed model outperforms the other models.
In [61], the authors implemented RNN to predict resource availability pattern based on the position of vehicle and RSU at the time of prediction. The RSU burden is shared by the parked and slow vehicles. The performance of the proposed method is better than the heuristic and other ML based resource allocation algorithms.
In [62], for improved communication and network security, the authors proposed a cluster-enabled cooperative scheduling based on RL (CCSRL). The experimental results show that the CCSRL provide better PDR and PLR as compared to the Wu’s scheme and mobile service amount based link scheduling (MSA-LS). MATLAB is used as simulation software, Veins software is used to generate the vehicle distribution, and SUMO generates the road topology.
In [63], the authors divide the coverage area into clusters and then federated DRL obtains the global model for each cluster. The global model is implemented every time there is a new vehicle added in the cluster. The performance of the proposed algorithm is better than the other decentralized learning scheme.
In [64], the researchers introduced a wireless full-duplex (FD) technique to enhance spectrum efficiency and optimize system throughput. This approach employs V2V links within an FD cellular network. Managing resource allocation and power control becomes challenging due to the interference between self-interference (SI) at FD base stations (BS) and V2I links. To address this issue, they proposed a Dual Graph Coloring-based Interference Management (DGCIM) scheme. The two graph coloring schemes employed are the Grouping Graph Coloring with Recursive Largest First (GPGC-RLF) and Greedy Graph Coloring (GYGC) algorithms, respectively. Simulation results demonstrate that the proposed model satisfies high reliability and low latency constraints for V2V links.
In [65] a VNF-RACAG scheme is proposed to derive the optimal number of clusters. It reduced and minimizes the end-to-end delay in edge networks as well as allocated physical resources to users and minimize the number of transfers between clusters. The VNF-RACAG scheme uses a NFV stochastic model as well as geographic contexts and transfer history of the users in the chaining time optimization process. The proposed scheme is compared with the WiNE and PSwH schemes. The simulation are done in MATLAB and the results showed that end-to-end delay is minimized by using the proposed VNF-RACAG algorithm. The VNF-RACAG scheme implemented gradient descent to calculate the number of clusters to minimize the end-to-end delay. After this, a graph partitioning algorithm is performed to minimize the movement between the clusters.
In [66], the authors performed the resource assignment and video selection for vehicular devices by using the stochastic optimization. The aim is to achieve maximum QoE and secure a threshold duration of video at each receiver. A highway environment is simulated with different vehicular densities. The proposed method achieves better results with compared with two different baseline methods.
In [67], Two-Arch2 algorithm is adopted by introducing a hierarchical clustering idea to optimize 5G based IoV architecture. The proposed algorithm is compared with NSGA-III, Two-Arch2, CA-MOEA and MOEA/D. The experimental results show that the improved algorithm can obtain the better resource allocation scheme than the other algorithms in terms of stability, energy consumption and load balancing.
In [68], a multi-platform intelligent offloading and resource allocation algorithm is proposed to dynamically organize the computing resources. KNN algorithm selects the best option available out of cloud computing, mobile edge computing, or local computing platforms. In addition, when the task is offloaded to a desired server, RL is implemented to solve the resource allocation strategy. The state is defined as the MEC computing capacity, actions are offloading decision and computation resource allocation and the reward is the minimum total cost. The proposed joint optimisation is compared with full MEC and Full local techniques and it is concluded that the proposed scheme reduced the total system cost and optimize the overall system performance.
In [69], a centralized multi-agent learning is adopted for joint communication, computing and caching resource allocation problem. The RSUs are controlled by central the controller to assign resources in their coverage areas. The soft actor critic (SAC) algorithm updates policy in the central controller and share with edge agents to regulate their resource allocation strategies. The proposed algorithm increases the system’s performance as compared to other schemes.
In [70], the authors optimized the resource allocation and computation offloading by using MDP. The aim is to maximize the network operators’ revenue and for this DDPG algorithm is used to optimize the the wireless and computing resources dynamically. Compared with average and random algorithms methods, the suggested method is more effectual with little resources. This paper ignores the dynamics of task offloading as optimization.
In [71], for V2V communication, the authors proposed a RL based decentralized resource allocation mechanism. It is applied to both unicast and broadcast scenarios. V2V links are agents and based on the minimum interference select their spectrum and transmitted power for V2I and V2V links. The transmitted power is divided into three levels and the agents select them based on their state information.
In [72], the authors perform queue-length resource allocation. At controller level, the network safety flows are managed. The safety flows get more priority ratio based on the criticality, and the non-safety flows get less priority. The bandwidth allocation is the main fairness allocation criterion to obtain maximum rate for different application. The simulation environment use mininet-wifi for multiple RSUs and vehicles to communicate in V2v and V2I scenarios. The authors implemented LSTM, CNN, and DNN and compare their results. LSTM outperforms all models in terms of accuracy.
In [73], the authors used RNN to automate the resource management for the IoV network. The dataset used is called GWAT-13 Materna dataset with 12 attributes. available on Materna 13, an open source directory. It has three traces expanded over three months period with each trace having 850 VMs data on average. The prediction results of ML models are used by ASR-CUMS to decide the resource requirements and update the physical resources.
In [74], the authors proposed a DL based CSI estimation technique is proposed. It is assumed that the environment remains the same so the DNN model can learn the non-linear CSI relations with insignificant overhead. Moreover, a resource allocation based on dynamic eMMB and URLLC network slicing is implemented for vehicles. The proposed scheme attains 50% reduction in overhead and 12% higher threshold violations compared to an ideal case with perfect CSI knowledge.
In [75] the authors maximize the overall system throughput by implementing the weighted minimum mean square error (WMMSE) algorithm for power allocation. Simulation results show that the DNN algorithm improves the approximation of the iterative WMMSE algorithm achieves the computational overhead reduction.
ML techniques have proven to be very effective in VANET environment yet face some issues. For instance, the SVM faces issues such as the choice of the optimized kernel and models complexity, for KNN, the optimum value of k changes from one dataset to another which makes it a time-consuming and complex method to find the optimum value of k, RF focuses on the construction of several DTs which is time consuming and in vehicular network for time-sensitive applications this method is not recommended. The RL is the most commonly applied technique in IoV. However, it is noted that in vehicular networks it suffers from the issues of dimensionality makes it harder to implement RL on physical systems.

4.2. AI for Resource Allocation in UAV Network

The mobility as well as LOS links provided by UAVs make them a promising alternate to the fixed BS for the wireless communication networks. Similarly, AI has gained a lot of interest in wireless communication as it has the ability to learn from data and also learn from the environment and take decision herself. That is why, the research community is working on integrating intelligence in UAVs networks using different AI algorithms. In this section, we cover the potential applications of AI in UAV based wireless networks. These networks when extended, can work as a basic but fundamental ground for UAV based vehicular networks.
In [76], the authors handle the radio resource management in cellular based UAVs by combining resource block (RB) allocation and beam-forming design. To realize this, at terrestrial BSs an outer MDP characterizes the dynamic RB possession environment while the inner MDP is used to track the time-varying feature of B2D small-scale fading. A DRL-aided hybrid D3QN-TD3 algorithm is proposed to maximize the reduction of the UAV user equipment’s (DUE’s) ergodic outage duration(EOD) for outer and inner MDPs. In discrete domain, RB coordination is achieved using the deep double duelling Q network (D3QN). The beam forming in continuous time is achieved using twin delayed deep deterministic policy gradient (TD3).
In [77], the authors combined the features of FL and MARL, for location deployment and resource allocation and proposed a multi-agent collaborative environment learning (MACEL) algorithm to optimize the network utility by ensuring the user coverage. The UAVs act as agents which are trained using the Deep-Q-Network (DQN) algorithm. The FL model does the aggregation of local gradients and weights at cloud server to make a global model. The final results demonstrate the superiority of the proposed model in adapting to the ever changing environment.
In [78], to deploy UAVs-BS, the authors proposed regression neural networks (GRNN) and RF based models. For this purpose, the system implements the load prediction algorithm (LPA) to predict maco-cell congestion based on load history generated by the mobile network. Then, to calculate the accurate number of UAV-BS required, the UAV-BSs clustering and positioning algorithm (UCPA), is implemented. The results showed that the RF model achieved the accuracy of over 85%.
In [79] the authors integrate DNN in UAV at MEC for communication resource allocation, model optimization and UAV trajectory control to ensure the service latency minimization while ensuring the learning accuracy and energy consumption requirements.
In [80], the authors proposed the optimal UAV-RIS deployment mechanism based on dueling DQN to facilitate multiple DL users with the help of the MC-PD-NOMA scheme. The total transmit power is minimized. Moreover, UAVs’s trajectories are optimized jointly along with sub-carrier allocations, phase shifts at each RIS. The results show that more total transmit power is achieved using the proposed model when compared with other baseline methods.
In [81], the authors maximize the sum rate of the UAV-enabled multi-cast network by jointly designing the UAV movement, re-configurable intelligent surface (RIS) reflection matrix, and beam-forming design from the UAV to users based on a multi-pass deep Q Network (BT-MP-DQN). In the proposed model, the UAV is the agent and beam-forming control and trajectory design are considered system actions. The movement of the UAV is discrete action whereas the beam-forming design is continuous action. It is shown that the achievable rate is maximized as well as the minimum rate of multi-cast group users is achieved.
In [82], the multi agent DRL model is applied for resource management on cellular and IoT networks by integrating UAV in the system. First, the resource management related challenges in UAVs-BS based IoT networks are pointed out. The K-means algorithm and round robin scheduling algorithms are used for clustering and service request queue respectively. The accuracy, RMSE and testing time(s), are used as metrics to compare the proposed method with previous works. The proposed system achieves 94% resource management accuracy, RMSE score of 2.40% and testing times(s) levels of 1.05s. In terms of precision and recall estimation (as 88.05%, 93.04% and 92%, 90.01% for two of the clusters), the proposed model performs better than the Q-learning approach.
In [83], the paper deploys mobile UAVs as BSs. The aim of this research is to achieve data collection and energy transmission maximization as well as minimum UAV energy consumption. This could be achieved by UAV transmission power, hovering time slots, and flight decisions optimization. This multi-objective optimization problem is solved by implementing the MJDDPG dynamic resource allocation algorithm. In this study, the experimental results of the proposed model are not compared with any other models to proof the authenticity of the model.
In [84],the authors proposed a dynamic information exchange management schemes in a UAV network. First scheme shares the channel exchange information. Then, the dynamic time slot model is implemented in second scheme where priority based time slot sharing is performed. For both the schemes, to solve the problem of dynamic management, a DQN network coupled with LSTM is proposed. The LSTM network ensure the fast convergence of the DQN model.
In [85], the authors proposed the MDP and then an Actor-Critic based RL technique (A2C) solution in UAVs to offload the computational tasks of the ground users and achieve the minimum mission time. This scheme achieves the terrestrial users offloading through UAVs and in turn the offloading of UAVs through MEC server.
In [86], the authors probe the offloading of the task in UAV via MEC servers to minimize latency and the energy of the UAVs. Each UAV is associated with its corresponding task by keeping check of the available energy along with the optimal MEC server selection two Q-learning models are proposed. In [87], the Multi Agent RL (MARL) model is proposed for resource allocation for multiple UAVs network. The authors perform the sub-channel and user selection and allocate power to each user by considering parameters such as the signal-to-noise ratio, LOS and Non-LOS conditions. The proposed MARL algorithm achieves a good trade-off between the information exchange overhead and the system performance.
In [88], the authors consider a swarm of HAPS for communication and aim at comparing the RL and SI algorithms. The SI algorithm performs better as compared to RL as it converges quickly by maintaining the stable user coverage profile. On the other hand, the RL algorithm shows some convergence dips. However, the overall coverage rates are higher for RL.
In [89], the authors proposed FL based resource allocation method for joint optimization of the UAV position and local accuracy of FL model and user computation and communication resource. After formulating the joint problem, these three problems are solved by divided them into separate sub problems. The proposed algorithm is compared with the fixed altitude UAV-assisted FL ratio and performs better learning and reduces the system overall energy consumption.
In [90], authors perform joint power allocation and scheduling for a UAV swarm network. In the network one drone is selected as a leader and all other drones are make a group of drones following the leader. Every group transmits the update of its local FL model to the leader drone so it could combine all the local parameters for global parameter update to the global model. While the drones exchange updates, the wireless transmissions gets affected by many internal and external losses and interference. The impact of these factors on FL model is also considered. Moreover, experimental results approve the FL convergence analysis and the combined design strategy minimize the convergence time by 35% in comparison to the other state of the art designs.
In [91], the authors perform task offloading to manage the resources by ensuring the energy and latency minimization for high-altitude balloon networks. The user association is decided using SVM-based FL model. The FL provided the privacy as the data is not being shared across the network.
To summarize, the ML methods have been recently (the last two years) widely exploited in dynamic resource allocation in UAV and fixed BS communication. The supervised learning, RL and FL are studied for fast making decision and response, resource allocation, task offloading, joint power allocation in high-dynamic UAV networks. However, the implementation of AI models is limited to the basic and much simpler state of the art models. The researchers ignored the limitation of these models while implementing them to solve the problem in hand.

4.3. AI for Resource Allocation in UAV-IoV Network

The heterogeneity of vehicular networks, their high dynamic nature with fast moving have made the vehicular network more complex and demand new requirements for networking algorithms that could meet the stringent network control and resource allocation demands such as efficient spectrum sharing, transmission power maximization and computational resource management to minimize the energy requirements. UAV-IoV networks are three dimensional and contrast to terrestrial networks, the (UAV) base station itself is moving. Therefore,traditional optimization techniques are unable to capture complex patterns. In this section we review AI/ML based resource allocation in UAV-IoV networks. A brief summary of these papers is provided in Table 3.
Resource management in UAV-IoV is divided into radio resource allocation and computation resources management. The radio resource allocation is further divided into spectrum and channel access optimization. The main goal of radio resource management is to limit channel interference, power usage and network congestion. The computation resource management includes service, task and traffic offloading in MEC where the edge cloud nodes located in BSs and/or UAVs. This decentralization of the system generates faster response times as compared to the central deployments.
In [92], the authors focus on bandwidth allocation, location control deployment and trajectory of UAVs for communication capacity maximization to enable the UAV to process more data at edge computing. They propose an actor-critic mixing network (AC-Mix) and multi-attentive DDPG (MA2DDPG) network. The AC-Mix is the combination of Qmix (relies on Q function and don not deal with continuous values) mixed with the actor–critic framework. The critic in AC-Mix has a individual value network and a mixing network maintained by a virtual agent. Each agent inputs a state–action pair to GRU network which in turn outputs the global action value. TensorFlow 1.15.0 is used to design all algorithms. For comparison, user fairness, and load balance metrics are presented. MA2DDPG convergence velocity is 30.0% and 63.3%, better than the MADDPG algorithm.
In [93], the authors proposed a mechanism for energy harvesting by UAV from BS and vehicles using wireless power transfer (WPT) and simultaneous wireless information and power transfer (SWIPT) techniques, respectively. Maximum data offloading to the UAV is the main goal of this research. For optimized resource management and UAV velocity a DRL-based resource allocation and speed optimization (DRL-RASO) model is adopted. The proposed model achieves much higher offloading rates data offloaded to the UAV than the DRL-FTPA and Dueling-DQN. The authors named a simple DRL based DDPG algorithm DRL-RASO and no modification is suggested in the basic algorithm. The reported difference between DRL-RASO and DRL-FTPA is 5.79%.
In [94], an UAV based vehicular network is built to deal with caching and computing problem in addition to BS. The energy minimization is achieved by combining the cache refreshing optimization, computation unloading and status age updates. The online decision making is performed using DDPG. The BS decides if the cache needs to be refreshed, task has to be executed and what should be the bandwidth distribution. The total energy consumption is the reward function. The learning performance of proposed model is compared with traditional DDPG algorithm in terms of convergence rate. Then, the energy consumption for four bench marks namely random refreshing, random offloading, popular refreshing and equal bandwidth is calculated. It is claimed that proposed model outperforms DDPG in terms of system energy consumption and computational capabilities of UAV MEC server. But the authors do not report any results obtained using DDPG model.
In [95], authors formulated a combined auction-coalition building method to assign UAV coalitions to various IoV groups. The coalition formation game maximizes the combined profits of all UAVs. The auction-coalition creation method is suggested to achieve the UAV coalition stability. The reported simulation results show that the FL communication latency is decreased.
In [96], the authors proposed a FL based approach for the development of IoV based applications. The authors used the Gale-Shapley algorithm to match the lowest cost UAV to each sub-region. The simulation results show that the lowest marginal cost of node coverage for a UAV is assigned to each sub-region for task completion. The UAV energy harvesting is suggested as a future work for UAVs to continue their flight without any need of going back to their charging stations.
The authors in [97], proposed a secure bandwidth allocation scheme based on the game theory on the IoV assisted by UAVs. Also, for delay reduction and increased data privacy the proposed blockchain-based system introduces an emerging consensus mechanism. Furthermore, to allocate the limited safe bandwidth, based on the real-time feedback of each UAV an optimal decision search algorithm based on gradient descent to achieve Stackelberg equilibrium is proposed. The performance of the proposed scheme is compared with the many-to-one scheme, the maximum signal-strength-indicator (max-RSSI) scheme, the maximum signal-to-interference-plus-noise-ratio (max-SINR) scheme, the Auction-based UAV Swarm Many-to-Many scheme (AMMA) and UE-Optimal Many-to-One Matching scheme (UMOA). The proposed scheme achieves better throughput of about 95% as compared to other models but the authors do not provide any data to strengthen their claim about privacy and secured bandwidth allocation.
In [98], the vehicular task offloading optimization problem is dealt by jointly considering the resource allocation, and the security assurance. Then, this problem is divided into two separate problems and finally iterative algorithm called LBTO is proposed. LBTO decides if a certain MEC is selected depending on the load of the MECs and uses the Lagrangian dual decomposition for optimized offloading ratio and the computation resource. Task to be processed is selected based on the size of the task, computing resources required to execute a task, task’s allowed latency and ratio of the offloaded task at UAV MEC or locally to total task. For security assurance wiretap coding is used to protect the vehicle’s MEC. The proposed algorithm provides better task offloading ratio and delay than the other algorithms. However, this research considers that the UAVs are fixed and that is why completely ignores their energy consumption during mobility in the objective function.
In [99], the authors proposed a system wise computation capacity maximization of UAV based MEC for a group of vehicles equipped with WPT. They proposed a second-order convex approximation based method to solve a sequence of sub-problems. The platooning vehicles and UAVs are coupled.The drawback of this technique is that it does not use the AI/ML to solve the non-convex problem to give a comparative analysis.
In [100] the authors proposed a MDP based model for UAVs to take trajectory decisions to maximize the communication coverage area. The Actor-Critic algorithm learns the environment. The problem under consideration is mixed integer non-linear and non-convex problem. The DRL model learns the underlying non-linearity and non-convexity optimally. The model inputs are residual energy of each UAV, number and position of vehicles, positions of UAVs with respect to ground level etc. The UAV travelling distance is taken as the action. The penalty on the network incurred if UAV does not provide coverage to a vehicle, the deployment of a new UAV, the remaining energy of each UAV and the UAV goes outside the designated path. The proposed model is claimed to be compared with random dispatch, fixed dispatch and hovering models for metrics such as energy consumption, time versus maximum performance and vehicular density versus average convergence. But, there is no comparison of the proposed model provided with other state of the art models. Overall 40% of improvement is reported in converge area of the UAVs using the proposed model.
In [101], the authors combined auction-integration (AI) formations to integrate UAV into the groups of IoV elements with target of achieving the total revenue maximization of a single UAV. The proposed method is compared with the merge-and-split algorithm to find the UAV coalitions formation to the IoV groups, and random UAV coalition partitioning with second-price auction.
In [102], the authors proposed a FL-based technique to make sure security for IoV applications. The accurate reporting of UAV types is achieved by applying MD contracts with its identity qualities and using many sources of variability. The least costly UAV for each sub-region is found using the Gale-Shapley method. The results show the success of the proposed matching scheme.
In [103], the authors proposed a model free Q network to select the best UAV advice with least stalling time. The results show that the proposed system provide a high-quality user experience. This paper is limited in its scope as the limitation of UAV server capacity, speed and vehicle speed are not considered.
The research in the area of UAV based IoV networks is in its infancy. Many issues in the research are not addressed and ignored. The main focus of research is on system energy minimization, energy harvesting, caching and bandwidth allocation. Although in achieving these goals the researches ignore the latency requirements, changes in UAV height, the variations in vehicular node density. To manage resources, reinforcement learning is the most rampantly used AI technique. However, the studies used the state of the art RL models to minimize the system energy and bandwidth allocation etc. The lack of dataset availability in UAV based IoV networks makes it very hard to use ML and DL models for resource management and to explore the proven potential of these AI models. Furthermore, DL techniques have not been investigated because of the restricted power and processing resources available in UAV. In addition to this, the most important issue of security and data privacy are not considered in the research. The unencrypted and unauthenticated channels are used for UAV based communication which make them vulnerable to cyber attacks. Federated Learning can play a pivotal role in providing the security and privacy by training the ML model on data without transferring it to cloud server.

5. AI based Routing in UAV and IoV Networks

In this section, we cover the research contribution in the area of routing protocols proposed and designed for IoV, UAV as well as UAV based IoV networks using AI.

5.1. AI for Routing in IoV Network

In opportunistic networks, node selection poses a crucial challenge due to nodes lacking information about the state of other nodes. Furthermore, in IoV, traditional routing protocols fall short of achieving optimal performance. To address these challenges, the authors introduced a machine learning-based multi-copy routing algorithm called iPRoPHET (Improved PRoPHET) in their work [104]. iPRoPHET leverages dynamically changing contextual information of nodes and the delivery probability of PRoPHET for effective message transfer. Employing a random forest, iPRoPHET classifies nodes as reliable or non-reliable forwarders based on contextual information provided during each routing decision. The training data is derived from simulations. The proposed model undergoes evaluation using metrics such as delivery probability, hop count, overhead ratio, and latency, demonstrating performance on par with similar multi-copy routing algorithms.
In [105], the authors proposed a stochastic chaos based adaptive routing with prediction (SCARP) to predict traffic flow using DL networks to suggest a node discovery routing principle. The connectivity loss and delay are minimized, and guarantee a secure data transmission between vehicles. In this research, region of Puducherry U.T, India is selected for traffic data collection. The simulation software namely SUMO and OMNET++ are used to create traffic and network scenarios respectively. The metrics namely accuracy, PDR, delay and sensitivity are used to compare the proposed method with existing state of the art routing algorithms.
In [106] a Q- learning based geographical routing scheme with intersection-based V2X routing (IV2XQ) is introduced. First the best road segment at the intersections for routing is selected using Q- learning at intersections. Then best relay node selection is performed using greedy routing strategy. The central server is the agent which uses the historical traffic data to select the optimal path. The environment which is our entire network, rewards the agent if it takes the right action and choose the correct road segment to forward a data packet. It is reported that the proposed scheme increased the PDR and considerably controlled the network congestion.
In [107], a RL based routing (best two hops) and context aware edge node selection scheme to forward packets scheme (CEPF) is proposed. Both unicast and broadcast communications are supported by CEPF. This routing protocol reduces the forward nodes and increases the resource efficiency. The decentralized fuzzy logic is implemented to select the edge nodes based on vehicular velocity, mobile nodes traveling in same direction, and communication link condition. The edge node is the vehicle with highest node score. For rout discovery operation RL is used in which each packet is the agent and the action agent takes is the selection of the next-hop node.
In [108], authors integrate the RL and fuzzy logic and proposed a reinforcement routing protocol named RRPV. A DynaQ technique is implemented on the fuzzy logic to build the model. The link stability and connection quality are two inputs for fuzzy logic-based system. The fuzzy system determines the link quality and this result is fed as the state transition probability in MDP. In the RL process, vehicles are agents and each agent has two states, namely F to send a packet and D to deliver packets to adjacent vehicle. Hello message delivery to the neighboring vehicle is the action of the vehicles. Moreover, the link condition and the euclidean distance of two neighboring vehicles define reward function. The routing table of each vehicle consists of Q-values and V-values.
In [109], the authors introduced traffic-aware routing protocol based on Q-learning (QTAR). It contains two routing algorithms based on Q learning to send data packets between vehicles (V2V Q learning) and between RSUs (R2R Q learning). Vehicles broadcast HelloV2V messages containing their velocity and location related information and the RSUs exchange HelloR2R messages with each other.
In [110] a routing protocol named RLRC for clustered network based on the K-harmonic means (KHM) clustering to assign vehicles to different cluster and RL to exchange data between two cluster heads (CHs) is proposed. In RLRC, hello message is used to share the vehicle velocity and position with neighbouring vehicles. In this process, each node behaves as an agent, state set is defined as neighboring CHs, and a next-hop CH selection is the action taken by the agents. The reward function is based on the link quality parameter.
In [111], the authors proposed a Q-learning based routing scheme called reliable self-adaptive routing scheme (RSAR). The vehicles are agents. The action is a beacon message including vehicle speed, location and Q value, sent to the next vehicle. Moreover, the decentralized learning process is adopted with the number of hops, bandwidth, and link reliability as learning parameters. The RSAR finds the most fit relay vehicular node and solve network segmentation problem.
In [112], the authors proposed a routing technique which allows the central server and vehicles and RSUs network nodes to access the updated traffic information based on intersection- based Q-learning (IRQ). The global traffic view is obtained by IRQ as well as by the central server to form a routing solution. Here, the central server behaves as an agent. The central server is also responsible for network congestion control in the routes. The IRQ uses a greedy routing approach in V2V and V2R routing decision where for V2V routing, the vehicle closest to the target is chosen to forward the data packet and in V2I scenario, the RSU located at intersection delivers data packets to the corresponding road section. If there is no vehicle available, RSU waits and holds the packet until it finds a vehicle to rely information.
In [113], a routing protocol based on Q-learning and fuzzy-based hierarchical (QFHR) is proposed. The routing algorithm is capable of doing traffic pattern recognition, routing between intersections, and at road sections. The RSUs are equipped with Q learning to find multiple routing paths. Moreover, the vehicles use the greedy technique to find the most fitted path in each road section. A fuzzy solution works as the alternate for route recovery in case the main algorithm fails and selects the next node.
To summarize, The integration of routing protocols with AI is mostly based on RL models. Although the research results show that RL is quite successful in taking the routing decisions yet it comes with certain limitations. The large state and action sets effect the convergence rate and increase delay in the routing process. Therefore, in future, the researchers must focus filtering state and action spaces based on the required criteria. Most of the research is using Q table and as state and action set increase, the dimension of the Q table increases which requires more memory to store it and in turn increase the latency demand of the system. So, the management of the Q-table should be considered in future research.

5.2. AI for Routing in UAV Network

In [114], the management of multiple cooperative UAVs is addressed. The routing problem in this system is divided into two stages: initial planning and routing solution. In the initial planning stage, regions to be visited are grouped into clusters based on the distance criterion (FCM algorithm), with each cluster assigned to a UAV. The route solving stage determines the best route for each agent, considering the clusters from the initial planning stage and a variant of the Orienteering Problem. The transformer deep learning architecture is employed to solve the Orienteering Problem with shared regions, coupled with a DRL framework. The proposed model is evaluated using multiple OP-MP-TN datasets under various environmental conditions, demonstrating its superiority over state-of-the-art models in cooperative and non-cooperative scenarios.
In the work presented by [115], UAV location optimization and relay path planning are jointly achieved using a graph neural network based on Reinforcement Learning (RGNN) algorithm. The proposed model exhibits significantly lower time complexity compared to traditional optimization methods. The Location GNN (LGNN) optimizes UAV locations, and the RGNN selects the optimal relay path based on information provided by the LGNN. The method outperforms brute-force approaches in terms of time complexity.
The study conducted by [116] introduces a novel routing protocol based on ant behavior routing, enhancing end-to-end security through data encryption using the Pheromone update process. Experiments conducted in Network Simulator-2 show that AntHocNet performs well in terms of packet drop rate, throughput, and bandwidth utilization, achieving significant optimizations compared to other routing techniques.
In [117], collision-free routing policies for UAVs are designed using multi-agent Reinforcement Learning (RL). The authors propose a multi-resolution, multi-agent, mean-field RL algorithm named 3M-RL for UAV flight planning. Each UAV makes decisions based on local observations without direct communication with other UAVs. The routing policy is trained using an Actor-Critic neural network with multi-resolution observations, demonstrating effectiveness in various complex scenarios in both 2D and 3D space.
The predictive ad-hoc routing combined with Reinforcement Learning and trajectory knowledge protocol (PARRoT) is introduced by [118]. This protocol aims to achieve lower latency and high robustness by predicting future node positions and sharing information with adjacent nodes. PARRoT separates networking from path planning, enhancing overall system efficiency.
In [119], fuzzy logic is employed to identify adjacent nodes in real-time, while Reinforcement Learning is used to reduce the number of hops in a routing algorithm named Fuzzy Logic Reinforcement Learning-based Routing Algorithm (FLRLR). FLRLR reduces the average number of simulation hops and ensures higher link connectivity, showing comparative advantages over the ant colony optimization (ACO) algorithm.
The adaptive and reliable routing protocol called ARdeep is proposed in [120]. This deep learning-based protocol autonomously distinguishes network variations using a Markov Decision Process (MDP) model. Factors such as Packet Error Rate (PER), link status, connection time, and nodes’ remaining energy influence routing decisions, and ARdeep outperforms QGeo in experimental comparisons.
The study by [121] introduces the Q-Learning-based Fuzzy Logic for Multi-Objective Routing Algorithm in Flying Ad-Hoc Networks (QLFLMOR). QLFLMOR uses Q-learning and fuzzy logic in UAVs to select the optimal routing path based on link and path-level parameters. Experimental results demonstrate that QLFLMOR achieves lower hop count and energy consumption compared to other routing algorithms.
To summarize, most routing protocols assumed that the UAV networks are fully connected but it is not the case in reality and the broken links cause failure of routing protocols. In the conventional routing protocols, the node mobility is designed for 2D spaces whereas the UAV moves in 3D space and in most of the researches UAV mobility is converted to 2D scenarios. In future, RL-based method should be modified and implemented to accommodate nonlinear UAV movement and implement light-weight routing protocols for small sized UAVs that are mostly in use in real world.

5.3. AI for Routing in UAV-IoV Network

Efficient data dissemination among vehicles and optimization of multi-hop path and relay selection is a complex task in IoV. The network latency and reliability considering the increasing vehicle density in future networks is very crucial in making routing decisions. In this regard, the UAV based routing in IoV is relatively new area and not properly explored yet. The summary of AI/ML UAV-IoV routing protocols is provided in Table 4.
In [122], traffic congestion problem is dealt using a Q-learning based load balancing routing (Q-LBR). It estimated the network load using low-overhead technique, Q-learning based load balancing and finally implements a reward control function for Q-learning convergence. The simulation results show that Q-LBR achieves better PDR, network utilization and latency with 8%, 28% and 30% improvement, respectively.
In [123], VRU algorithm is proposed to route data for vehicles using UAVs and intra-UAV data routing. The NS-2.35 simulator under Linux Ubuntu 12.04 is used to generate urban scenario to evaluate the VRU routing performance. VanetMobiSim and MobiSim generates mobility of vehicles and UAVs, respectively. results show that VRU improved the PDR by 16%, detection ratio by 7% decreases end-to-end delay by an average of 13% and overhead by 40% compared to other reviewed routing protocol.
In [124], an adaptive UAV assisted geographic routing with Q-Learning (QAGR) is proposed. Routing is done using two different components namely aerial and base components. UAVs uses the combination of fuzzy-logic and depth-first-search (DFS) algorithm to find the global routing path. This routing path information is transferred to the requesting vehicle at ground. A fix-sized Q-table is maintained at vehicle which is updated with the global routing path.
In [125], the relay selection problem is addressed for UAV based VANET. The authors calculate the communication interruption probabilities, energy consumption and delay consumption. Q-learning is being used for relay selection. The simulation results exhibit the performance improvement in terms of delivery ratio, delivery latency, and signaling overhead as compared to five other schemes, namely, STPTC-VANETs, STP-A2G VANETs, TC-A2G VANETs, CORF-A2G VANETs, and UMTAP-A2G VANETs.
In [126], a routing solution employing a flooding technique is adopted to enhance data delivery reliability and ensure robust paths for vehicles through ad hoc collaboration with UAVs. The evaluation of the proposed model is conducted using the NS-2 simulation tool, with vehicle mobility simulated using the Simulation of Urban Mobility (SUMO) software. For the UAVs, a random walk mobility pattern is applied to 10 UAVs, with a maximum altitude of 300 meters during flight. Notably, this paper does not incorporate machine learning techniques and relies solely on conventional routing methods. Evaluation metrics encompass PDR, end-to-end delay (EED), overhead, and average number of hops. It’s worth mentioning that the authors do not compare their proposed technique with other routing methods. Nevertheless, the suggested protocol demonstrates a PDR that is 90% superior to RBVT-R and OLSR.
In the research in UAV based IoV routing protocols the control messages are exchanged periodically and the flooding routing messages consume excessive bandwidth resulting in high overhead. To address this issue, broadcast control mechanisms offer the flexibility to dynamically adjust the broadcast time interval of control messages. Additionally, by selectively filtering nodes based on factors such as link quality and movement information, it becomes possible to regulate the flooding process of routing messages. In addition to this, most of the routing protocols designed using RL model are considering QoS requirements. The researcher should focus on including other objectives such as link quality and delay and include them in reward function.

6. Challenges and Open Issues with Future Directions

This section discusses limitations of ML/AI algorithms and the simulation software used to train ML algorithms.

6.1. Major Limitations and Challenges of AI/ML

Numerous AI/ML based IoV resource management and routing algorithms are proposed in the last five years. Although AI/ML approaches are data driven and can produce fairly accurate solutions in most scenarios, there a number of limitations with these approaches as well. Here are some of the key limitations associated with ML and DL:
  • Application Specific ML based models are application specific. It means if we train a DL model on certain vehicular application such as network data congestion prediction or classification, the model would be able to provide high quality results in same area, but it would not predict or classify the vehicular traffic congestion in a different area.
  • Noisy and Incomplete Data: ML agents may have to deal with noisy and incomplete data, which can impact their learning and decision-making processes.
Furthermore, the ML and DL models are data dependent and their success lies in the availability of data. Especially, most DL algorithms need adequate data. However, in UAV based vehicular communication the historical data for many time sensitive tasks such as resource management, mobility prediction, routing decisions is not readily available. Therefore, there is a need to pay more attention to provide open source and highly reliable data related to UAV vehicles. It means we need to have a mechanism to produce and estimate the accurate data-set size to train and test the ML and DL algorithms.
This is the reason that most of the research used RL for offloading the task and routing. RL and it’s variants are proved to be very effective in handling the non-convex problems such as task management, energy efficiency and routing. However, in RL, the agent’s action depends on the reward or penalty it gets which in turn could come handy in training the model. Specifically, in routing related problems, as the state and action sets become large, it affects the convergence and increase the latency and increase the dimentionality of the Q tables which results in more memory consumption.
Recently, FL has emerged as a reliable solution a because it provides data privacy and reduces the time complexity. Unfortunately, the FL could be subject to backdoor attacks where the integrity of the model can be threaten by injecting the poisoning data/model. In addition to this, convergence of the FL model is another issue as convergence is problem specific such as loss function convexity and the number of times a model has to be updated. So, in case there is no sufficient data, the model will fail to give accurate results. Moreover, the UAV-IoV network is heterogeneous consists of drones with different sizes and specifications, and the vehicles with dynamic computational and processing capabilities and varying GPUs. When FL is implemented in such a heterogeneous network, all drones and vehicles will have different response times. In the FL, the model updates at each communication round and the delays introduced at this time causes the slow convergence of the model.
To conclude the discussion, although AI/ML has some drawbacks, yet the ML, RL and FL based solutions for UAV based IoV networks have produced better results for resource management and routing when compared with other techniques to handle the non-convex vehicular and UAV based network problems.

6.2. Testbed and Datasets

During this survey paper, it is noted that most of the research is done using network simulator version (NS2), NS3, MATLAB, SUMO and OMNET++ Vissim, AVISPA and RaspberryPi to create vehicular traffic scenarios and network analysis. The accuracy of these tools is questionable and the performance evaluation of these methods is not comparable with the real world scenarios. The real strengths and weaknesses of these algorithms can be estimated when they are implemented in real world scenarios. As mentioned in the section above, the research community should focus on providing the reliable wireless and cellular data to train the AI models on real world data and scenarios.

7. Conclusion

UAV based aerial networks add a third spacial dimension to wireless networks. UAVs are also special because they can serve as mobile base stations. Their mobility will add more versatility as well as more complexity to vehicular networks. Therefore, AI/ML will play a much important role in UAV based IoV.
In this paper, we present an extensive comparative study on the application of AI algorithms in the UAV based IoV paradigms. We have reviewed resource management and routing techniques in UAV based IoV using different AI models.
AI-based algorithms for UAV-IoV applications have enhanced the system performance as compared to the traditional algorithms. When multiple AI algorithms are combined together to utilised their best features, it brought out an almost optimum solution for resource management and routing in UAV-IoV networks. Also, the overall system throughput improved with energy minimization and higher latency. However, the high computational resource utilization by AI algorithms in a dynamic vehicular and UAV environment is the major issue. For this reason, the emerging MEC and VEC architectures often attempt to offload the computing tasks to a RSUs or to Fog nodes made it possible to have these resources out of the vehicles and UAVs to share computation burden of AI with edge computation servers. IoV has already shown a great success in optimizing the network operations. nevertheless, the inclusion of UAVs has made the network more dynamic and mobile. AI/ML has the potential to handle such a dynamic and mobile network.
Based on the papers we surveyed, it can be incurred that in future UAV based IoV, empowered by AI/ML can autonomously operate without any dependence on third party to perform configuration and optimization.

Author Contributions

Conceptualization, S.A.A.S, X.F; data curation, S.A.A.S; formal analysis, S.A.A.S; funding acquisition, X.F, investigation, S.A.A.S; project administration, X.F, R.K; supervision, X.F, R.K; validation, X.F, R.K; visualization,S.A.A.S; writing- original draft, S.A.A.S; writing- review and editing, S.A.A.S., X.F. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by Toronto Metropolitan University under the Communications Lab supervised by Department of Electrical, Computer and biomedical Engineering, Toronto, Ontario, Canada.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Hashemi, S., and Zarei, M. (2021). "Internet of Things backdoors: resource management issues, security challenges, and detection methods". Transactions on Emerging Telecommunications Technologies, 32, e4142.
  2. Gillis, Alexander (2021). "What is internet of things (IoT)?". IOT Agenda. Retrieved 03 August 2023.
  3. Tang, C., Wei, X., Liu, C., Jiang, H., Wu, H., Li, Q. (2020). UAV-Enabled Social Internet of Vehicles: Roles, Security Issues and Use Cases. In: Xiang, Y., Liu, Z., Li, J. (eds) Security and Privacy in Social Networks and Big Data. SocialSec 2020. Communications in Computer and Information Science, vol 1298. Springer, Singapore.
  4. Jamalzadeh, Melody, Mohsen Maadani, and Mojdeh Mahdavi. "EC-MOPSO: an edge computing-assisted hybrid cluster and MOPSO-based routing protocol for the Internet of Vehicles." Annals of Telecommunications 77.7-8 (2022): 491-503. [CrossRef]
  5. Krishna, M. "A Survey UAV-Assisted VANET Routing Protocol." International Journal of Computer Science Trends and Technology (IJCST)–Vol 8 (2020).
  6. Guerna, A.; Bitam, S.; Calafate, C.T. "Roadside Unit Deployment in Internet of Vehicles Systems: A Survey". Sensors 2022, 22, 3190. [CrossRef]
  7. Ghazal, T.M.; Hasan, M.K.; Alshurideh, M.T.; Alzoubi, H.M.; Ahmad, M.; Akbar, S.S.; Al Kurdi, B.; Akour, I.A. IoT for Smart Cities: Machine Learning Approaches in Smart Healthcare—A Review. Future Internet 2021, 13, 218. [CrossRef]
  8. S. Yaqoob, A. Ullah, M. Awais, I. Katib, A. Albeshri, R. Mehmood , et al . 2021. Novel congestion avoidance scheme for Internet of Drones. Computer Communications 169 (2021), 202–210. [CrossRef]
  9. Nina Mazyavkina, Sergey Sviridov, Sergei Ivanov, Evgeny Burnaev, Reinforcement learning for combinatorial optimization: A survey, Computers and Operations Research, Volume 134, 2021. [CrossRef]
  10. Saravanan, M.; Ganeshkumar, P. Routing using reinforcement learning in vehicular ad hoc networks. Comput. Intell. 2020, 36, 682–697. [CrossRef]
  11. Sun, Y., Lin, Y., Tang, Y. (2019). A Reinforcement Learning-Based Routing Protocol in VANETs. In: Liang, Q., Mu, J., Jia, M., Wang, W., Feng, X., Zhang, B. (eds) Communications, Signal Processing, and Systems. CSPS 2017. Lecture Notes in Electrical Engineering, vol 463.
  12. L. Liang, H. Ye and G. Y. Li, "Toward Intelligent Vehicular Networks: A Machine Learning Framework," in IEEE Internet of Things Journal, vol. 6, no. 1, pp. 124-135, Feb. 2019. [CrossRef]
  13. W. Tong, A. Hussain, W. X. Bo and S. Maharjan, "Artificial Intelligence for Vehicle-to-Everything: A Survey," in IEEE Access, vol. 7, pp. 10823-10843, 2019. [CrossRef]
  14. F. Tang, Y. Kawamoto, N. Kato and J. Liu, "Future Intelligent and Secure Vehicular Network Toward 6G: Machine-Learning Approaches," in Proceedings of the IEEE, vol. 108, no. 2, pp. 292-307, Feb. 2020. [CrossRef]
  15. F. Tang, B. Mao, N. Kato and G. Gui, "Comprehensive Survey on Machine Learning in Vehicular Network: Technology, Applications and Challenges," in IEEE Communications Surveys and Tutorials, vol. 23, no. 3, pp. 2027-2057, thirdquarter 2021. [CrossRef]
  16. M. A. Hossain, R. M. Noor, K. -L. A. Yau, S. R. Azzuhri, M. R. Z’aba and I. Ahmedy, "Comprehensive Survey of Machine Learning Approaches in Cognitive Radio-Based Vehicular Ad Hoc Networks," in IEEE Access, vol. 8, pp. 78054-78108, 2020. [CrossRef]
  17. Z. Du, C. Wu, T. Yoshinaga, K. -L. A. Yau, Y. Ji and J. Li, "Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues," in IEEE Open Journal of the Computer Society, vol. 1, pp. 45-61, 2020. [CrossRef]
  18. Elmustafa Sayed Ali, Mohammad Kamrul Hasan, Rosilah Hassan, Rashid A. Saeed, Mona Bakri Hassan, Shayla Islam, Nazmus Shaker Nafi, Savitri Bevinakoppa, "Machine Learning Technologies for Secure Vehicular Communication in Internet of Vehicles: Recent Advances and Applications", Security and Communication Networks, vol. 2021, Article ID 8868355, 23 pages, 2021. [CrossRef]
  19. Nurcahyani, I.; Lee, J.W. Role of Machine Learning in Resource Allocation Strategy over Vehicular Networks: A Survey. Sensors 2021, 21, 6542. [CrossRef]
  20. Abdelkader Mekrache, Abbas Bradai, Emmanuel Moulay, Samir Dawaliby, Deep reinforcement learning techniques for vehicular networks: Recent advances and future trends towards 6G, Vehicular Communications, Volume 33, 2022. [CrossRef]
  21. Gillani, Maryam, Hafiz Adnan Niaz, and Muhammad Tayyab. "Role of machine learning in WSN and VANETs." International Journal of Electrical and Computer Engineering Research 1.1 (2021): 15-20.
  22. Abir Mchergui, Tarek Moulahi, Sherali Zeadally, Survey on Artificial Intelligence (AI) techniques for Vehicular Ad-hoc Networks (VANETs), Vehicular Communications, Volume 34, 2022. [CrossRef]
  23. M. Noor-A-Rahim, Z. Liu, H. Lee, G. G. M. N. Ali, D. Pesch and P. Xiao, "A Survey on Resource Allocation in Vehicular Networks," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 2, pp. 701-721, Feb. 2022. [CrossRef]
  24. Lansky, J.; Rahmani, A.M.; Hosseinzadeh, M. Reinforcement Learning-Based Routing Protocols in Vehicular Ad Hoc Networks for Intelligent Transport System (ITS): A Survey. Mathematics 2022, 10, 4673. [CrossRef]
  25. Javed, A.R.; Hassan, M.A.; Shahzad, F.; Ahmed, W.; Singh, S.; Baker, T.; Gadekallu, T.R. Integration of Blockchain Technology and Federated Learning in Vehicular (IoT) Networks: A Comprehensive Survey. Sensors 2022, 22, 4394. [CrossRef]
  26. Maria Christopoulou, Sokratis Barmpounakis, Harilaos Koumaras, Alexandros Kaloxylos, Artificial Intelligence and Machine Learning as key enablers for V2X communications: A comprehensive survey, Vehicular Communications, Volume 39, 2023. [CrossRef]
  27. Atefeh Hemmati, Mani Zarei, Alireza Souri, UAV-based Internet of Vehicles: A systematic literature review, Intelligent Systems with Applications, Volume 18, 2023. [CrossRef]
  28. Arash Heidari, Nima Jafari Navimipour, Mehmet Unal, and Guodao Zhang. 2023. Machine Learning Applications in Internet-of-Drones: Systematic Review, Recent Deployments, and Open Issues. ACM Comput. Surv. 55, 12, Article 247 (December 2023), 45 pages. [CrossRef]
  29. "IEEE Draft Guide for Wireless Access in Vehicular Environments (WAVE) - Architecture," in IEEE P1609.0/D9, July 2017 , vol., no., pp.1-104, 14 July 2017.
  30. "IEEE Standard for Information Technology-Telecommunications and information exchange between systems-Local and Metropolitan networks-specific requirements-Part II: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) specifications: Amendment 9: Interworking with External Networks," in Amendment to IEEE Std 802.11-2007 as amended by IEEE Std 802.11k-2008, IEEE Std 802.11r-2008, IEEE Std 802.11y-2008, IEEE Std 802.11w-2009, IEEE Std 802.11n-2009, IEEE Std 802.11p-2010, IEEE Std 802.11z-2010, and IEEE Std 802.11v-2011 , vol., no., pp.1-208, 25 Feb. 2011. [CrossRef]
  31. J. Li, M. Shi, J. Li and D. Yao, "Media Access Process Modeling of LTE-V-Direct Communication Based on Markov Chain," 2018 IEEE Intelligent Vehicles Symposium (IV), Changshu, China, 2018, pp. 61-66. [CrossRef]
  32. L. U. Khan, “Visible light communication: Applications, architecture, standardization and research challenges,” Digit. Commun. Netw., vol. 3, no. 2, pp. 78-88, May 2017.
  33. N. Cen, J. Jagannath, S. Moretti, Z. Guan, and T. Melodia, "LANET:Visible-light ad hoc networks," Ad Hoc Netw., vol. 84, pp. 107-123, Mar. 2019. [CrossRef]
  34. G. Araniti, C. Campolo, M. Condoluci, A. Iera, and A. Molinaro, “LTE for vehicular networking: A survey,” IEEE Commun. Mag., vol. 51, no. 5, pp. 148-157, May 2013. [CrossRef]
  35. J. Choi, V. Va, N. Gonzalez-Prelcic, R. Daniels, C. R. Bhat, and R. W. Heath, Jr., “Millimeter-wave vehicular communication to support massive automotive sensing,” IEEE Commun. Mag., vol. 54, no. 12, pp. 160-167, Dec. 2016. [CrossRef]
  36. V. Va, T. Shimizu, G. Bansal, and R. W. Heath, Jr., Millimeter Wave Vehicular Communications: A Survey. Hanover, MA, USA: Now, 2016.
  37. Apostolos (Tolis) Papathanassiou and Alexey Khoryaev, "Cellular V2X as the essential enabler of superior global connected transportation services,” IEEE 5G Tech Focus, vol. 1, no. 2, pp. 1-2, Jun. 2017.
  38. PC5. Initial Cellular V2X Standard Completed. Accessed: Aug. 08, 2023. [Online]. Available: https://www.3gpp.org/news-events/3gpp-news/v2x-r14.
  39. S. Husain, A. Kunz, A. Prasad, E. Pateromichelakis, K. Samdanis and J. Song, "The Road to 5G V2X: Ultra-High Reliable Communications," 2018 IEEE Conference on Standards for Communications and Networking (CSCN), Paris, France, 2018, pp. 1-6. [CrossRef]
  40. D. P. Moya Osorio et al., "Towards 6G-Enabled Internet of Vehicles: Security and Privacy," in IEEE Open Journal of the Communications Society, vol. 3, pp. 82-105, 2022. [CrossRef]
  41. Commission delegated regulation (EU) 2019/945, 2019, Official Journal of the European Union, 12th, Mar.2019, URL https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX%3A32019R0945.
  42. Altawy, Riham, and Amr M. Youssef. "Security, privacy, and safety aspects of civilian drones: A survey." ACM Transactions on Cyber-Physical Systems 1.2 (2016): 1-25.
  43. Villa, T.F.; Salimi, F.; Morton, K.; Morawska, L.; Gonzalez, F. Development and Validation of a UAV Based System for Air Pollution Measurements. Sensors 2016, 16, 2202. [CrossRef]
  44. Vasylenko M. P. "Telemetry System of Unmanned Aerial Vehicles" Electronics and control systems. (2018) 95-100.
  45. Chao, H., Cao, Y. and Chen, Y. Autopilots for small unmanned aerial vehicles: A survey. Int. J. Control Autom. Syst. 8, 36–44 (2010). [CrossRef]
  46. F. Hoflinger, J. Muller, R. Zhang, L. M. Reindl and W. Burgard, "A Wireless Micro Inertial Measurement Unit (IMU)," in IEEE Transactions on Instrumentation and Measurement, vol. 62, no. 9, pp. 2583-2595, Sept. 2013. [CrossRef]
  47. Yalin Liu, Hong-Ning Dai, Qubeijian Wang, Mahendra K. Shukla, Muhammad Imran, Unmanned aerial vehicle for internet of everything: Opportunities and challenges, Computer Communications, Volume 155, 2020, Pages 66-83, ISSN 0140-3664. [CrossRef]
  48. Amira Chriki, Haifa Touati, Hichem Snoussi, Farouk Kamoun, FANET: Communication, mobility models and security issues, Computer Networks, Volume 163, 2019, 106877, ISSN 1389-1286, (https://www.sciencedirect.com/science/article/pii/S1389128618309034). [CrossRef]
  49. Sharma, S., and Kaushik, B. (2019). A survey on internet of vehicles: Applications, security issues and solutions. Vehicular Communications, 20, Article 100182.
  50. Chaurasia, R., and Mohindru, V. (2021). Unmanned Aerial Vehicle (UAV): A comprehensive survey. Unmanned aerial vehicles for Internet of Things (IoT) (pp.1–27).
  51. Ad Hoc Network, NIST. https://csrc.nist.gov/glossary.
  52. Cao, H., Garg, S., Kaddoum, G., Hassan, M. M., and AlQahtani, S. A. (2022)" Intelligent virtual resource allocation of QoS-guaranteed slices in B5G-Enabled VANETs for intelligent transportation systems"IEEE Transactions on Intelligent Transportation Systems, 23(10), 19704-19713.
  53. L. Liang, H. Ye and G. Y. Li, "Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning," in IEEE Journal on Selected Areas in Communications, vol. 37, no. 10, pp. 2282-2292, Oct. 2019. [CrossRef]
  54. Saleh, A. H., and Anpalagan, A. (2022, May). AI Empowered Computing Resource Allocation in Vehicular Ad-hoc NETworks. In 2022 7th International Conference on Business and Industrial Research (ICBIR) (pp. 221-226). IEEE.
  55. Pan, Q., Wu, J., Nebhen, J., Bashir, A. K., Su, Y., and Li, J. (2022)." Artificial intelligence-based energy efficient communication system for intelligent reflecting surface-driven vanets". IEEE Transactions on Intelligent Transportation Systems, 23(10), 19714-19726. [CrossRef]
  56. Ibrar, M., Akbar, A., Jan, S. R. U., Jan, M. A., Wang, L., Song, H., and Shah, N. (2020). "Artnet: Ai-based resource allocation and task offloading in a reconfigurable internet of vehicular networks".IEEE Transactions on Network Science and Engineering, 9(1), 67-77. [CrossRef]
  57. Qafzezi, E., Bylykbashi, K., Ampririt, P., Ikeda, M., Matsuo, K., and Barolli, L. (2022). "An intelligent approach for cloud-fog-edge computing SDN-VANETs based on fuzzy logic: effect of different parameters on coordination and management of resources".Sensors, 22(3), 878. [CrossRef]
  58. Haris, Muhammad, Munam Ali Shah, and Carsten Maple. "Internet of intelligent vehicles (IoIV): an intelligent VANET based computing via predictive modeling." IEEE Access (2023). [CrossRef]
  59. Xing, Yang, Chen Lv, and Dongpu Cao. "Personalized vehicle trajectory prediction based on joint time-series modeling for connected vehicles." IEEE Transactions on Vehicular Technology 69.2 (2019): 1341-1352. [CrossRef]
  60. L. Hou, L. Lei, K. Zheng and X. Wang, "A Q -Learning-Based Proactive Caching Strategy for Non-Safety Related Services in Vehicular Networks," in IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4512-4520, June 2019. [CrossRef]
  61. S. -S. Lee and S. Lee, "Resource Allocation for Vehicular Fog Computing Using Reinforcement Learning Combined With Heuristic Information," in IEEE Internet of Things Journal, vol. 7, no. 10, pp. 10450-10464, Oct. 2020. [CrossRef]
  62. Y. Xia, L. Wu, Z. Wang, X. Zheng and J. Jin, "Cluster-Enabled Cooperative Scheduling Based on Reinforcement Learning for High-Mobility Vehicular Networks," in IEEE Transactions on Vehicular Technology, vol. 69, no. 11, pp. 12664-12678, Nov. 2020. [CrossRef]
  63. X. Zhang, M. Peng, S. Yan and Y. Sun, "Deep-Reinforcement-Learning-Based Mode Selection and Resource Allocation for Cellular V2X Communications," in IEEE Internet of Things Journal, vol. 7, no. 7, pp. 6380-6391, July 2020. [CrossRef]
  64. Q. Yan, B. -J. Hu and Q. Wen, "Joint Resource Allocation and Power Control for V2V Communication of High-density Vehicle Network," 2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring), Helsinki, Finland, 2021, pp. 1-6. [CrossRef]
  65. S. Song, C. Lee, H. Cho, G. Lim and J. -M. Chung, "Clustered Virtualized Network Functions Resource Allocation based on Context-Aware Grouping in 5G Edge Networks," in IEEE Transactions on Mobile Computing, vol. 19, no. 5, pp. 1072-1083, 1 May 2020. [CrossRef]
  66. H. Khan, S. Samarakoon and M. Bennis, "Enhancing Video Streaming in Vehicular Networks via Resource Slicing," in IEEE Transactions on Vehicular Technology, vol. 69, no. 4, pp. 3513-3522, April 2020. [CrossRef]
  67. B. Cao, Z. Sun, J. Zhang and Y. Gu, "Resource Allocation in 5G IoV Architecture Based on SDN and Fog-Cloud Computing," in IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 6, pp. 3832-3840, June 2021. [CrossRef]
  68. Y. Cui, Y. Liang and R. Wang, "Resource Allocation Algorithm With Multi-Platform Intelligent Offloading in D2D-Enabled Vehicular Networks," in IEEE Access, vol. 7, pp. 21246-21253, 2019. [CrossRef]
  69. Z. Lyu, Y. Wang, M. Liu and Y. Chen, "Service-Driven Resource Management in Vehicular Networks Based on Deep Reinforcement Learning," 2020 IEEE 31st Annual International Symposium on Personal, Indoor and Mobile Radio Communications, London, UK, 2020, pp. 1-6. [CrossRef]
  70. Y. -H. Xu, C. -C. Yang, M. Hua and W. Zhou, "Deep Deterministic Policy Gradient (DDPG)-Based Resource Allocation Scheme for NOMA Vehicular Communications," in IEEE Access, vol. 8, pp. 18797-18807, 2020. [CrossRef]
  71. H. Ye, G. Y. Li and B. -H. F. Juang, "Deep Reinforcement Learning Based Resource Allocation for V2V Communications," in IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3163-3173, April 2019. [CrossRef]
  72. S. Khan Tayyaba et al., "5G Vehicular Network Resource Management for Improving Radio Access Through Machine Learning," in IEEE Access, vol. 8, pp. 6792-6800, 2020. [CrossRef]
  73. A. Muhammad, T. A. Khan, K. Abbass and W. -C. Song, "An End-to-end Intelligent Network Resource Allocation in IoV: A Machine Learning Approach," 2020 IEEE 92nd Vehicular Technology Conference (VTC2020-Fall), Victoria, BC, Canada, 2020, pp. 1-5. [CrossRef]
  74. H. Khan, M. M. Butt, S. Samarakoon, P. Sehier and M. Bennis, "Deep Learning Assisted CSI Estimation for Joint URLLC and eMBB Resource Allocation," 2020 IEEE International Conference on Communications Workshops (ICC Workshops), Dublin, Ireland, 2020, pp. 1-6. [CrossRef]
  75. J. Gao, M. R. A. Khandaker, F. Tariq, K. -K. Wong and R. T. Khan, "Deep Neural Network Based Resource Allocation for V2X Communications," 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall), Honolulu, HI, USA, 2019, pp. 1-5. [CrossRef]
  76. Y. Li and A. H. Aghvami, "Radio Resource Management for Cellular-Connected UAV: A Learning Approach," in IEEE Transactions on Communications, vol. 71, no. 5, pp. 2784-2800, May 2023. [CrossRef]
  77. Z. Dai, Y. Zhang, W. Zhang, X. Luo and Z. He, "A Multi-Agent Collaborative Environment Learning Method for UAV Deployment and Resource Allocation," in IEEE Transactions on Signal and Information Processing over Networks, vol. 8, pp. 120-130, 2022. [CrossRef]
  78. Alfaia, R.D.; Souto, A.V.d.F.; Cardoso, E.H.S.; Araújo, J.P.L.d.; Francês, C.R.L. Resource Management in 5G Networks Assisted by UAV Base Stations: Machine Learning for Overloaded Macrocell Prediction Based on Users’ Temporal and Spatial Flow. Drones 2022, 6, 145. [CrossRef]
  79. C. Deng, X. Fang and X. Wang, "UAV-Enabled Mobile-Edge Computing for AI Applications: Joint Model Decision, Resource Allocation, and Trajectory Optimization," in IEEE Internet of Things Journal, vol. 10, no. 7, pp. 5662-5675, 1 April1, 2023. [CrossRef]
  80. A. Khalili, E. M. Monfared, S. Zargari, M. R. Javan, N. M. Yamchi and E. A. Jorswieck, "Resource Management for Transmit Power Minimization in UAV-Assisted RIS HetNets Supported by Dual Connectivity," in IEEE Transactions on Wireless Communications, vol. 21, no. 3, pp. 1806-1822, March 2022. [CrossRef]
  81. 2023; Pengshuo Ji, Jie Jia, Jian Chen, Liang Guo, An Du, Xingwei Wang, Reinforcement learning based joint trajectory design and resource allocation for RIS-aided UAV multicast networks, Computer Networks, Volume 227, 2023,109697, ISSN 1389-1286. [CrossRef]
  82. Munaye, Y.Y.; Juang, R.-T.; Lin, H.-P.; Tarekegn, G.B.; Lin, D.-B. Deep Reinforcement Learning Based Resource Management in UAV-Assisted IoT Networks. Appl. Sci. 2021, 11, 2163. [CrossRef]
  83. Ting Lyu, Haiwang Zhang, Haitao Xu, "Resource Allocation in UAV-Assisted Wireless Powered Communication Networks for Urban Monitoring", Wireless Communications and Mobile Computing, vol. 2022, Article ID 7730456, 15 pages, 2022. [CrossRef]
  84. Y. Lin, M. Wang, X. Zhou, G. Ding, and S. Mao, “Dynamic spectrum interaction of UAV flight formation communication with priority: A deep reinforcement learning approach,” IEEE Trans. Cogn. Commun. Netw., vol. 6, no. 3, pp. 892–903, Sep. 2020. [CrossRef]
  85. S. Zhu, L. Gui, N. Cheng, Q. Zhang, F. Sun, and X. Lang, “UAV-enabled computation migration for complex missions: A reinforcement learning approach,” IET Commun., vol. 14, no. 5, pp. 2472–2480, 2020. [CrossRef]
  86. K. Kim, Y. M. Park, and C. S. Hong, “Machine learning based edge assisted UAV computation offloading for data analyzing,” in Proc. IEEE Int. Conf. Inf. Netw. (ICOIN), 2020, pp. 117–120.
  87. J. Cui, Y. Liu, and A. Nallanathan, “Multi-agent reinforcement learning-based resource allocation for UAV networks,” IEEE Trans. Wireless Commun., vol. 19, no. 2, pp. 729–743, Feb. 2020. [CrossRef]
  88. O. Anicho, P. B. Charlesworth, G. S. Baicher, A. Nagar, and N. Buckley, “Comparative study for coordinating multiple unmanned HAPS for communications area coverage,” in Proc. IEEE Int. Conf. Unmanned Aircraft Syst. (ICUAS), 2019, pp. 467–474.
  89. Liu, C.; Zhu, Q. Joint Resource Allocation and Learning Optimization for UAV-Assisted Federated Learning. Appl. Sci. 2023, 13, 3771. [CrossRef]
  90. T. Zeng, O. Semiari, M. Mozaffari, M. Chen, W. Saad and M. Bennis, "Federated Learning in the Sky: Joint Power Allocation and Scheduling with UAV Swarms," ICC 2020 - 2020 IEEE International Conference on Communications (ICC), Dublin, Ireland, 2020, pp. 1-6. [CrossRef]
  91. S. Wang et al., “Federated learning for task and resource allocation in wireless high altitude balloon networks,” [Online]. Available: arxiv.abs/2003.09375. [CrossRef]
  92. J. Wang, X. Zhang, X. He and Y. Sun, "Bandwidth Allocation and Trajectory Control in UAV-Assisted IoV Edge Computing Using Multiagent Reinforcement Learning," in IEEE Transactions on Reliability, vol. 72, no. 2, pp. 599-608, June 2023. [CrossRef]
  93. Z. Zhang, X. Xie, C. Xu and R. Wu, "Energy Harvesting-Based UAV-Assisted Vehicular Edge Computing: A Deep Reinforcement Learning Approach," 2022 IEEE/CIC International Conference on Communications in China (ICCC Workshops), Sanshui, Foshan, China, 2022, pp. 199-204. [CrossRef]
  94. N. Hu, X. Qin, N. Ma, Y. Liu, Y. Yao and P. Zhang, "Energy-efficient Caching and Task offloading for Timely Status Updates in UAV-assisted VANETs," 2022 IEEE/CIC International Conference on Communications in China (ICCC), Sanshui, Foshan, China, 2022, pp. 1032-1037. [CrossRef]
  95. C. Yang, B. Liu, H. Li, B. Li, K. Xie and S. Xie, "Learning Based Channel Allocation and Task Offloading in Temporary UAV-Assisted Vehicular Edge Computing Networks," in IEEE Transactions on Vehicular Technology, vol. 71, no. 9, pp. 9884-9895, Sept. 2022. [CrossRef]
  96. W. Y. Bryan Lim et al., "Multi-Dimensional Contract-Matching for Federated Learning in UAV-Enabled Internet of Vehicles," GLOBECOM 2020 - 2020 IEEE Global Communications Conference, Taipei, Taiwan, 2020, pp. 1-6. [CrossRef]
  97. Cheng, Y., Xu, S., Cao, Y., He, Y., Xiao, K. (2022). SBA-GT: A Secure Bandwidth Allocation Scheme with Game Theory for UAV-Assisted VANET Scenarios. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13472. Springer, Cham. [CrossRef]
  98. He, Y.; Zhai, D.; Huang, F.; Wang, D.; Tang, X.; Zhang, R. Joint Task Offloading, Resource Allocation, and Security Assurance for Mobile Edge Computing-Enabled UAV-Assisted VANETs. Remote Sens. 2021, 13, 1547. [CrossRef]
  99. Y. Liu et al., "Joint Communication and Computation Resource Scheduling of a UAV-Assisted Mobile Edge Computing System for Platooning Vehicles," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 8435-8450, July 2022. [CrossRef]
  100. M. Samir, D. Ebrahimi, C. Assi, S. Sharafeddine and A. Ghrayeb, "Leveraging UAVs for Coverage in Cell-Free Vehicular Networks: A Deep Reinforcement Learning Approach," in IEEE Transactions on Mobile Computing, vol. 20, no. 9, pp. 2835-2847, 1 Sept. 2021. [CrossRef]
  101. Jer Shyuan Ng and Wei Yang Bryan Lim and Hong-Ning Dai and Zehui Xiong and Jianqiang Huang and Dusit Niyato and Xian-Sheng Hua and Cyril Leung and Chunyan Miao. "Joint Auction-Coalition Formation Framework for Communication-Efficient Federated Learning in UAV-Enabled Internet of Vehicles." IEEE Transactions on Intelligent Transportation Systems, 22, 2326–2344.
  102. Lim, W. Y. B., Huang, J., Xiong, Z., Kang, J., Niyato, D., Hua, X. S., et al. (2021). Towards federated learning in UAV-enabled internet of vehicles: A multi-dimensional contract-matching approach. IEEE Transactions on Intelligent Transportation Systems, 22, 5140–5154. [CrossRef]
  103. Zheng, K., Sun, Y., Lin, Z., and Tang, Y. (2020). UAV-assisted online video downloading in vehicular networks: A reinforcement learning approach. In 2020 IEEE 91st vehicular technology conference (VTC2020-Spring) (pp. 1–5).
  104. Srinidhi, N. N., Sagar, C. S., Shreyas, J., and SM, D. K. (2020). "An improved PRoPHET-Random forest based optimized multi-copy routing for opportunistic IoT networks". Internet of Things, 11, 100203. [CrossRef]
  105. Nadarajan, J., Kaliyaperumal, J. "QOS aware and secured routing algorithm using machine intelligence in next generation VANET". Int J Syst Assur Eng Manag (2021). [CrossRef]
  106. L. Luo, L. Sheng, H. Yu and G. Sun, "Intersection-Based V2X Routing via Reinforcement Learning in Vehicular Ad Hoc Networks," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 6, pp. 5446-5459, June 2022. [CrossRef]
  107. An, C.; Wu, C.; Yoshinaga, T.; Chen, X.; Ji, Y. A Context-Aware Edge-Based VANET Communication Scheme for ITS. Sensors 2018, 18, 2022. [CrossRef]
  108. Jafarzadeh, O., Dehghan, M., Sargolzaey, H. et al. A Model-Based Reinforcement Learning Protocol for Routing in Vehicular Ad hoc Network. Wireless Pers Commun 123, 975–1001 (2022). [CrossRef]
  109. J. Wu, M. Fang, H. Li and X. Li, "RSU-Assisted Traffic-Aware Routing Based on Reinforcement Learning for Urban Vanets," in IEEE Access, vol. 8, pp. 5733-5748, 2020. [CrossRef]
  110. X. Bi, D. Gao and M. Yang, "A Reinforcement Learning-Based Routing Protocol for Clustered EV-VANET," 2020 IEEE 5th Information Technology and Mechatronics Engineering Conference (ITOEC), Chongqing, China, 2020, pp. 1769-1773. [CrossRef]
  111. Zhang, D.; Zhang, T.; Liu, X. Novel self-adaptive routing service algorithm for application in VANET. Appl. Intell. 2019, 49, 1866–1879. [CrossRef]
  112. Khan, M.U.; Hosseinzadeh, M.; Mosavi, A. An Intersection-Based Routing Scheme Using Q-Learning in Vehicular Ad Hoc Networks for Traffic Management in the Intelligent Transportation System. Mathematics 2022, 10, 3731. [CrossRef]
  113. Rahmani, A.M.; Naqvi, R.A.; Yousefpoor, E.; Yousefpoor, M.S.; Ahmed, O.H.; Hosseinzadeh, M.; Siddique, K. A Q-Learning and Fuzzy Logic-Based Hierarchical Routing Scheme in the Intelligent Transportation System for Smart Cities. Mathematics 2022, 10, 4192. [CrossRef]
  114. Daniel Fuertes, Carlos R. del-Blanco, Fernando Jaureguizar, Juan José Navarro, Narciso García, Solving routing problems for multiple cooperative Unmanned Aerial Vehicles using Transformer networks, Engineering Applications of Artificial Intelligence, Volume 122, 2023, 106085, ISSN 0952-1976. [CrossRef]
  115. Wang, X.; Fu, L.; Cheng, N.; Sun, R.; Luan, T.; Quan, W.; Aldubaikhy, K. Joint Flying Relay Location and Routing Optimization for 6G UAV–IoT Networks: A Graph Neural Network-Based Approach. Remote Sens. 2022, 14, 4377. [CrossRef]
  116. Sadoon Hussain, Ahmed Sami, Abida Thasin, Redhwan M. A. Saad, "AI-Enabled Ant-Routing Protocol to Secure Communication in Flying Networks", Applied Computational Intelligence and Soft Computing, vol. 2022, Article ID 3330168, 9 pages, 2022. [CrossRef]
  117. W. Wang, Y. Liu, R. Srikant and L. Ying, "3M-RL: Multi-Resolution, Multi-Agent, Mean-Field Reinforcement Learning for Autonomous UAV Routing," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 8985-8996, July 2022. [CrossRef]
  118. B. Sliwa, C. Schuler, M. Patchou, C. Wietfeld, "PARRoT: Predictive Ad-hoc Routing fueled by reinforcement learning and trajectory knowledge," 2020, arXiv:2012.05490.
  119. C. He, S. Liu and S. Han, "A Fuzzy Logic Reinforcement Learning-Based Routing Algorithm For Flying Ad Hoc Networks," 2020 International Conference on Computing, Networking and Communications (ICNC), Big Island, HI, USA, 2020, pp. 987-991. [CrossRef]
  120. J. LIU, Q. WANG, C. HE and Y. XU, "ARdeep: Adaptive and Reliable Routing Protocol for Mobile Robotic Networks with Deep Reinforcement Learning," 2020 IEEE 45th Conference on Local Computer Networks (LCN), Sydney, NSW, Australia, 2020, pp. 465-468. [CrossRef]
  121. Yang, Q., Jang, SJ. and Yoo, SJ. Q-Learning-Based Fuzzy Logic for Multi-objective Routing Algorithm in Flying Ad Hoc Networks. Wireless Pers Commun 113, 115–138 (2020). [CrossRef]
  122. Roh, B.-S.; Han, M.-H.; Ham, J.-H.; Kim, K.-I. Q-LBR: "Q-Learning Based Load Balancing Routing for UAV-Assisted VANET". Sensors 2020, 20, 5685. [CrossRef]
  123. H. Fatemidokht, M. K. Rafsanjani, B. B. Gupta and C. -H. Hsu, "Efficient and Secure Routing Protocol Based on Artificial Intelligence Algorithms With UAV-Assisted for Vehicular Ad Hoc Networks in Intelligent Transportation Systems," in IEEE Transactions on Intelligent Transportation Systems, vol. 22, no. 7, pp. 4757-4769, July 2021. [CrossRef]
  124. S. Jiang, Z. Huang and Y. Ji, "Adaptive UAV-Assisted Geographic Routing With Q-Learning in VANET," in IEEE Communications Letters, vol. 25, no. 4, pp. 1358-1362, April 2021. [CrossRef]
  125. Y. He, D. Zhai, Y. Jiang and R. Zhang, "Relay Selection for UAV-Assisted Urban Vehicular Ad Hoc Networks," in IEEE Wireless Communications Letters, vol. 9, no. 9, pp. 1379-1383, Sept. 2020. [CrossRef]
  126. O. S. Oubbati, N. Chaib, A. Lakas, P. Lorenz and A. Rachedi, "UAV-Assisted Supporting Services Connectivity in Urban VANETs," in IEEE Transactions on Vehicular Technology, vol. 68, no. 4, pp. 3944-3951, April 2019. [CrossRef]
Preprints 93253 g001
Figure 2. UAV Centralized Communication
Figure 2. UAV Centralized Communication
Preprints 93253 g001
Preprints 93253 g002
Figure 3. UAV Decentralized Communication
Figure 3. UAV Decentralized Communication
Preprints 93253 g002
Table 2. Summary of Existing Surveys.
Table 2. Summary of Existing Surveys.
Reference Main Research Area Domain AI Technique Covered
[12] Resource management, security and congestion control VANET ML and RL
[13] Mobile edge offloading, security, transportation VANET ML and RL
[14] Resource allocation, security, cognitive radio VANET ML and DL
[15] Spectrum allocation CR-VANET ML and DL
[16] Security, traffic safety and congestion CR-VANET ML, DL and RL
[17] FL based wireless IoT applications CR-IoT FL
[18] MEC decisions based offloading VANET ML and DRL
[19] Resource allocation scenarios VANET ML and DRL
[20] Caching, resource and infrastructure management IoV DRL
[21] Wireless sensor networks VANET ML
[22] Security, routing, resource and mobility management VANET ML and DL
[23] Resource allocation techniques C-V2X ML
[24] Position, cluster and topology-based routing algorithms VANET RL and DRL
[25] FL based security and privacy applications VANET FL
[26] Handover, caching and resource management, routing V2X Communication ML, DL, DRL, FL
[27] Privacy, security, congestion and network delays in fog computing UAV-IoV None
[28] Resource, mobility and security management and object detection IoD ML, DL, DRL
Table 3. AI/ML Solution for UAV-IoV based Resource Allocation.
Table 3. AI/ML Solution for UAV-IoV based Resource Allocation.
Reference Learning Mechanism Contribution Evaluation
[92] Bandwidth allocation, location control deployment and trajectory of UAVs Multi-attentive agent deep deterministic policy gradient (MA2DDPG) Improved convergence velocity of AC-Mix and MA2DDPG by 30.0% and 63.3%
[93] Energy harvesting based UAV-assisted vehicular edge computing framework to maximize the amount of data offloaded to the UAV DRL-based resource allocation and speed optimization (DRL-RASO) model Reward and offloading amount 5.79% and 7.645% higher
[94] Energy minimization by considering cache refreshing, computation offloading and aging of status updates RL based DDPG Not Reported
[95] Auction-coalition building method to allocate UAV coalitions to various IoV component groups FL Decreased FL communication latency
[96] Traffic prediction and car park occupancy management FL Not Reported
[97] bandwidth allocation scheme based on the game theory blockchain-based system Throughput of about %95
[98] Task offloading and security assurance LBTO at 5s and 20s latency highest task processing ratio
[99] Capacity maximization using vehicle platooning quadratic programming sub-problems Not Reported
[100] Efficient communication coverage via trajectories decisions making of UAVs DRL convergence improvement of 40%
[101] Combined auction-integration formations for AV integration with IoV elements FL highest profit of 102
[102] IoV applications for confidentiality via cooperative ML FL Not Reported
[103] Q network to select best UAV advice RL No exact number reported
Table 4. AI/ML Solution for UAV-IoV based Routing.
Table 4. AI/ML Solution for UAV-IoV based Routing.
Reference Learning Mechanism Contribution Evaluation
[122] a Q-learning based load balancing routing (Q-LBR) Q-Learning Improved PDR, network utilization and latency by more than 8%, 28% and 30%.
[123] VRU based routing to handle frequent topology changes Linux Ubuntu 12.04 for routing protocol, VanetMobiSim, MobiSim Improved PDR 16%, detection ratio 7%, decreases end-to-end delay 13% and overhead by 40%.
[124] UAV assisted QAGR algorithm Simulated in NS-3, Q-Learning 90% PDR achieved
[125] Relay selection for A2G VANETs Q- Learning 96% PDR achieved
[126] Traffic prediction and car park occupancy management NS2 and SUMO PDR 90% more than compared protocols
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated