1. Introduction

2. Problem Formulation

Table 1. Variables and their Description

3. Proposed Method

Figure 1. A Three Phase Approach

Figure 1. A Three Phase Approach

3.0.1. Genetic Algorithm, Ant Colony Optimization, and Simulated Annealing with Angular Perturbation Technique (GAACOSA-AP)

In this section, we present the hybrid approach for solving the School Bus Routing Problem (SBRP), combining Genetic Algorithm (GA), Ant Colony Optimization (ACO) and Simulated Annealing (SA), along with the novel angular perturbation technique. The equations and variables are introduced as follows.

We define the set of bus stops as $\mathbf{V}=\{v_0,v_1,\dots ,v_n\}$, where $v_0$ represents the school, and each $v_i$ for $i\ge 1$ represents a specific bus stop. The objective is to minimize the total distance traveled by bus while visiting all stops and returning to the school.

The fitness function for the Genetic Algorithm is expressed as follows:

$$\mathrm{Fitness}\left(\pi \right)=D\left(\pi \right)=\sum _{i=0}^{n-1}{d}_{\pi \left(i\right)\pi (i+1)}+d_{\pi \left(n\right)\pi \left(0\right)}$$

Here, $D\left(\pi \right)$ is the total distance for a route $\pi$, where $\pi \left(i\right)$ denotes the i-th stop in the sequence. The term $d_{\pi \left(i\right)\pi (i+1)}$ is the distance between consecutive stops $\pi \left(i\right)$ and $\pi (i+1)$, and $d_{\pi \left(n\right)\pi \left(0\right)}$ represents the distance from the last stop to the school. The GA uses crossover and mutation operators to evolve the population, ensuring diversity and exploration of various route configurations.

After the initial population is generated by GA, Ant Colony Optimization (ACO) is applied to refine these routes. ACO simulates the behavior of ants, where each ant builds a potential route based on pheromone trails. The pheromone matrix ${\tau }_{ij}$ tracks the level of pheromones for traveling between stops $v_i$ and $v_j$, with $v_i$ and $v_j$ representing consecutive bus stops. The probability that an ant will choose to travel from stop $v_i$ to stop $v_j$ is given by the following:

$$P_{ij}\left(t\right)={\displaystyle \frac{{\tau }_{ij}{\left(t\right)}^{\alpha }·{(1/d_{ij})}^{\beta }}{{\sum }_{k\in \mathrm{allowed}}{\tau }_{ik}{\left(t\right)}^{\alpha }·{(1/d_{ik})}^{\beta }}}$$

In this equation, $\alpha$ controls the influence of the pheromone trail, and $\beta$ controls the influence of the inverse distance $d_{ij}$ between stops $v_i$ and $v_j$. The denominator sums all the next allowed stops k (other possible stops). As ants traverse the routes, they deposit pheromones, reinforcing shorter and more desirable paths.

After ACO has refined the routes, the angular perturbation technique is applied to further improve the solution. For three consecutive stops $v_i$, $v_j$, and $v_k$, the angle ${\theta }_{ijk}$ at stop $v_j$ is calculated using the cosine rule:

$$cos{\theta }_{ijk}={\displaystyle \frac{d_{ij}^2+d_{jk}^2-d_{ik}^2}{2d_{ij}{d}_{jk}}}$$

where $d_{ij}$, $d_{jk}$, and $d_{ik}$ are the distances between stops $v_i$, $v_j$, and $v_k$. The goal of this perturbation step is to minimize the angle ${\theta }_{ijk}$ to reduce sharp turns, resulting in a more efficient route. Smaller angles generally indicate smoother transitions and reduced travel distance.

Once ACO and angular perturbation have been applied, Simulated Annealing (SA) is used to further improve the solution by escaping local optima. SA allows the system to accept worse solutions with a certain probability, helping the algorithm explore other regions of the solution space. The acceptance probability of a worse solution is defined by:

$$P(\Delta D)=e^{-{\displaystyle \frac{\Delta D}{T}}}$$

where $\Delta D=D\left({\pi }^{\prime }\right)-D\left(\pi \right)$ is the difference in distances between the new route ${\pi }^{\prime }$ and the current route $\pi$. The temperature parameter T decreases with time, reducing the probability of accepting worse solutions as the algorithm progresses. This balance between exploration and exploitation allows the algorithm to refine the current best solution while still exploring alternative routes.

In summary, this hybrid approach integrates GA for generating an initial population of routes, ACO for refining routes based on pheromone trails, angular perturbation for optimizing the geometry of the routes by minimizing sharp angles, and SA for ensuring the solution does not get trapped in local optima.

Algorithm 1: GAACOSA-AP for SBR

The Figure 3, demonstrates, SA’s effectiveness in navigating complex solution spaces by accepting suboptimal solutions temporarily to eventually reach the best possible outcome.

3.0.2. Breadth-First Search and Particle Swarm Optimization with Angular Pertubation (BFSPSO-AP)

In this hybrid approach to optimizing the route of school buses, the process begins with breadth-first search (BFS) for the traversal of the route, followed by particle swarm optimization (PSO) and angular perturbation (AP) for optimizing the route. BFS starts by treating the school as the starting point, denoted as $v_0$, and systematically explores the graph, where each vertex represents a bus stop and each edge represents a connection between stops. In each step, the BFS traverses an edge $e_{ij}$, which is the connection between bus stops $v_i$ and $v_j$. BFS ensures that all bus stops are visited once, providing an initial route configuration that serves as a foundation for further optimization. The graphs in Figure 4 and Figure 5, illustrate the BFS traversal for a school bus visiting 10 bus stops. The bus at Figure 4, starts at Stop 1, represented by the green square, and follows the red edges, visiting nearby stops in sequence. The numbers next to each stop indicate the order in which the bus visits each location to pick up children. In the first diagram, the bus starts its route at Stop 1 and moves to Stops 2 and 3, which are the closest neighbors to the starting point. From Stop 2, the bus proceeds to Stops 4 and 5, while from Stop 3, it continues to Stops 6 and 7. The bus then completes its journey by visiting Stops 8, 9, and 10 before returning to Stop 1, thus demonstrating a structured and efficient path with minimal intersections.

In Figure 5, follows the same traversal from Stop 1 to Stop 10, but in this case, the route is more complex with multiple path intersections, suggesting a less optimized approach compared to the first figure. These diagrams demonstrate how BFS works by visiting adjacent stops first before moving further away, presenting two different scenarios for completing the bus route efficiently.

Once BFS generates the initial traversal, PSO is used to optimize the route. In PSO, each potential solution is represented as a particle, with a set of particles $P=\{p_1,p_2,\cdots ,p_n\}$, where each particle $p_i$ represents a candidate bus route. Each particle has a position $x_i$ and a velocity $v_i$, which define its movement through the solution space. The position $x_i\left(t\right)$ at a given iteration t represents the sequence in which the bus stops are visited, and the velocity $v_i\left(t\right)$ determines how the route is updated in subsequent iterations. The fitness function $f\left(p_i\right)$ measures the quality of the solution and is defined as the total distance from the road:

$$f\left(p_i\right)=\sum _{k=1}^{m-1}d(v_k,v_{k+1}),$$

where $d(v_k,v_{k+1})$ is the distance between consecutive bus stops $v_k$ and $v_{k+1}$, and m is the total number of stops. The objective is to minimize $f\left(p_i\right)$, thus minimizing the total distance traveled by the bus.

During each iteration, the velocity of each particle is updated using the following equation:

$$v_i(t+1)=w{v}_i\left(t\right)+c_1{r}_1(p_{\mathrm{best}}-x_i\left(t\right))+c_2{r}_2(g_{\mathrm{best}}-x_i\left(t\right)),$$

where w is the inertia weight, $c_1$ and $c_2$ are acceleration coefficients, $r_1$ and $r_2$ are random values between 0 and 1, $p_{\mathrm{best}}$ is the best known particle position and $g_{\mathrm{best}}$ is the best global position found by the swarm. The position of the particle is then updated as:

$$x_i(t+1)=x_i\left(t\right)+v_i(t+1).$$

After PSO identifies a near-optimal route, Angular Perturbation (AP) is applied to refine the route further. AP modifies the angles between three consecutive bus stops to explore alternative configurations that could reduce the total distance. Consider three consecutive bus stops, $v_a$, $v_b$, and $v_c$. The angle $\theta$ formed by the vectors $\overrightarrow{v_a{v}_b}$ and $\overrightarrow{v_b{v}_c}$ can be slightly adjusted to explore different paths. AP introduces a small change $\Delta \theta$ to this angle, perturbing the route to find more efficient configurations.

The total distance function is then updated to reflect these changes:

$$f\left(p_i^{\prime }\right)=\sum _{k=1}^{m-1}d(v_k^{\prime },v_{k+1}^{\prime }),$$

where $v_k^{\prime }$ and $v_{k+1}^{\prime }$ are the new positions of the bus stops after the perturbation. This optimization process continues until further perturbations no longer yield significant improvements in the total distance. Through this combination of BFS, PSO, and Angular Perturbation, the bus route is iteratively optimized to minimize the overall travel distance while ensuring that all bus stops are visited.

Algorithm 2: BFSPSO-AP for SBR

3.1. Reinforcement Learning with Angular Pertubation (RL-AP)

Reinforcement Learning (RL) combined with Angular Perturbation (AP) is a method used to minimize the total distance traveled while ensuring that all bus stops are visited exactly once. The environment is modeled as a graph $G=(V,E)$, where V represents the bus stops and E represents the routes between these stops. The goal of the RL agent is to learn an optimal route R that minimizes the total distance traveled.

The RL agent interacts with the environment by transitioning between states $s_t$, where each state represents the sequence of bus stops visited at time step t. The agent selects actions $a_t$, which involve choosing the next bus stop to visit. After each action, the agent receives a reward $r_t$ based on the change in the total distance of the route. The reward is defined as:

$$r_t=D_{t-1}-D_t,$$

where $D_t$ is the total distance after visiting the next bus stop, and $D_{t-1}$ is the total distance before visiting the next stop. A positive reward is provided when the total distance decreases, thus encouraging the agent to find more efficient routes.

At each time step, the agent selects an action $a_t$ according to a policy $\pi (a_t\mid s_t)$, which defines the strategy to select the next bus stop. Initially, the policy is random, but it is gradually refined over time through exploration and learning. The agent updates its knowledge using the Q-learning algorithm, where the Q-value $Q(s_t,a_t)$ is updated according to the following equation:

$$Q(s_t,a_t)=Q(s_t,a_t)+\alpha \left[r_t+\gamma \underset{a}{max}Q(s_{t+1},a)-Q(s_t,a_t)\right],$$

where $\alpha$ is the learning rate, $\gamma$ is the discount factor, and ${max}_{a}Q(s_{t+1},a)$ represents the expected future maximum reward for the next state $s_{t+1}$.

After the RL agent learns a near-optimal route, Angular Perturbation (AP) is applied to further refine the solution. This involves selecting three consecutive bus stops $v_a$, $v_b$, and $v_c$ along the route, and adjusting the angle $\theta$ between the vectors $\overrightarrow{v_a{v}_b}$ and $\overrightarrow{v_b{v}_c}$. The angle is perturbed by a small amount $\Delta \theta$ to explore alternative configurations that may reduce the total distance. The perturbed route $R^{\prime }$ is then evaluated, and if it results in a shorter total distance, the agent updates the current best route $R_{\mathrm{best}}$.

As shown in Figure 6, the diagram illustrates how Reinforcement Learning (RL) is applied to find an optimal route for a school bus visiting 10 stops. The different colored paths represent various episodes in the learning process, where the RL agent explores potential routes and gradually improves its actions based on the feedback it receives. The gray path (Episode 1) shows the agent’s initial exploration of the environment, where the bus follows a suboptimal and inefficient route, connecting the stops in a random or exploratory manner as the agent is still learning the structure of the environment and the rewards. In the orange path (Episode 2), there is noticeable improvement as the bus starts to follow a more structured route, reducing unnecessary detours. This indicates that the agent has learned from its earlier experiences and is adjusting its actions to improve efficiency. By the time it reaches the green path (Episode 3), the agent has further refined its strategy, and the bus follows a much more efficient and direct route between stops, suggesting that the RL agent is nearing an optimal solution.

This overall process, combining the learning capabilities of RL with the geometric refinement introduced by Angular Perturbation, allows the agent to continuously improve the bus route and minimize the total distance. The RL agent iteratively refines its policy and route through exploration and perturbation, ensuring an efficient solution.

Algorithm 3: RL-AP for SBR

4. Computational Experiment

5. Comprehensive Analysis Using Clustering and K-Nearest Neighbor Evaluation

6. Conclusion and Future Work

Abbreviations

References

1. Dündar, H.; Ömürgönülşen, M.; Soysal, M. A Review on Sustainable Urban Vehicle Routing. Journal of Cleaner Production 2021, 285, 125444. [Google Scholar] [CrossRef]
2. Miranda, D.M.; de Camargo, R.S.; Conceição, S.V.; Porto, M.F.; Nunes, N.T.R. A Metaheuristic for the Rural School Bus Routing Problem with Bell Adjustment. Expert Syst. Appl. 2021, 180, 115086. [Google Scholar] [CrossRef]
3. Delussu, F.; Imran, F.; Mattia, C.; Meo, R. Fuel Prediction and Reduction in Public Transportation by Sensor Monitoring and Bayesian Networks. Sensors 2021, 21, 4733. [Google Scholar] [CrossRef] [PubMed]
4. Calvete, H.I.; Galé, C.; Iranzo, J.A.; Toth, P. A Partial Allocation Local Search Matheuristic for Solving the School Bus Routing Problem with Bus Stop Selection. Mathematics 2020, 8, 1214. [Google Scholar] [CrossRef]
5. Wang, X.; Tian, J. Dynamic Programming for NP-Hard Problems. Procedia Engineering 2011, 15, 3396–3400. [Google Scholar] [CrossRef]
6. Ellegood, W.A.; Solomon, S.; North, J.; Campbell, J.F. School Bus Routing Problem: Contemporary Trends and Research Directions. Omega 2020, 95, 102056. [Google Scholar] [CrossRef]
7. Plaisted, D.A. New NP-Hard and NP-Complete Polynomial and Integer Divisibility Problems. Theoretical Computer Science 1984, 31, 125–138. [Google Scholar] [CrossRef]
8. Utama, D.M.; Dewi, S.K.; Wahid, A.; Santoso, I.; Pham, D. The Vehicle Routing Problem for Perishable Goods: A Systematic Review. Cogent Eng. 2020, 7, 1816148. [Google Scholar] [CrossRef]
9. Sun, C. A Study of Solving Traveling Salesman Problem with Genetic Algorithm. In Proceedings of the 2020 9th International Conference on Industrial Technology and Management (ICITM), Oxford, UK, 11–13 February 2020; pp. 307–311. [Google Scholar] [CrossRef]
10. Laporte, G. The Traveling Salesman Problem, the Vehicle Routing Problem, and Their Impact on Combinatorial Optimization. International Journal of Strategic Decision Sciences (IJSDS) 2010, 1, 82–92. [Google Scholar] [CrossRef]
11. Ngoo, C.M.; Goh, S.L.; Sze, S.N.; Sabar, N.R.; Ahmad Hijazi, M.H.; Kendall, G. A Survey of Mat-Heuristics for Combinatorial Optimisation Problems: Variants, Trends and Opportunities. Applied Soft Computing 2024, 164, 111947. [Google Scholar] [CrossRef]
12. Shang, P.; Yang, L.; Zeng, Z.; Tong, L. (Carol). Solving School Bus Routing Problem with Mixed-Load Allowance for Multiple Schools. Computers I& Industrial Engineering 2021, 151, 106916. [Google Scholar] [CrossRef]
13. Tole, K.; Milani, M.; Mwakondo, F. Particle Swarm Algorithm for Improved Handling of the Mirrored Traveling Tournament Problem. Tehnički vjesnik 2021, 28, 1647–1653. [Google Scholar] [CrossRef]
14. Bouzid, M.; Masmoudi, O.; Yalaoui, A. Exact Methods and Heuristics for Order Acceptance Scheduling Problem under Time-of-Use Costs and Carbon Emissions. Appl. Sci. 2021, 11, 8919. [Google Scholar] [CrossRef]
15. Vieira, B.S.; Ribeiro, G.M.; Bahiense, L.; Cruz, R.; Mendes, A.B.; Laporte, G. Exact and Heuristic Algorithms for the Fleet Composition and Periodic Routing Problem of Offshore Supply Vessels with Berth Allocation Decisions. European Journal of Operational Research 2021, 295(3), 908–923. [Google Scholar] [CrossRef]
16. Valle, C.A.; Martinez, L.C.; da Cunha, A.S.; Mateus, G.R. Heuristic and Exact Algorithms for a Min–Max Selective Vehicle Routing Problem. Computers I& Operations Research 2011, 38, 1054–1065. [Google Scholar] [CrossRef]
17. Doerner, K.F.; Gronalt, M.; Hartl, R.F.; Kiechle, G.; Reimann, M. Exact and Heuristic Algorithms for the Vehicle Routing Problem with Multiple Interdependent Time Windows. Computers and Operations Research 2008, 35(9), 3034–3048. [Google Scholar] [CrossRef]
18. Dordevic, M.; Tešić, R.; Todorović, S.; Jokić, M.; Das, D.K.; Stević, Ž.; Vrtagic, S. Development of Integrated Linear Programming Fuzzy-Rough MCDM Model for Production Optimization. Axioms 2022, 11, 510. [Google Scholar] [CrossRef]
19. Gilpin, S.; Davidson, I. A Flexible ILP Formulation for Hierarchical Clustering. Artificial Intelligence 2017, 244, 95–109. [Google Scholar] [CrossRef]
20. Podgorelec, D.; Žalik, B.; Mongus, D.; Vlahek, D. A New Alternating Suboptimal Dynamic Programming Algorithm with Applications for Feature Selection. Mathematics 2024, 12, 1987. [Google Scholar] [CrossRef]
21. Gharbi, A. Exploring Heuristic and Optimization Approaches for Elevator Group Control Systems. Appl. Sci. 2024, 14, 995. [Google Scholar] [CrossRef]
22. Irizar, I. A Mathematical Framework for Optimum Design and Operation of SBR Processes. Journal of Water Process Engineering 2021, 39, 101703. [Google Scholar] [CrossRef]
23. Thinakaran, N.; Jayaprakash, J.; Elanchezhian, C. Greedy Algorithm for Inventory Routing Problem in a Supply Chain—A Review. Materials Today: Proceedings 2019, 16, 1055–1060. [Google Scholar] [CrossRef]
24. Jin, T.; Huang, K.; Tang, J.; Xiao, X. Optimal Streaming Algorithms for Multi-Armed Bandits. In Proceedings of the 38th International Conference on Machine Learning; Meila, M., Zhang, T., Eds.; PMLR, 2021; Volume 139, pp. 5045–5054. Available online: https://proceedings.mlr.press/v139/jin21a.html (accessed on 13 September 2024).
25. Holland, J.H. Adaptation in Natural and Artificial Systems; University of Michigan Press: Ann Arbor, MI, USA, 1975. [Google Scholar]
26. Kirkpatrick, S.; Gelatt, C.D.; Vecchi, M.P. Optimization by Simulated Annealing. Science 1983, 220, 671–680. [Google Scholar] [CrossRef] [PubMed]
27. Glover, F. Tabu Search—Part I. ORSA Journal on Computing 1989, 1, 190–206. [Google Scholar] [CrossRef]
28. Zhai, J.; Song, D. Optimal Instance Subset Selection from Big Data Using Genetic Algorithm and Open Source Framework. J. Big Data 2022, 9, 87. [Google Scholar] [CrossRef]
29. Rashidi, T.H.; Zokaei-Aashtiani, H.; Mohammadian, A.K. School Bus Routing Problem in Large-Scale Networks: New Approach Utilizing Tabu Search on a Case Study in Developing Countries. Transportation Research Record 2009, 2137(1), 140–147. [Google Scholar] [CrossRef]
30. Thakkar, H.K.; Shukla, H.; Sahoo, P.K. Chapter 2 - Metaheuristics in Classification, Clustering, and Frequent Pattern Mining. In Cognitive Big Data Intelligence with a Metaheuristic Approach; Mishra, S., Tripathy, H.K., Mallick, P.K., Sangaiah, A.K., Chae, G.S., Eds.; Academic Press: 2022; Series Cognitive Data Science in Sustainable Computing, pp. 21–70. [CrossRef]
31. Yin, R.; Lu, P. A Cluster-First Route-Second Constructive Heuristic Method for Emergency Logistics Scheduling in Urban Transport Networks. Sustainability 2022, 14, 2301. [Google Scholar] [CrossRef]
32. Niculescu, V. On Generalizing Divide and Conquer Parallel Programming Pattern. Mathematics 2022, 10, 3925. [Google Scholar] [CrossRef]
33. Lopes, C.; Rodrigues, A.M.; Romanciuc, V.; Ferreira, J.S.; Öztürk, E.G.; Oliveira, C. Divide and Conquer: A Location-Allocation Approach to Sectorization. Mathematics 2023, 11, 2553. [Google Scholar] [CrossRef]
34. Li, Y.; Wang, J.; Liang, M.; Song, H.; Liao, J.; Lan, Y. A Novel Two-Stage Approach for Automatic Extraction and Multi-View Generation of Litchis. Agriculture 2024, 14, 1046. [Google Scholar] [CrossRef]
35. Feng, J.; Wang, L.; Yu, H.; Jiao, L.; Zhang, X. Divide-and-Conquer Dual-Architecture Convolutional Neural Network for Classification of Hyperspectral Images. Remote Sens. 2019, 11, 484. [Google Scholar] [CrossRef]
36. Kang, X.; Lin, X. Graph-Based Divide and Conquer Method for Parallelizing Spatial Operations on Vector Data. Remote Sens. 2014, 6, 10107–10130. [Google Scholar] [CrossRef]
37. Kumari, M.; De, P.K.; Chaudhuri, K.; Narang, P. Utilizing a Hybrid Metaheuristic Algorithm to Solve Capacitated Vehicle Routing Problem. Results in Control and Optimization 2023, 13, 100292. [Google Scholar] [CrossRef]
38. He, K.; Tole, K.; Ni, F.; Yuan, Y.; Liao, L. Adaptive Large Neighborhood Search for Solving the Circle Bin Packing Problem. Comput. Oper. Res. 2021, 127, 105140. [Google Scholar] [CrossRef]
39. Yuan, Y.; Tole, K.; Ni, F.; He, K.; Xiong, Z.; Liu, J. Adaptive Simulated Annealing with Greedy Search for the Circle Bin Packing Problem. Comput. Oper. Res. 2022, 144, 105826. [Google Scholar] [CrossRef]
40. Tole, K.; Moqa, R.; Zheng, J.; He, K. A Simulated Annealing Approach for the Circle Bin Packing Problem with Rectangular Items. Comput. Ind. Eng. 2023, 176, 109004. [Google Scholar] [CrossRef]
41. He, K.; Tole, K.; Ni, F.; Yuan, Y.; Liao, L. Adaptive Large Neighborhood Search for Circle Bin Packing Problem. CoRR 2020, abs/2001.07709. Available online: https://arxiv.org/abs/2001.07709 (accessed on 17 September 2024).
42. Tole, K.; Moqa, R.; Zheng, J.; He, K. A simulated annealing approach for the circle bin packing problem with rectangular items. Comput. Ind. Eng. 2023, 176, 109004. [Google Scholar] [CrossRef]
43. Parra Inza, E.; Vakhania, N.; Sigarreta Almira, J.M.; Hernández Mira, F.A. Exact and Heuristic Algorithms for the Domination Problem. European Journal of Operational Research 2024, 313, 926–936. [Google Scholar] [CrossRef]