1. Introduction

2. Literature Review

3. Proposed System

3.3. System Design

In this section, we developed a traffic light phase control system that implements additional two-wheeled vehicle lane and the Deep Q-Network (DQN) algorithm to optimize traffic distribution at intersections as shown on Figure 3. This system consists of several key components that work synergistically to achieve the goal of reducing congestion and improving traffic efficiency. The environment serves as a representation of a real-world intersection where vehicles move and are regulated by the traffic light system. Information about the position of vehicles on each road leading to the intersection is collected and used as a state representing the environmental condition at a given time.

Each state consists of spatial information about the position of vehicles in discrete cells dividing the roads leading to the intersection. The DQN algorithm receives this state and calculates the reward based on vehicle waiting times, which is used as a metric to evaluate the system’s performance in reducing congestion. This reward is used to update the algorithm’s policy, enabling it to take more optimal actions in the future.

The DQN algorithm generates actions in the form of traffic light phases (red, yellow, green) applied at the intersection. These actions aim to reduce vehicle waiting times and improve overall traffic flow. The system operates iteratively, where each action taken by the algorithm affects the environment, resulting in a new state and new reward for the next iteration.

The results of implementing this system demonstrate that using a reinforcement learning approach, traffic light phase control can be dynamically optimized according to real traffic conditions. This approach offers an adaptive and effective solution to traffic control problems, focusing on realism and ease of implementation, unlike previous methods that used information-rich states but were difficult to implement in reality.

3.3.1. Environment Design

This study involves configuring the SUMO (Simulation of Urban Mobility) environment to simulate traffic conditions at the Kiaracondong-Ibrahim Adjie intersection in Bandung, Indonesia. The simulation includes key parameters such as vehicle separation, simulation and data collection time steps, and lane widths to ensure realistic traffic modeling. Traffic light phase configurations are based on real-world data from ATCS Bandung, focusing on peak traffic periods to optimize flow. Additionally, dedicated exclusive two-wheeled vehicle lanes are designed according to Southeast Asian traffic standards.

SUMO Environment Configuration

Figure 4. The intersection of Kiaracondong and Ibrahim Adjie features a roundabout with four arms: west, east, north, and south. Dedicated exclusive two-wheeled vehicle lanes, marked in grey, provide a safe and continuous path through the roundabout, separate from 4-wheel or more vehicle lanes. Traffic signals at each entry point manage the flow, enhancing overall safety and efficiency.

Figure 4. The intersection of Kiaracondong and Ibrahim Adjie features a roundabout with four arms: west, east, north, and south. Dedicated exclusive two-wheeled vehicle lanes, marked in grey, provide a safe and continuous path through the roundabout, separate from 4-wheel or more vehicle lanes. Traffic signals at each entry point manage the flow, enhancing overall safety and efficiency.

In this study, the configuration parameters of SUMO (Simulation of Urban Mobility) as shown on Table 4 were adjusted based on literature reviews to ensure realistic and relevant simulations of actual traffic conditions at the Kiaracondong-Ibrahim Adjie intersection. Initially, The minimum recommended separation between vehicles has been set at 1.5m. This parameter is essential to avoid collisions and ensure traffic safety, aligning with studies highlighting the importance of proper lane configurations for safety and efficiency [18].

Next, the simulation time step was set at every 1 second, allowing SUMO to update the state of vehicles and the environment responsively. This brief interval is essential for real-time TSC using RL algorithms [29]. For data collection, the interval was set at every 5 seconds. This interval ensures that minor changes in traffic conditions can be detected and evaluated promptly, supporting more accurate and in-depth analysis.

The best path selection was used to ensure that SUMO determines the most efficient vehicle paths based on current traffic conditions. This approach is important to create realistic and accurate simulations in measuring the impact of changes in traffic signal settings and two-wheeled vehicle lane configurations [30]. The regular lane width was set at 3.2m, reflecting the standard width for four-wheeled vehicle lanes, which is important for maintaining traffic safety and smooth flow.

Finally, the two-wheeled vehicle lane width was set at 3m. This determination is based on previous research that indicates the optimal width for exclusive two-wheeled vehicle lanes to reduce traffic density and improve safety [20,22]. The wider two-wheeled vehicle lane helps separate the flow of motorcycles from larger vehicles, reducing accidents and improving overall traffic efficiency.

With these configuration parameters, the SUMO simulation is expected to reflect the actual traffic conditions at the Kiaracondong-Ibrahim Adjie intersection, Bandung, Indonesia. This allows for accurate analysis of the impact of implementing exclusive two-wheeled vehicle lanes and Reinforcement Learning-based traffic signal control in reducing queue lengths and vehicle waiting times.

Traffic Light Phase Configuration

The traffic light plan data was collected from field configuration data provided by ATCS Bandung, which manages traffic control. There are 8 plans in operation for the traffic light settings at the Kiaracondong-Ibrahim Adjie intersection as shown on Figure 5. The plan changes are based on traffic density as agreed upon by ATCS Bandung. This study focuses on periods of maximum activity from early morning hours between 6:00 AM and 9:00 AM, as well as the late afternoon hours between 3:00 PM and 6:00 PM. During these periods, ATCS has determined that Plan 1 will be implemented from 6:00 AM to 9:00 AM, and Plan 3 will be applied from 3:00 PM to 6:00 PM. Consequently, the testing for both RL and Non-RL programs will employ Plan 1 and Plan 3.

The Table 5 illustrates the duration of green, yellow, and red phases for each plan and direction. It is important to note that the green light duration varies for each plan and direction, ensuring flexibility and adaptation to different traffic conditions. However, the yellow light duration is consistently set at 3 seconds and the red light duration is set at 2 seconds across all plans and directions. This consistency in the yellow and red light durations helps to maintain safety and predictability in traffic signal timings.

The Figure 5 is a visual representation of the traffic light duration data shown in Table 5. Each diagram in the image illustrates the traffic light duration plans for eight different plans (Plan 1 to Plan 8).

In each diagram, the horizontal axis represents time in seconds (s), while the vertical axis represents the traffic light phases. There are four phases represented by numbers 1 to 4 on the vertical axis. The green color on the lines indicates the duration of the green light, while the red color indicates the duration of the red light.

3.3.2. Agent Configuration

There are three elements agents are used in this section: state, action, and reward.

State, The representation of the state of a traffic intersection in terms of cell density using SUMO (Simulation of Urban Mobility) can be explained as follows. Initially, the simulation retrieves a list of vehicle IDs present in the environment using SUMO functions. For each vehicle, the position within its lane and the lane’s identifier are obtained. The lane position is then converted into a distance from the traffic light by subtracting the lane position from 750, which is considered the maximum length of the cell, indicating a highly congested road. This distance is used to map the vehicle to a specific cell representing a smaller segment of the road. The lane in which the vehicle is located is identified by checking the lane ID and mapping it to a corresponding lane group (north, west, south, east). Each lane group is assigned an index (0 to 3). The cell index is determined based on the distance from the traffic light, dividing this distance into several intervals. For example, a distance of 0-6m may be assigned cell index 0, a distance of 6-11m may be assigned cell index 1, and so on, up to a distance of 400-750m assigned cell index 9. The vehicle’s position is calculated by combining this lane group index with the cell index, resulting in a unique number within the range of 0 to 39. If the lane group is valid (with an index between 0 and 3), the position in the ’state’ array corresponding to the vehicle’s position is set to 1, indicating that this cell is occupied by a vehicle. The determination of the cell index lengths is the result of multiple experiments to achieve a satisfactory reward response, as shown in Figure 11. This function ultimately returns the ’state’ array, which indicates the cell density at the intersection and can be utilized for additional traffic analysis.

Figure 6. Cell Index Representation.

Figure 6. Cell Index Representation.

Action, In the traffic simulation by using SUMO, actions act as simulation output that into the environment and will change the phase of traffic light continously. actions are selected and executed iteratively at each simulation step. The current state of the intersection is first collected to assess the positions and numbers of vehicles in various road segments. Based on this state, an action is chosen using an epsilon-greedy policy, which allows the model to either explore random actions or exploit the best action based on model predictions. If selected action is distinct from the preceding one, yellow light is initially activated to ensure a safe transition before switching to next green light. Green light duration is then determined, and the corresponding green light phase is activated. During each simulation step, reward is calculated based on the change in total vehicle waiting time at the intersection. Data regarding the state, action, reward, and next state is stored in memory to train the model at the end of the episode. This process enables the model to gradually learn to optimize traffic flow by reducing waiting times and queue lengths at the intersection.

Figure 7. The Four Phases of Kiaracondong-Ibrahim Adjie Intersection.

Figure 7. The Four Phases of Kiaracondong-Ibrahim Adjie Intersection.

Reward, Reward calculation process is performed at each simulation step to assess the effectiveness of the actions taken. Total waiting time at the intersection is first collected, accounting for the time vehicles spend stopped on the incoming roads. The reward is determined as the distinction between the earlier total waiting time and the present total waiting time. Reduction in total waiting time results in a positive reward, indicating an improvement in traffic flow, while an increase leads to a negative reward. This reward reflects how well the action has mitigated congestion and enhanced traffic flow. This reward data, along with the state, action, and next state, is stored in memory to train the model at the end of the episode, enabling the model to learn from experience and improve future traffic light control decisions. The reward computation in the traffic simulation is based on total WT at the intersection. Key variables employed in the calculation are described as follows:

• $W_t$: Total cumulative waiting time at time t.
• $W_{t-1}$: Total cumulative waiting time at time $t-1$.
• $r_t$: Reward at time t.

The reward on each simulation step is determined by the change in the total waiting time from the past time step ($t-1$) to the present time step (t):

$$r_t=W_{t-1}-W_t$$

(6)

If the waiting time decreases, fewer vehicles are waiting. then $r_t$ will be positive, indicating that the action taken is beneficial. Conversely, if the cumulative waiting time increases, $r_t$ will be negative, indicating that the action taken is detrimental.

The accumulated negative reward over one episode is calculated by summing all the negative rewards.

Let $R_{\mathrm{neg}}$ be the accumulated negative reward over one episode, then:

$$R_{\mathrm{neg}}=\sum _{t=1}^{T}min(r_t,0)$$

(7)

where T is steps for one episode. State of the intersection is represented as a vector of cell densities, where each cell corresponds to a specific segment of the incoming lanes. The action space consists of different traffic light phases that can be activated, each corresponding to a specific direction (e.g., east, west, north, south).

4. Experiments

4.2. DQN Implementation

Figure 9 is the integration of a SUMO environment with a reinforcement learning (RL) agent to optimize traffic light control. In the SUMO environment, the traffic light phase $s_t$ and vehicle delay $r_t$ are observed, representing the state and reward, respectively. The RL agent receives these observations, processes them through an online neural network to estimate Q-values, which predict future rewards for specific actions, and updates these estimates using a target neural network for stability. The agent’s experiences, which include various states, actions, rewards, and the resulting states, are saved in a memory buffer. The agent’s experiences are then sampled in batches to train the neural network.

The loss is calculated as the difference between the predicted Q-values and the target Q-values, and this loss is backpropagated to improve the network’s accuracy. The agent then selects the best action $a_t$ relying on the Q-values, adjusting the traffic light phases to minimize vehicle delays and improve queue length and waiting times at the intersection (environment). This continuous feedback loop allows the RL agent learning and applying efficient traffic management strategies, showcasing potential of RL in enhancing urban traffic systems.

Table 6. Terminology List.

Table 6. Terminology List.

Parameter Name	Symbol
Current State	$s_t$
Next State	$s_{t+1}$
Action	$a_t$
Reward	$r_{t+1}$
Learning Rate	$\alpha$
Discount Factor	$\gamma$
Q-value	$Q(s,a)$
QL	Queue Length
WT	Waiting Times
DQN	Deep Q-learning Network

In a Q-Learning program, several crucial parameters are used to effectively train an agent within a dynamic environment. The Current State ($s_{\mathrm{t}}$) represents the agent’s state within the environment at a spesific time t. This state provides the agent with the necessary information about the environment, which is essential for decision-making. After the agent’s action, the environment transitions to Next State ($s_{\mathrm{t}+1}$), reflecting state of the environment after the action has been taken. The action taken by the agent, referred to as Action ($a_{\mathrm{t}}$), is determined based on a policy that the agent follows, aiming to maximize its cumulative reward.

The Reward ($r_{\mathrm{t}+1}$) is immediate feedback received by the agent upon shifting to next state, offering a measure of the effectiveness of the action taken. This reward guides the agent’s learning process, assisting it in discerning which actions lead to favorable outcomes. To update its knowledge, the agent employs the Learning Rate ($\alpha$), a parameter that dictates the speed at which new information is incorporated into the existing knowledge base. A higher learning rate implies rapid adaptation to new experiences, while a lower rate results in slower adaptation.

A critical parameter is Discount Factor ($\gamma$) that influences the agent’s decision-making process by determining the significance of future rewards. As the discount factor approaches 1, the agent places greater emphasis on future rewards, encouraging long-term planning. Conversely, a lower discount factor prioritizes immediate rewards. The Q-value (Q(s,a)) is a function that estimates the expected utility of performing a specific action in a given state, providing guidance to the agent in selecting actions that maximize its long-term cumulative reward.

Collectively, these parameters enable the Q-Learning algorithm to iteratively refine the agent’s policy. The agent learns from each interaction with the environment, updating its Q-values to reflect the estimated benefits of actions in various states. Over time, this process allows the agent to develop an optimal policy that maximizes its total reward, thereby navigating the environment both efficiently and effectively. This iterative learning approach is foundational to the success of reinforcement learning techniques in dynamic and uncertain environments.

4.4. DQN Training Process

4.4.1. DQN Training Configuration

Based on Figure 5, the plans executed during rush hours are Plan 1 and Plan 3. These plans are implemented to manage the high traffic volumes typically experienced during these times. The specific durations for each phase of the traffic signals—green, yellow, and red—are critical for ensuring smooth traffic flow and minimizing congestion. The detailed durations for each direction (East, West, South, and North) and for each type of phase are summarized in Table 5. The table offers a concise depiction of the timing settings employed to enhance traffic control at the Kiaracondong-Ibrahim Adjie intersection during high-traffic periods. This information will be utilized for both training and evaluation purposes.

RL training is a process in where an agent learns to make an best decisions through action with the environment. In the context of traffic management, this agent aims to reduce vehicle wait times at intersections by optimizing traffic light phases. The RL agent is trained to take actions based on the present state and receive feedback as rewards, reflecting how well these actions achieve the desired goals. In this study, data for training RL was obtained from raw datasheets of the ATCS Bandung city over three months. This data includes the number of vehicles coming from the west, east, north, and south. The training process was conducted over 100 episodes with the following parameters, as shown in Table 7.

Number of layers: This parameter refers to the number of hidden layers in the neural network used for training the reinforcement learning model. More layers can capture more complex patterns, but they also require more computational resources.

Layer width: This indicates the number of neurons in each hidden layer of the neural network. A wider layer can capture more features of the input data, improving the model’s learning capacity.

Batch size: The batch size is the number of training samples used in one iteration before updating the model parameters. A larger batch size can lead to more stable updates, but it requires more memory.

Learning rate: The learning rate is a hyperparameter that controls how much to change the model in response to the estimated error each time the model weights are updated. A smaller learning rate means the model is updated slowly, which can lead to better convergence but requires more training epochs.

Training epochs: The number of epochs is the number of complete passes through the entire training dataset. More epochs allow the model to learn better, but too many can lead to overfitting.

epsilon start: This is the initial value of epsilon in epsilon-greedy policy, which controls the exploration-exploitation trade-off in reinforcement learning. A higher epsilon encourages exploration.

epsilon end: This is the minimum value of epsilon to which it decays. Lower epsilon encourages exploitation, where the agent uses its learned policy more than exploring new actions.

epsilon decay: This parameter determines the rate at which epsilon decreases from epsilon start to epsilon end. A slower decay means the agent will explore more before converging to a stable policy.

The parameters in Table 7 were obtained from an experimental with change the parameters one by one until achieving a positive trend reward output as shown on Figure 11.

Figure 11. Training Reward Plot.

Figure 11. Training Reward Plot.

4.4.2. Training Result

Figure 11 illustrates the cumulative negative reward obtained during the training process of a Deep Q-Network (DQN) algorithm over a series of episodes. Initially, the cumulative negative reward exhibits a marked increase, indicative of the agent’s initial struggle to perform optimally as it explores the action space and learns the environment dynamics. This phase is characterized by substantial negative rewards, reflecting the agent’s trial-and-error learning approach.

As training progresses, the cumulative negative reward begins to oscillate, showing significant fluctuations around a gradually increasing trend. This behavior is emblematic of the exploration and exploitation trade-off inherent in RL algorithms, where the agent oscillates between trying new actions and exploiting known strategies that yield better outcomes. Towards the later episodes, the cumulative negative reward trend appears to stabilize slightly, suggesting that agent is learning in avoiding actions and resulting in high negative rewards and is approaching a more optimal policy. However, the persistent fluctuations indicate that the learning process is not entirely stable, and further training or hyperparameter tuning may be required to achieve more consistent performance.

The trend analysis reveals that during the initial episodes (0 to around 20), the cumulative negative reward increases significantly, reflecting poor performance as the agent explores the environment. From episode 20 to around 60, the reward fluctuates but generally trends upwards, indicating that the agent’s performance is improving with experience. After episode 60, the fluctuations continue, but the trend stabilizes, hinting at some level of convergence.

The significant reward fluctuations, particularly in the middle episodes, can be attributed to the agent balancing exploration of new actions and exploitation of learned policies. These fluctuations underscore the dynamic nature of the learning process in DQN algorithms.

In terms of performance evaluation, the overall increase in the cumulative negative reward suggests that the agent is learning to reduce negative outcomes over time. The stabilization towards the later episodes indicates that the agent is approaching an optimal policy, although further training might be necessary to achieve more consistent performance. This comprehensive analysis of the DQN algorithm’s learning trajectory highlights both the progress made and the challenges remaining in optimizing the agent’s performance.

Figure 12 illustrates the average length of queue at traffic intersections over a series of episodes during the training of a Deep Q-Network (DQN) algorithm. Initially, the average queue length is high, starting at approximately 150 vehicles and gradually decreasing to around 90 vehicles within the first 20 episodes, reflecting the agent’s initial inefficiency in managing traffic.

As training progresses, between episodes 20 and 60, the average queue length continues to decrease and fluctuates around 80 vehicles, indicating the agent’s improvement in traffic signal control. After episode 60, the average queue length stabilizes around 70-80 vehicles, with occasional spikes, suggesting that the agent has developed a more consistent and effective policy.

The fluctuations in average queue length, especially in the initial and middle episodes, highlight the agent’s exploration and exploitation process, experimenting with different signal timings to find the most effective approach.

This performance evaluation aligns with the cumulative negative reward analysis. The high average queue length in the initial episodes correlates with high negative rewards, indicating initial inefficiency. As training progresses, the decrease in average queue length and negative rewards suggests the agent’s improved traffic management. Stabilization in the later episodes corresponds with a stable cumulative negative reward trend, indicating the agent’s convergence towards an optimal policy.

Overall, the DQN algorithm effectively learns and optimizes traffic signal control strategies over time, leading to reduced queue lengths and negative rewards, improved traffic flow, and minimized congestion.

Figure 13 illustrates the cumulative delay in seconds experienced by vehicles over a series of episodes during the training of a Deep Q-Network (DQN) algorithm. Initially, the cumulative delay is high, starting at around 1.8 million seconds, indicating significant waiting times for vehicles. As the training progresses, there is a noticeable and substantial decrease in cumulative delay, demonstrating the DQN agent’s learning and adaptation to optimize traffic flow. By the later episodes, the cumulative delay stabilizes around 1 million seconds, with only minor fluctuations. This decrease and eventual stabilization in cumulative delay indicate that the agent has developed an effective strategy for managing traffic signals, leading to more efficient traffic flow and significantly reduced vehicle waiting times. Overall, the DQN algorithm successfully learns to minimize traffic delays, showing clear improvements in traffic management efficiency over the training period.

5. Results & Discussion

6. Conclusions

References

1. Badan Pusat Statistik. Perkembangan Jumlah Kendaraan Bermotor Menurut Jenis (Unit), 2023.
2. Sutton R., S.; Barto, A. G. Reinforcement Learning Definition. In Reinforcement Learning:An Introduction; Sutton R., S., Barto, A. G, Eds.; MIT Press: London, England, 2018. [Google Scholar]
3. Le, N.; Rathour, V.S.; Yamazaki, K. Deep reinforcement learning in computer vision: a comprehensive survey. Artif Intell Rev 2022, 55, 2733–2819. [Google Scholar] [CrossRef]
4. Datta, S..; Li, Y.; Matthew, M.; Ren, Y.; Shickel, B.; Ozragat, T; Rashidi, P. Reinforcement learning in surgery. Surgery 2021, 170, 329–332. [Google Scholar] [CrossRef] [PubMed]
5. Silver, D. A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go Through Self-play. Science 2018, 362, 1140–1144. [Google Scholar] [CrossRef] [PubMed]
6. Bernstein, A.V.; Burnaev, E.V.; Kachan, O.N. Reinforcement Learning for Computer Vision and Robot Navigation. Machine Learning and Data Mining in Pattern Recognition 2018, 10935, pp–258. [Google Scholar]
7. L. Wang.; Z. Pan.; J. Wang. A Review of Reinforcement Learning Based Intelligent Optimization for Manufacturing Scheduling. Complex System Modeling and Simulation 2021, 1, 257–270. [CrossRef]
8. Zachariah, B; Ayuba, P; Damuut, L. Optimization of Traffic Light Control System of An Intersection Using Fuzzy Inference System. Science World Journal.
9. Biea, Y.; Jia, Y.; Ma, D. Multi-agent Deep Reinforcement Learning Collaborative Traffic Signal Control Method Considering Intersection Heterogeneity. Transportation Research Part C 2024, 164. [Google Scholar] [CrossRef]
10. Savithramma, R; Sumathi, R; Sudhira, H. A Comparative Analysis of Machine Learning Algorithms in Design Process of Adaptive Traffic Signal Control System. 1st International Conference on Artificial Intelligence, Computational Electronics and Communication System (AICECS) 2022, 2161. [Google Scholar]
11. Chunliang, W; Zhenliang, M; Inhi, K. Multi-Agent Reinforcement Learning for Traffic Signal Control: Algorithms and Robustness Analysis. 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC).
12. Zhongqing, Su; Congduan, Li; Inhi, K. Dynamic Route Guidance System Based on Real-time Vehicle-Road Collaborations with Deep Reinforcement Learning. 2023 IEEE 98th Vehicular Technology Conference (VTC2023-Fall), 2023; 1–5.
13. Wei, H; Zheng, G; Gayah, v; Li, Z. A Survey on Traffic Signal Control Methods. arXiv 2020, arXiv:1904.08117.
14. Ding, Z. , Y., Yuan, H., Dong, H. Reinforcement Learning Definition. In Introduction to Reinforcement Learning; Dong, H., Ding, Z., Zhang, S., Eds.; Springer: Singapore, 2020. [Google Scholar]
15. Krajzewicz, D. SUMO definition. In Traffic Simulation with SUMO – Simulation of Urban Mobility; Barceló, J., Ed.; Springer: New York, 2010. [Google Scholar]
16. P. A. Lopez et al. Microscopic Traffic Simulation using SUMO. 2018 21st International Conference on Intelligent Transportation Systems (ITSC), 2575–2582.
17. Krajzewicz, D.; Erdmann, J.; Behrisch, M.; Bieker-Walz, L. Recent Development and Applications of SUMO - Simulation of Urban MObility. International Journal On Advances in Systems and Measurements 2012, 3&4. [Google Scholar]
18. Ibrahim, M.; Hamid, H.; Hua, L.; Voon, W. Evaluating the effect of lane width and roadside configurations on speed, lateral position and likelihood of comfortable overtaking in exclusive motorcycle lane. Accident Analysis and Prevention 2018, 111, 63–70. [Google Scholar] [CrossRef] [PubMed]
19. Hsu, T.; Wen, K. Effect of Novel Divergence Markings on Conflict Prevention Regarding Motorcycle-involved Right Turn accidents of Mixed Traffic Flow. Journal of Safety Research 2019, 69, 167–176. [Google Scholar] [CrossRef] [PubMed]
20. Le, T.; Nurhidayati, Z. A Study of motorcycle lane Design in Some Asian Countries. Sustainable Development of Civil, Urban and Transportation Engineering Conference 2016, 142, 292–298. [Google Scholar] [CrossRef]
21. Mama, S.; Taneerananon, P. Effective Motorcycle Lane Configuration Thailand: A Case Study of Southern Thailand. ENGINEERING JOURNAL 2015, 20. [Google Scholar] [CrossRef]
22. Sukamto, S.; Priyanto, S. Planning and Design of motorcycle lanes in Yogyakarta :: Case study on Janti Road, Yogyakarta. System and Transportation Engineering 2009. [Google Scholar]
23. Idris, M. KRITERIA LAJUR SEPEDA MOTOR UNTUK RUAS JALAN ARTERI SEKUNDER. Puslitbang Jalan dan Jembatan 2010, 27. [Google Scholar]
24. Lee, H; Han, Y; Kim, Y. Reinforcement Learning for Traffic Signal Control: Incorporating A Virtual Mesoscopic Model for Depicting Oversaturated Traffic Conditions. Engineering Applications of Artificial Intelligence 2023, 126, 107005. [CrossRef]
25. Rasheed, F; Yau, K; Low, Y. Deep Reinforcement Learning for Traffic Signal Control Under Disturbances: A Case Study on Sunway city,Malaysia. Future Generation Computer Systems 2020, 109, 431–445. [CrossRef]
26. Ouyang, C; Zhan, Z; Lv, F. A Comparative Study of Traffic Signal Control Based on Reinforcement Learning Algorithms. World Electric Vehicle Journal 2024, 15, 246. [CrossRef]
27. Fuad, M.R.T.; Fernandez, E.O.; Mukhlish, F.; Putri, A.; Sutarto, H.Y.; Hidayat, Y.A.; Joelianto, E. Adaptive Deep Q-Network Algorithm with Exponential Reward Mechanism for Traffic Control in Urban Intersection Networks. Sustainability 2022, 14, 14590. [CrossRef]
28. Qi, F; He, R; Yan, L; Yao, J; Wang, P; Zhao, X. Traffic Signal Control with Deep Q-Learning Network(DQN) Algorithm at Isolated Intersection. 2022 34th Chinese Control and Decision Conference (CCDC) 2022. [CrossRef]
29. Vidali, A; Crociani, L; Vizzari, G; Bandini, S. A Deep Reinforcement Learning Approach to Adaptive Traffic Lights Management. CSAI- Complex Systems & Artificial Intelligence Research Center, 2019; https://www.semanticscholar.org/paper/A-Deep-Reinforcement-Learning-Approach-to-Adaptive-Vidali-Crociani/9c1abe22641981f9caadb8074c7878ef7bb9d81a#citing-papers.
30. Kanis, S.; Samson, L.; Bloembergen, D. Back to Basics: Deep Reinforcement Learning in Traffic Signal Control. arXiv 2021, arXiv:2109.07180.
31. Liu, B.; Ding, Z. A Distributed Deep Reinforcement Learning Method for Traffic Light Control. Neurocomputing 2022, 490, 390–399. [CrossRef]
32. Noaeen, M.; Niak, A.; Goodman, L.G.; Crebo, J.; Abrar, T.; Abad, Z.S.H.; Bazann, A.L.; Bar, F. Reinforcement Learning in Urban Network Traffic Signal Control: A Systematic Literature Review. International Journal of Intelligent Transportation Systems Research 2023, 21, 192–206. [CrossRef]
33. Noaeen, M.; Niak, A.; Goodman, L.G.; Crebo, J.; Abrar, T.; Abad, Z.S.H.; Bazann, A.L.; Bar, F. Reinforcement Learning in Urban Network Traffic Signal Control: A Systematic Literature Review. Expert Systems with Applications 2022, 299, 116830. [CrossRef]
34. Chu, T.; Wang, J.; Codeca, L.; Li, Z. Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control. IEEE Transactions on Intelligent Transportation Systems 2020, 21, 1086–1095. [CrossRef]
35. Guzman, J.A.; Pizarro, G.; Nunez, F. A Reinforcement Learning-Based Distributed Control Scheme for Cooperative Intersection Traffic Control. IEEE Access 2023, 11, 57037–57045. [CrossRef]
36. Studyana, M.D.; Sjafruddin, A.; Kusumantoro, I.P.; Soeharyadi, Y. A Fuzzy logic model for traffic signal intersection. AIP Conf 2018, 1, 060020. [CrossRef]
37. Harahap, E.; Darmawan, D.; Fajar, Y.; Ceha, R.; Rachmiatie, A. Modeling and simulation of queue waiting time at traffic light intersection. Journal of Physics: Conference Series 2019, 1188, 012001. [CrossRef]