In this paper we present the exploration of indoor positioning technologies for UAVs, and navigation techniques for path planning and obstacle avoidance. The objective is to perform warehouse inventory tasks using a drone to search for barcodes or markers to identify the objects. For the indoor positioning techniques, we employ Visual-Inertial Odometry (VIO), Ultra-Wideband (UWB), AprilTag fiducial markers, and Simultaneous Localization and Mapping (SLAM). These algorithms encompass global positioning, local positioning, and pre-mapping positioning, comparing the merits and drawbacks of various techniques and trajectories. In UAV navigation, we combine SLAM-based RTAB-Map indoor mapping and navigation path planning of the ROS for indoor environments. This system enables precise drone positioning indoors and utilizes global and local path planners to generate flight paths that avoid dynamic, static, unknown, and known obstacles, demonstrating high practicality and feasibility. To achieve the warehouse inventory inspection, a reinforcement learning approach is proposed to recognize the markers by adjusting the UAV's viewpoint. We address several main problems in inventory management, including efficiently planning paths while ensuring a certain level of detection rate. Two reinforcement learning techniques, AC (Actor-Critic) and PPO (Proximal Policy Optimization), are implemented based on AprilTag identification. Testing is performed in both simulated and real-world environments, and the effectiveness of the proposed method is validated.Code is available at https://github.com/kellen080/Navigation and https://github.com/kellen080/Indoor\_Positioning.

Evolutionary algorithms and swarm intelligence algorithms find applicability in reinforcement learning of neural networks due to their independence from gradient-based methods. To achieve successful training of neural networks using these algorithms, careful considerations must be made to select appropriate algorithms due to the availability of various algorithmic variations. In Part1, 2 and 3, the author previously reported experimental evaluations on Evolution Strategy, Genetic Algorithm, and Differential Evolution for reinforcement learning of neural networks, utilizing the Acrobot control task. This article constitutes Part4 of the series of comparative research. In this study, Particle Swarm Optimization is adopted as an instance of major swarm intelligence algorithms. The experimental result shows that PSO performed worse than all of DE, GA and ES. The difference between PSO and DE was statistically significant (p<.01). In addition, PSO exhibited lower capability in exploring solutions in high-dimensional search spaces than DE, GA, and ES did. A larger swarm size compensated for the weakness of PSO in global exploration, thus making itself more beneficial than a larger number of swarm search iterations.

The author previously reported an experimental result of evolutionary reinforcement learning of neural network controllers for the pendulum task. In the previous work, a conventional multilayer perceptron was employed in which connection weights were real numbers. In this study, the author experimentally applies an evolutionary algorithm to the reinforcement training of binary neural networks. In both studies, the same task and the same evolutionary algorithm are utilized, i.e. the pendulum control problem and Evolution Strategy respectively. The only differences lie in the memory size per connection weight and the model size of the neural network. The findings from this study are (1) the performance of the binary MLP with 32 hidden units was inferior to that of the real-valued MLP with 16 hidden units; however, this difference was not statistically significant (p >.05); (2) the trained binary MLP successfully swung the pendulum swiftly into an inverted position and maintained its stability after inversion, as the real-valued MLP had done; and (3) the memory size required to record the binary MLP with 32 hidden units is 3.1% or 6.2% of the memory size required to record the real-valued MLP with 16 hidden units.

Modern power system integrates more and more new energy and use a large number of power electronic equipment. This makes it face more challenges in online optimization and real-time control. Deep reinforcement learning(DRL) has the ability of processing big data and high-dimensional features, as well as the ability of independently learning and optimizing decision-making in complex environments. In this paper, we explore DRL based online combination optimization method of grid section for large complex power system. In our method, to improve the convergence speed of the model, we propose to discretize the output action of the unit and simplify the action space. We also design a reinforcement learning loss function with strong constraints to further improve the convergence speed of the model and facilitate the algorithm to obtain the stable solution. Moreover, to avoid the local optimal solution problem caused by the discretization of the output action, we propose to use the annealing optimization algorithm to make the granularity of the unit output finer. We verify our method on IEEE 118-bus system. The experimental results show that our model has fast convergence speed and better performance, and can obtain stable solutions.

This article discuss the estimation of electrical stresses input that is required to determine lifetime degradation of the HV cables installed within overhead transmission lines (OHTL’s). Abort from lightning effects, these electrical stresses are mainly generated due to repetitive switching operation of the transmission line that produces transient overvoltage (TOV) stress and high frequency stress. The transient overvoltage (TOV) due to switching-off the transmission line at the interface points between the cable and OHTL is estimated using artificial neural network (ANN). PSCAD software is used to build the transmission line model for 749 different case in order to build the required database for ANN training. Two different algorithm input to train ANN with one hidden layer and one output layer using different number of nodes are modeled in MATLAB. The results performance is continuously compared and evaluated till acceptable Regression is achieved to insure the error is less than +/-1.5%. The high frequency due to switching-on the transmission line is also recorded and analyzed. The results show that this new method is efficient, accurate and useful as there is no cable monitoring system is available for each HV cable installed within the transmission line.

Unmanned Aerial Vehicles (UAVs) have become increasingly important in various applications, including environmental monitoring, disaster response, and surveillance, due to their flexibility, efficiency, and ability to access hard-to-reach areas. Effective path planning for multiple UAVs exploring a target area is crucial for maximizing coverage and operational efficiency. This study presents a novel approach to optimizing collaborative navigation for UAVs using Multi-Agent Reinforcement Learning (MARL). To enhance the efficiency of this process, we introduce the Adaptive Dimensionality Reduction (ADR) framework, which includes Autoencoders (AEs) and Principal Component Analysis (PCA) for dimensionality reduction and feature extraction. The ADR framework significantly reduces computational complexity by simplifying high-dimensional state spaces while preserving crucial information. Additionally, we incorporate communication modules to facilitate inter-UAV coordination, further improving path planning efficiency. Our experimental results demonstrate that the proposed approach significantly enhances exploration performance and reduces computational complexity, showcasing the potential of combining MARL with ADR techniques for advanced UAV navigation in complex environments.

The use of drones for various applications has become increasingly popular in recent years, and machine learning has played a significant role in this trend. In this paper, we provide a comprehensive survey of the classification and application of machine learning in drones. The paper begins with an overview of the different types of machine learning algorithms and their applications in drones, including supervised learning, unsupervised learning, and reinforcement learning. Next, we present a detailed analysis of various real-world applications of machine learning in drones, such as object recognition, route planning, obstacle avoidance, search area optimization, and autonomous search. The paper also discusses the challenges and limitations of using machine learning in drones, such as data privacy, data quality, and computational requirements. Finally, the paper concludes with a discussion of the future directions of machine learning in drones and its potential impact on various industries and fields. This paper provides a valuable resource for researchers, practitioners, and students interested in the intersection of machine learning and drones.

Continuous greenhouse gas emissions are causing global warming and impacting the habitats of many animals. Researchers in the field of electric power are making efforts to mitigate this situation. Operating and maintaining the power grid in an economic, low-carbon, and stable is challenging. To address the issue, we propose a grid dispatching technique that combines prediction technology, reinforcement learning, and optimization technology. Prediction technology can forecast future power demand and solar power generation, while reinforcement learning and optimization technology can make charging and discharging decisions for energy storage devices based on current and future grid conditions. In the power system, the aggregation of distributed energy resources increases uncertainty, particularly due to the fluctuating generation of renewable energy. This requires the use of advanced predictive control techniques to ensure long-term economic and decarbonization goals. In this paper, we present a real-time dispatching framework that integrates deep learning-based prediction, reinforcement learning-based decision-making, and stochastic optimization techniques. The framework can rapidly adapt to target uncertainty caused by various factors in real-time data distribution and control processes. The proposed framework achieved global Champion in the NeurIPS Challenge 2022 competition and demonstrated its effectiveness in practical scenarios of intelligent building energy management.

A groundbreaking integrated predictive model, termed Cyber Forensics Behavioral Analysis (CFBA), has been developed in an environment characterized by rapidly evolving and increasingly sophisticated cyber threats. This model merges Cyber Behavioral Sciences with Digital Forensics to enhance the prediction accuracy and effectiveness of cyber threats originating from specific Autonomous System Numbers (ASNs). It has been observed that traditional cybersecurity strategies, predominantly focused on technical aspects, must be improved in addressing the complex landscape of cyber threats. Consequently, a novel approach has been proposed in this research, combining technical expertise with insights into the behavior of cybercriminals, thereby addressing a significant gap in existing methodologies. A mixed-methods approach has been employed in the study, integrating quantitative and qualitative research methods. This approach has created a comprehensive framework incorporating digital forensics, cybersecurity, computer science, forensic psychology, and the cyber behavioral sciences which incorporates disciplines scuch as cyberpsychology and forensic cyberpsychology. The study is anchored around four key concepts: (1) Forensic Cyberpsychology, which focuses on understanding psychological aspects of cybercriminal behavior; (2) Digital Forensics, involving the collection and analysis of digital evidence from cyber incidents; (3) Predictive Modeling, which utilizes historical data and patterns to anticipate potential cyber threats; and (4) the Cyber Behavioral Analysis Metric (CBAM) and Cyber Behavioral Score (CBS), tools designed for evaluating and scoring ASNs based on their behavior in terms of cybersecurity threat risks. Challenges encountered in the development of this integration have been significant. Initial setbacks included the misalignment of traditional cyber defense methods with the evolving nature of cyber threats and the absence of a holistic approach in pre-existing models. Through the narrative of these challenges and the subsequent development of the CFBA model, the research highlights the necessity for an interdisciplinary approach to cybersecurity. The outcomes of this study are not merely theoretical but provide practical, actionable tools and frameworks that markedly improve the precision of predicting cyber threats, especially from ASNs. The CFBA model represents a significant shift in cybersecurity strategies, highlighting the importance of integrating behavioral insights with technical knowledge. The study emphasizes the need for ongoing research and collaboration in cybersecurity, advocating for an approach that is as comprehensive and multifaceted as the cyber threats it seeks to mitigate. This research has been conducted to contribute significantly to cybersecurity, offering new perspectives and methodologies in a rapidly changing domain.

The increasing popularity of carbon-reinforced concrete (CRC) is attributed to its exceptional tensile properties, low density, no corrosion phenomenon, and remarkable flexibility, allowing it to be easily shaped into various forms. This research investigates the feasibility of using a special 2D Netzgitterträger (NetzGT) reinforcement system, featuring a net-shaped fabricated textile made of multiple diagonally offset rovings with overlapping edge strands, as a viable alternative to traditional steel reinforcement in concrete beams. This reinforcement is manufactured from carbon rovings with three different diagonal angles of 50⁰, 60⁰, and 70⁰ respectively. Laboratory experiments were conducted to assess the mechanical behavior of beams reinforced with the 2D NetzGT reinforcement. Bending and shear tests were performed on beams with varying numbers of overlapped edge roving and roving angles to evaluate the tensile capacity and failure characteristics of beams. The increase in the number of overlapped edge rovings led to a noticeable increase in the maximum tensile force. Tensile tests on strands were also performed with the increasing number of overlapped rovings to analyze their tensile strength. Additionally, single yarn pull-out tests were also conducted to examine the influence of the roving angle on the bond strength between the carbon textile roving and the concrete matrix

The modern supply chain (SC) is growing in terms of data, devices, users, and stakeholders, which introduced new security challenges and threats, especially with the reliance on centralized servers or cloud platforms. In addition, increased trust among system participants exposes the SC to a higher risk of vulnerabilities which require strong security measures. This article proposes a hybrid security framework for SC systems, BC-DRLzSC, that integrates Blockchain (BC) and Deep Reinforcement Learning (DRL) designed to operate in a zero trust (ZT) environment. In particular, we propose a decentralized BC-based approach integrated with smart contracts to manage system participant registration and authentication and to control access to system resources. BC-DRLzSC adopts a ZT architecture to reinforce SC security, which can be achieved with an advocate to verify each entity’s trustworthiness before granting or retaining access to system resources. Incorporating the ZT architecture, with BC and DRL, can potentially and significantly bolster SC system security. DRL is employed to develop a proactive attack detection model that continuously monitors the incoming traffic from authenticated nodes within the network and predicts any malicious actions. Finally, we evaluate the performance of our proposed DRL solution using the NSL-KDD dataset.

Abstract:The objects of fine-grained image categories(e.g., bird species) are various subclass under different categories. Because the differences between subclass are very subtle and most of them are concentrated in multiple local areas, the task of fine-grained image recognition is very challenging. At the same time, some fine-grained networks tend to focus on a certain region when judging the target category, resulting in the lack of other auxiliary regional features. To this end, Inception V3 is used as the backbone network, and an enhanced and complementary fine-grained image classification network is designed. While adopting the method of reinforcement learning to obtain more detailed fine grain image features, the complementary network can obtain the complementary discriminant area of the target through the method of attention erasure to increase the network's perception of the overall target. Finally, experiments are conducted on CUB-200-2011, FGVC Aircraft and Stanford dogs three open datasets. The experimental results show that the proposed model has better performance.

This paper presents a novel approach to designing beam codebooks for downlink multiuser hybrid multiple input multiple output (MIMO) wireless communication systems, leveraging multi-agent reinforcement learning (MARL). The primary objective is to develop an environment-specific beam codebook composed of non-interfering beams, learned by cooperative agents within the MARL framework. Machine learning (ML) based beam codebook design for downlink communications have been based on channel state information (CSI) feedback or only reference signal received power (RSRP) consisting of an offline training and user clustering phase. In massive MIMO, the full CSI feedback data is of large size and is resource-intensive to process, making it challenging to implement efficiently. RSRP alone for a stand-alone base station is not a good marker of the position of a receiver. Hence, in this work, uplink CSI estimated at the base station along with feedback of RSRP and binary acknowledgment of the accuracy of received data is utilized to design the beamforming codebook at the base station. Simulations using sub-array antenna and ray-tracing channel demonstrate the proposed system’s ability to learn topography-aware beam codebook for arbitrary beams serving multiple user groups simultaneously. The proposed method extends beyond mono-lobe and fixed beam architectures by dynamically adapting arbitrary shaped beams to avoid inter-beam interference, enhancing overall system performance. This work leverages MARL’s potential in creating efficient beam codebooks for hybrid MIMO systems, paving the way for enhanced multiuser communication in future wireless networks.

While researchers have made notable progress in bio-inspired swimming robot development, a persistent challenge lies in creating propulsive gaits tailored to these robotic systems. The California sea lion achieves its robust swimming abilities through a careful coordination of foreflippers and body segments. In this paper, reinforcement learning (RL) was used to develop a novel sea lion foreflipper gait for a bio-robotic swimmer using a numerically modelled computational representation of the robot. This model integration enabled reinforcement learning to develop desired swimming gaits in the challenging underwater domain. The novel RL gait outperformed the characteristic sea lion foreflipper gait in the simulated underwater domain. When applied to the real-world robot, the RL constructed novel gait performed as well or better than the characteristic sea lion gait in many factors. This work shows the potential for using complimentary bio-robotic and numerical models with reinforcement learning to enable the development of effective gaits and maneuvers for underwater swimming vehicles.

This study assesses the impact of incorporating an adaptive learning mechanism into an agent-based model (ABMS) simulating behavior on a university campus during the course of a pandemic outbreak, with a particular case on the COVID-19 pandemic. The aim is to reduce overcrowding and infections on campus through the use of Reinforcement Learning (RL). Our findings indicate that RL is a viable approach for effectively representing agents’ behavior within this context. The results reveal specific temporal patterns of overcrowding violations. While our study successfully mitigated campus crowding, it had limited influence on altering the course of the epidemic. This highlights the necessity for comprehensive epidemic control strategies that consider the role of individual decision-making influenced by adaptive learning, along with the implementation of targeted interventions. This research significantly contributes to our understanding of adaptive learning within complex systems and offers valuable insights for shaping future public health policies in similar community settings. Future research directions encompass exploring various parameter settings and updating representations of the disease’s natural history.

The application of space robotic manipulators and heightened autonomy for ios represents a paramount pursuit for leading space agencies, given the substantial threat posed by space debris to operational satellites and forthcoming space endeavors. This work presents a guidance algorithm based on drl to solve for the space manipulator path-planning during the motion synchronization phase with the mission target. The goal is the trajectory generation and control of a spacecraft equipped with a 7-dof robotic manipulator, such that its end effector remains stationary with respect to the target point of capture. The ppo drl algorithm is used to optimize the manipulator’s guidance law, and the autonomous agent generates the desired joints rates of the robotic arm, which are then integrated and passed to a model-based feedback linearization controller. The agent is first trained to optimize its guidance policy and then tested extensively to validate the results against a simulated environment representing the motion synchronization scenario of an ios mission.

In this study, reinforcement learning (RL) was used in factory simulation to optimize storage devices for use in Industry 4.0 and digital twins. First, we defined an RL environment, modeled it, and validated its ability to simulate a real physical system. Subsequently, we introduced a method to calculate reward signals and apply them to the environment to ensure the alignment of the behavior of the RL agent with the task objective. The stocker simulation model was used to validate the effectiveness of RL. The model is a storage device that simulates logistics in a manufacturing production area. The results revealed that RL is a useful tool for automating and optimizing complex logistics systems and increase the applicability of RL in logistics. We proposed a novel method for creating an agent through learning using the proximal policy optimization algorithm, and the agent was optimized by configuring various learning options. The application of reinforcement learning resulted in an effectiveness of 30% to 100%, and methods can be expanded to other fields.

Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (Predictability, Computability, Stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning (Yu and Kumbier, 2020), to the design of RL algorithms for the digital interventions setting. Further, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We illustrate the use of the PCS framework for designing an RL algorithm for Oralytics, a mobile health study aiming to improve users' tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

A detailed literature review is performed in this study to address solutions for the full-body design and control of an aerial manipulator. Deep Reinforcement Learning methods are growing to be utilized recently to cope with various uncertainties. The pros and cons of these theories will be explained as well as introducing the advantages of Fuzzy Reinforcement Learning methods. State-of-the-Art, possible challenges, potential approaches, and a summary of desired precision devices are discussed in this study.

The steep rise in reinforcement learning (RL) in various applications in energy as well as the penetration of home automation in recent years are the motivation for this article. It surveys the use of RL in various home energy management system (HEMS) applications. There is a focus on deep neural network (DNN) models in RL. The article provides an overview of reinforcement learning. This is followed with discussions on state-of-the-art methods for value, policy, and actor–critic methods in deep reinforcement learning (DRL). In order to make the published literature in reinforcement learning more accessible to the HEMS community, verbal descriptions are accompanied with explanatory figures as well as mathematical expressions using standard machine learning terminology. Next, a detailed survey of how reinforcement learning is used in different HEMS domains is described. The survey also considers what kind of reinforcement learning algorithms are used in each HEMS application. It suggests that research in this direction is still in its infancy. Lastly, the article proposes four performance metrics to evaluate RL methods.

The multi-UAV target search problem is crucial in the field of autonomous Unmanned Aerial Vehicle (UAV) decision-making. The algorithm design of Multi-agent Reinforcement Learning (MARL) methods has become integral to research on multi-UAV target search, due to its adaptability to the rapid online decision-making required by UAVs in complex, uncertain environments. In non-cooperative target search scenarios, targets may have the ability to escape, complicating UAVs' search efforts and hindering the convergence of the MARL algorithm training. This paper investigates the multi-UAV target search problem in scenarios involving static obstacles and dynamic escape targets, modeling the problem within the framework of Decentralized Partially Observable Markov Decision Process. Based on this model, a Spatio-Temporal Efficient Exploration network and a Global Convolutional Local Ascent mechanism are proposed. Subsequently,we introduce a Multi-UAV Escape Target Search Algorithm Based on MAPPO(ETS-MAPPO) for addressing the escape target search difficulty problem. Simulation results demonstrate that the ETS-MAPPO algorithm outperforms five classical MARL algorithms in terms of the number of target searches,the area coverage rate, and other metrics.

This paper presents an innovative multi-agent, data-driven reinforcement learning (RL) approach to develop and utilize the thermal equivalent network model that represents the motor's thermal dynamics. A multi-agent reinforcement learning is designed and trained to adjust the model parameters using data from several motor driving cycles. To ensure the incoming driving cycle matches the historical data before employing the pre-trained RL agents, offline statistical analysis and clustering techniques are developed and used. Numerical simulations highlighted the RL agent's ability to develop strategies that effectively address the variability of driving cycles. The proposed RL framework showed its capability to accurately reflect the motor's thermal behavior under various driving conditions.

Climate change and development have increased urban flooding, requiring modernization of stormwater infrastructure. Retrofitting standard passive systems with controllable valves/pumps is promising, but requires real-time control (RTC). One method of automating RTC is reinforcement learning (RL), a general technique for sequential optimization and control in uncertain environments. The notion is that an RL algorithm can use inputs of real-time flood data and rainfall forecasts to learn a policy for controlling the stormwater infrastructure to minimize measures of flooding. In real-world conditions, rainfall forecasts and other state information, are subject to noise and uncertainty. To account for these characteristics of the problem data, we implemented Deep Deterministic Policy Gradient (DDPG), an RL algorithm that is distinguished by its capability to handle noise in the input data. DDPG implementations were trained and tested against a passive flood control policy. Three primary cases were studied: (i) perfect data, (ii) imperfect rainfall forecasts, and (iii) imperfect water level and forecast data. Rainfall episodes (100) that caused flooding in the passive system were selected from 10 years of observations in Norfolk, Virginia, USA; 85 randomly selected episodes were used for training and the remaining 15 unseen episodes served as test cases. Compared to the passive system, all RL implementations reduced flooding volume by 70.5% on average, and performed within a range of 5%. This suggests that DDPG is robust to noisy input data, which is essential knowledge to advance the real-world applicability of RL for stormwater RTC.

This paper presents a study on the thermal efficiency improvement of the user equipment room in the district heating system based on reinforcement learning [1], and suggests a general method of constructing a learning network(DQN)[2] using deep Q learning[3], which is a reinforcement learning algorithm that does not specify a model. In addition, we introduce the big data platform system and the integrated heat management system for the energy field in the massive data processing from the IoT sensor installed in large number of thermal energy control facilities.

This paper investigates the impact of adding exclusive two-wheeled vehicle lane based on motorcycle data and using reinforcement learning at one of Bandung’s heterogeneous intersections which uses data collected from ATCS Bandung. The Kiaracondong-Ibrahim Adjie intersection in Bandung is notorious for severe congestion, especially during rush hours. Traditional traffic management methods often fall short, necessitating innovative solutions to mitigate long wait times and extensive queue lengths. This study leverages Reinforcement Learning (RL), specifically Deep Q-learning Network algorithm, combined with introduction of additional two-wheeled vehicle lanes to optimize traffic flow at this busy intersection. The research involved configuring the SUMO platform to accurately simulate the intersection’s traffic conditions, including parameters such as lane widths and vehicle dynamics. The RL model was trained over 100 episodes using quantitative real traffic data from ATCS Bandung city. The training process minimize vehicle waiting time and queue length by adjusting traffic light phases dynamically. Results from the simulations indicated significant improvements in traffic management. Queue lengths in regular lanes decreased by 64.89% under RL control, while two-wheeled vehicle lanes saw a 26.39% reduction. Waiting times in regular lanes dropped by 80.49%, and in two-wheeled vehicle lanes by 39.96%. The study demonstrated that integrating DQN with dedicated two-wheeled vehicle lanes could substantially enhance traffic flow and reduce congestion at critical urban intersections. The findings underscore the potential of advanced RL techniques in urban traffic management. However, the study acknowledges the need for further research with more extensive resources and time to develop even more efficient traffic control systems. Future work should focus on refining these methods for broader application and exploring other innovative technologies to sustainably address urban traffic challenges.

The objective of the this study was to reduce the benzo[a]pyrene-induced hepatotoxicity method in herbal medicine products and to give proof that neferine, daidzein, genistein possess antihepa-totoxic effects. B[a]P which classified as a group 1 carcinogen and metabolized to B[a]P-7,8-dihydrodiol-9,10-epoxide (BPDE) causes mutagenic DNA addition products. The reduc-tion of BPDE-DNA adduct formation by B[a]P-7,8-dihydrodiol-9,10-epoxide (BPDE) was derived by benzo[a]pyrene (B[a]P). In HepG2 cells, B[a]P exhibited toxicity and substance treatment of the cells with B[a]P with neferine in lotus and daidzein, genistein in soybean reduced the BPDE-DNA ad-ducts level. The level of B[a]P-metabolites in the substance treatment of the cells was presented that BPDE levels were reduced by neferine in lotus and daidzein, genisten in soybean. These results suggest that neferine in lotus and daidzein, genistein in soybean prevent B[a]P-induced hepato-toxicity for BPDE-DNA adduct formation.

BackgroundThe use of antimicrobials to treat food animals may result in antimicrobial residues in foodstuffs of animal origin. The European Medicines Association (EMA) and World Health Organization (WHO) define safe antimicrobial concentrations in food based on acceptable daily intakes (ADIs). It is unknown if ADI doses of antimicrobials in food could influence the antimicrobial susceptibility of human-associated bacteria. ObjectivesThis aim of this study was to evaluate if the consumption of ADI doses of erythromycin could select for erythromycin resistance in a Galleria mellonella model of Streptococcus pneumoniae infection. Methods A chronic model of S. pneumoniae infection in G. mellonella larvae was used for the experiment. Inoculation of larvae with S. pneumoniae was followed by injections of EMA (0.0875 μg/ml) and WHO (0.012 μg/ml) ADI doses of erythromycin proceeded by isolation of S. pneumoniae colonies on selective agar plates. Minimum inhibitory concentrations (MICs) of resistant colonies were measured, and whole genome sequencing (WGS) was performed followed by variant calling to determine the genetic modifications.   Results Exposure to single doses of both EMA and WHO ADI doses of erythromycin resulted in the emergence of erythromycin resistance in S. pneumoniae. Emergent resistance to erythromycin was associated with a mutation in rplA. Conclusion In our in vivo model, even single dose of erythromycin that are classified as acceptable by the WHO and EMA induced significant increases in erythromycin MICs in S. pneumoniae. These results suggest the need to include the induction of antimicrobial resistance (AMR) as a significant criterion for determining ADIs.

Attitude control of a novel regional truss-braced wing aircraft with low stability characteristics is addressed in this paper using Reinforcement Learning (RL). In recent years, RL has been increasingly employed in challenging applications, particularly, autonomous flight control. However, a significant predicament confronting discrete RL algorithms is the dimension limitation of the state-action table and difficulties in defining the elements of the RL environment. To address these issues, in this paper, a detailed mathematical model of the mentioned aircraft is first developed to shape an RL environment. Subsequently, Q-learning, the most prevalent discrete RL algorithm will be implemented in both the Markov Decision Process (MDP), and Partially Observable Markov Decision Process (POMDP) frameworks to control the longitudinal mode of the air vehicle. In order to eliminate residual fluctuations that are a consequence of discrete action selection, and simultaneously track variable pitch angles, a Fuzzy Action Assignment (FAA) method is proposed to generate continuous control commands using the trained Q-table. Accordingly, it will be proved that by defining an accurate reward function, along with observing all crucial states (which is equivalent to satisfying the Markov Property), the performance of the introduced control system surpasses a well-tuned Proportional–Integral–Derivative (PID) controller.

This paper reviews recent works related to applications of reinforcement learning in power system optimal control problems. Based on an extensive analysis of works in the recent literature, we attempt to better understand what is the gap between reinforcement learning methods that rely on complete or incomplete information about the model dynamics, and data-driven reinforcement learning approaches. More specifically we ask how such models change based on the application or the algorithm, what are the currently open theoretical and numerical challenges in each of the leading applications, and which reinforcement-based control strategies will rise in the following years. The reviewed research works are divided to ``model-based'' methods and ``model-free'' methods, in order to highlight the current developments and trends within each of these two groups. The optimal control problems reviewed are energy markets, grid stability and control, energy management in buildings, electrical vehicles and energy storage.

Transcutaneous injection laryngoplasty is a well-known procedure for treating paralyzed vocal fold by injecting augmentation material into it. Hence, vocal fold localization plays a vital role in the preoperative planning as the fold location is required to determine the optimal injection route. In this communication, we propose a mirror environment based reinforcement learning (RL) algorithm for localizing the right and left vocal folds in preoperative neck CT. RL-based methods commonly showed noteworthy outcome in general anatomic landmark localization problem in the recent years. However, such methods suggest training individual agent for localizing each fold, though the right and left vocal folds are located in close proximity and have high feature-similarity. Utilizing the lateral symmetry between the right and left vocal folds, the proposed mirror environment allows for a single agent for localizing both the folds by treating the left fold as a flipped version of the right fold. Thus, localization of both folds can be trained using a single training session which utilizes the inter-fold correlation and avoids redundant feature learning. Experiment with 120 CT volumes showed improved localization performance and training efficiency of the proposed method compared with the standard RL method.

Aiming at the problem of model error and tracking dependence in the process of intelligent vehicle motion planning, an intelligent vehicle model transfer trajectory planning method based on deep reinforcement learning is proposed, which obtain an effective control action sequence directly. Firstly, an abstract model of the real environment is extracted. On this basis, Deep Deterministic Policy Gradient (DDPG) and vehicle dynamic model are adopted to jointly train a reinforcement learning model, and to decide the optimal intelligent driving maneuver. Secondly, the actual scene is transferred to equivalent virtual abstract scene by transfer model, furthermore, the control action and trajectory sequences are calculated according to trained deep reinforcement learning model. Thirdly, the optimal trajectory sequence is selected according to evaluation function in the real environment. Finally, the results demonstrate that the proposed method can deal with the problem of intelligent vehicle trajectory planning for continuous input and continuous output. The model transfer method improves the model generalization performance. Compared with the traditional trajectory planning, the proposed method output continuous rotation angle control sequence, meanwhile, the lateral control error is also reduced.

In recent years, advancements in brain science and neuroscience have significantly influenced the field of computer science, particularly in the domain of reinforcement learning (RL). Drawing insights from neurobiology and neuropsychology, researchers have leveraged these findings to develop novel mechanisms for understanding intelligent decision-making processes in the brain. Concurrently, the emergence of quantum computing has opened new frontiers in artificial intelligence, leading to the development of quantum machine learning (QML). This study introduces a novel model that integrates Quantum Spiking Neural Networks (QSNN) and Quantum Long Short-Term Memory (QLSTM) architectures, inspired by the complex workings of the human brain. Specifically designed for reinforcement learning tasks in energy-efficient environments, our approach progresses through two distinct stages mirroring sensory and memory systems. In the initial stage, analogous to the brain’s hypothalamus, low-level information is extracted to emulate sensory data processing patterns. Subsequently, resembling the hippocampus, this information is processed at a higher level, capturing and memorizing correlated patterns. We conduct a comparative analysis of our model against existing quantum models such as Quantum Neural Networks and their classical counterparts, elucidating its unique contributions. Through empirical results, we aim to underscore the effectiveness of our approach in optimizing energy consumption in practical scenarios.

The collision avoidance mechanism adopted by the IEEE 802.11 standard is not optimal. The mechanism employs a binary exponential backoff (BEB) algorithm in the medium access control (MAC) layer. Such an algorithm increases the backoff interval whenever a collision is detected to minimize the probability of subsequent collisions. However, the expansion of the backoff interval causes degradation of the radio spectrum utilization (i.e., bandwidth wastage). That problem worsens when the network has to manage the channel access to a dense number of stations, leading to a dramatic decrease in network performance. Furthermore, a wrong backoff setting increases the probability of collisions such that the stations experience numerous collisions before achieving the optimal backoff value. Therefore, to mitigate bandwidth wastage and, consequently, maximize the network performance, this work proposes using reinforcement learning (RL) algorithms, namely Deep Q Learning (DQN) and Deep Deterministic Policy Gradient (DDPG), to tackle such an optimization problem. As for the simulations, the NS-3 network simulator is used along with a toolkit known as NS3-gym, which integrates a reinforcement-learning (RL) framework into NS-3. The results demonstrate that DQN and DDPG have much better performance than BEB for static and dynamic scenarios, regardless of the number of stations. Moreover, the performance difference is amplified as the number of stations increases, with DQN and DDPG showing a 27% increase in throughput with 50 stations compared to BEB. Furthermore, DQN and DDPG presented similar performances.

Deep reinforcement learning (DRL) has achieved significant results in many Machine Learning (ML) benchmarks. In this short survey we provide an overview of DRL applied to trading on financial markets, including a short meta-analysis using Google Scholar, with an emphasis on using hierarchy for dividing the problem space as well as using model-based RL to learn a world model of the trading environment which can be used for prediction. In addition, multiple risk measures are defined and discussed, which not only provide a way of quantifying the performance of various algorithms, but they can also act as (dense) reward-shaping mechanisms for the agent. We discuss in detail the various state representations used for financial markets, which we consider critical for the success and efficiency of such DRL agents. The market in focus for this survey is the cryptocurrency market.

The working environment of mining electric locomotives is wet and muddy coal mine 1 roadway. There may be idling or slipping between the wheels and rails of mining electric locomotives 2 due to low friction between the wheel and rail and insufficient utilization of creep rate. Therefore, it 3 is necessary to control the creep rate within a reasonable range. In this paper, the autonomous control 4 algorithm for mining electric locomotives based on improved ε-greedy is theoretically proven to be 5 convergent and effective firstly. Secondly, after analyzing the contact state between the wheel and rail 6 under wet and slippery road conditions, it was concluded that the value of creep rate is an important 7 factor affecting the autonomous driving of mining electric locomotives. Therefore, the autonomous 8 control method for mining electric locomotives based on creep control is proposed in this paper. 9 Finally, the effectiveness of the proposed method was verified through simulation. The problem of 10 wheel slipping and idling caused by insufficient friction of mining electric locomotives in coal mining 11 environments is effectively suppressed. Autonomous operation of vehicles with optimal driving 12 efficiency can be achieved through quantitative control and utilization of the creep rate between 13 wheels and rails

The convergence of artificial intelligence and the Internet of Things (IoT) has made remarkable strides in the realm of industry. The IoT devices in this computing model gather data from various sources and send it to edge servers for real-time processing, which presents a challenge to the existing message queue systems in Artificial Intelligence of Things (AIoT) Edge computing. These systems lack the adaptability to respond to the current state of the system, such as changes in the number of devices, message size, and frequency, and optimize the message transmission mechanism accordingly. Hence, it is critical to devise an approach that can effectively decouple message processing and mitigate workload fluctuations in the AIoT computing environment. To this end, this study introduces a distributed message queue system that is specifically tailored to the AIoT edge environment, utilizing a reinforcement learning approach to optimize message queue performance. Empirical findings reveal that this pioneering method significantly improves system throughput while handling varying message scenarios.

In tunnel excavation with boring machines, the tunnel face is supported to avoid collapse and minimise settlement. This article proposes the use of reinforcement learning, specifically the Deep Q-Network algorithm, to predict the face support pressure. The approach is tested both analytically and numerically. By using the soil properties ahead of the tunnel face and the overburden depth as the input, the algorithm is capable of predicting the optimal tunnel face support pressure, adapting to changes in geological and geometrical conditions.

Ever-increasing need for improving the livability of a city and improve outcomes for its residents, over the last decade, the adoption of technology to develop urbanised societies around the world has given rise to the need for developing smart cities. The speed at which the world population is growing, the use of Internet of Things in smart cities have really advanced the quality of life. One significant area of concern within the smart city framework is waste management. If the waste within a city is not adequately managed, then it leads to issues in the health of the citizens. Additionally, the waste management has such a high impact on the environmental footprint, hence the need to have a smart way of managing waste is of critical importance. Through our research, we analyse the challenges of waste management within a city to understand the impact of the problem on to the citizens and overall city operations. We then investigate ways in which we can solve these problems using the emerging technologies, such as the Internet of Things, to collect valuable data of large volumes arriving at an astronomical rate, then apply multi-agent deep reinforcement learning algorithms to harness the power of big data to extract meaningful information and actionable insights. We ingest data generated by our Internet of Things into our algorithm for three main purposes including providing the notifications to an external system, for example, a map navigation engine out of the scope for this project but a future extension for route optimisation and waste vehicle tracking; extracting and reporting the actionable insights from the underlying data; and consuming the extracted data for predictive forecasting to draw out the unknown patterns of waste fill levels within various geographical locations and again send out triggers and notification to external systems for example a waste collection authority who can efficiently schedule the waste collection vehicles and optimise the route. To achieve the above mentioned outcomes, we propose a framework that is agnostic of the hardware that it connects to and can effectively interface with a wide variety of hardware keeping a level of abstraction in the architecture.

With the digital transformation of the logistics industry, smart logistics algorithms have become a core technology to improve efficiency and reduce costs. This paper reviews the development history of traditional logistics technology and discusses the key role of technologies such as the Internet of Things, big data analysis, artificial intelligence, and automation in logistics technology innovation. It focuses on the application of intelligent logistics algorithms in path optimization, intelligent scheduling, data mining and prediction, and intelligent warehousing. To solve the problem of inconsistency between training and testing objectives, this paper proposes DRL4Route, a deep reinforcement learning-based path optimization framework, and designs the DRL4Route-GAE model. Validated through extensive offline experiments and online deployments, the model significantly outperforms existing optimal benchmark methods on real datasets, improving the location deviation squared metrics and the top-three location prediction accuracy metrics. These research results provide important support to further promote the intelligent development of the logistics industry.

The fifth Generation (5G) mobile networks use millimeter Waves (mmWaves) to offer giga bit data rates. However, unlike microwaves, mmWave links are prone to user and topographic dynamics. They easily get blocked and end up forming irregular cell patterns for 5G. This in turn cause too early, too late, or wrong handoffs (HOs). To mitigate HO challenges, sustain connectivity and avert unnecessary HO, we propose a HO scheme based on Jump Markov Linear System (JMLS) and Deep Reinforcement Learning (DRL). JMLS is widely known to account for abrupt changes in system dynamics. DRL likewise emerges as an artificial intelligence technique for learning highly dimensional and time-varying behaviors. We combine the two techniques to account for time-varying, abrupt, and irregular changes in mmWave link behaviour by predicting likely deterioration patterns of target links. The prediction is optimized by meta training techniques that also reduces training sample size. Thus, the JMLS-DRL platform formulates intelligent and versatile HO policies for 5G. Results show our proposed prediction scheme about target link behavior post HO to be highly reliable. The scheme also averts unnecessary HOs thus ably supports longer dew time.

The enormous diversity of hair fibers found in human population is striking, while the basic structure of human fibers is often the same! But why do some fibers curl? Curling is related to the machinery and processes that produce different keratin filament orientation in relation to the axis of hair growth, influenced by developmental and molecular control. It is established that a curly hair shaft is produced by a curved hair follicle that have an angulated dermal implantation, with a bulb bent in the shape of golf club, while the outer root sheath is dissymmetrical; have an outer side that is convex bulging and thin and a concave side in which the cells express alpha-smooth muscle protein reflecting internal mechanical force. Yet, the exact cause of hair curling is not known. All the previous studies were short of impacting curly hair treatment. My hypothesis is that the curling mechanical force is generated by a hypersensitive contracted Arrector Pili muscle (APM) of the hair follicle. In curly hair, each APM fiber is shortened; pulling its attachment spot upward towards the epidermis and obliquely or laterally; curling the outer root sheath to produce the curved follicle described above. This contraction has the influence at a time when the young hair root is being formed and keratin microfibrils are being oriented and packed. APM is a multi-unit muscle wherein each fiber contracts independently to create the enormous diversity of curly hair types. APM contraction can be modulated by oral administration of selective smooth muscle relaxing treatment that relax the curved follicle from within. Further studies are needed to confirm the role of APM in shaping curly hair follicle including AI studies, i.e., computer-aided target identification to demonstrate APM active site that can interact with herbal small molecules.

Maritime surveillance technology faces challenges of astronomical size in this vast ocean world—challenges vital to global security and environmental protection alike. Such as these may be reversible or not. Remotely sensed monitoring also uses photos taken from satellites, synthetic aperture radar (SAR) equipment, and unmanned aerial vehicles (UAVs) to fully monitor maritime activities but also ferret out suspicious behaviour like piracy, bootleg shipments, and illegal fishing. This mixed-method approach--quantitatively and qualitatively——measures the efficiency of difficulties associated with, and legal implications behind aerospace-monitored maritime surveillance. The results show that remote sensing provides a great deal of sea-level situational awareness; however, there are also many complex technical, operational, and legal issues, including information overcrowding, complications in integration, privacy issues, as well as the need for worldwide legal harmonisation. It ends with recommendations on ways to ultimately increase capacity, how to write standard procedures for secondary development and proposals for revising laws to absorb these new technologies. Its goal, therefore, is to benefit the global anti-maritime crime effort. Ultimately, though, it dreams that we might construct an entirely marine environment where all marine activities are conducted both safely and freely, without fear.

Recently, Gohar et al. introduced a novel, local, and well-behaved fractional calculus. It possesses all the classical properties, and Its locality imposes simplicity and accuracy in modeling fractional order systems. In this article, we further develop the definitions and extend the classical properties of Gohar fractional calculus to address some of the open problems in Calculus. The fractional Gronwall's integral inequality, Taylor power series expansion, and Laplace transform are defined and applied to overcome some of the limitations in the classical integer-order calculus. The fractional Laplace transform is applied to solve Bernoulli-type logistic and Bertalanffy nonlinear fractional differential equations, and the criteria under which it can be applied to solve linear differential equations are investigated.

Smart Internet of Vehicles (IoVs) combined with Artificial Intelligence (AI) will contribute to vehicle decision-making in the Intelligent Transportation System (ITS). Multi-Vehicle Pursuit games (MVP), a multi-vehicle cooperative ability to capture mobile targets, is becoming a hot research topic gradually. Although there are some achievements in the field of MVP in the open space environment, the urban area brings complicated road structures and restricted moving spaces as challenges to the resolution of MVP games. We define an Observation-constrained MVP (OMVP) problem in this paper and propose a Transformer-based Time and Team Reinforcement Learning scheme (T3OMVP) to address the problem. First, a new multi-vehicle pursuit model is constructed based on decentralized partially observed Markov decision processes (Dec-POMDP) to instantiate this problem. Second, by introducing and modifying the transformer-based observation sequence, QMIX is redefined to adapt to the complicated road structure, restricted moving spaces and constrained observations, so as to control vehicles to pursue the target combining the vehicle’s observations. Third, a multi-intersection urban environment is built to verify the proposed scheme. Extensive experimental results demonstrate that the proposed T3OMVP scheme achieves significant improvements relative to state-of-the-art QMIX approaches by 9.66%~106.25%. Code is available at https://github.com/pipihaiziguai/T3OMVP.

Robotic systems are crucial in modern manufacturing. Complex assembly tasks require the collaboration of multiple robots. Their orchestration is challenging due to tight tolerances and precision requirements. In this work we set up two Franka Panda robots performing a peg-in-hole insertion task. We structure the control system hierarchically, planning the robots’s trajectories feedback-based with a central policy trained with reinforcement learning. These trajectories are executed by a low-level impedance controller on each robot. To enhance training convergence, we use reverse curriculum learning incorporating domain randomization, varying initial configurations of the task. After training, we test the system in a simulation, studying the impact of curriculum parameters on emerging process characteristics like process time and variance. Finally, we transfer the trained model to a real-world setup, comparing results with simulation as well as classical path planning and control approaches.

The transportation industry is rapidly transitioning from Internal Combustion Engine (ICE) based vehicles to Electric Vehicles (EVs) to promote clean energy. However, large-scale adoption of EVs can compromise the reliability of the power grids by introducing large uncertainty in the demand. Demand response with a controlled charge scheduling strategy for EVs can mitigate such issues. In this paper, a deep reinforcement learning- based charge scheduling strategy is developed for individual EVs by considering user’s dynamic driving behavior and charging preferences. The temporal dynamics of user’s anxiety about charging the EV battery is rigorously addressed. A dynamic weight allocation technique is applied to continuously tune user’s priority for charging and cost-saving with respect to charging duration. The sequential charging control problem is formulated as a Markov decision process, and an episodic approach to the deep deterministic policy gradient (DDPG) algorithm with target policy smoothing and delayed policy update techniques is applied to develop the optimal charge scheduling strategy. A real-world dataset that captures user’s driving behavior, such as arrival time, departure time, and charging duration, is utilized in this study. The extensive simulation results reveal the effectiveness of the proposed algorithm in minimizing energy cost while satisfying user’s charging requirements.

Deep reinforcement learning is one of the research hotspots in artificial intelligence and has been successfully applied in many research areas, however, the low training efficiency and high demand for samples are problems that limit the application To address these problems, a hierarchical episodic control model extending episodic memory to the domain of hierarchical reinforcement learning is proposed in this paper. The model is theoretically justified and employs a hierarchical implicit memory planning approach for counterfactual trajectory value estimation. Starting from the final step and recursively moving back along the trajectory, a hidden plan is formed within the episodic memory. Experience is aggregated both along trajectories and across trajectories, and the model is updated using a multi-headed backpropagation similar to bootstrapped neural networks. This model extends the parameterized episodic memory framework to the realm of hierarchical reinforcement learning and is theoretically analyzed to demonstrate its convergence and effectiveness. Experiments conducted in Four Room, Mujoco, and UE4-based active tracking , highlight that the hierarchical episodic control model effectively enhances training efficiency. It demonstrates notable improvements in both low-dimensional and high-dimensional environments, even in cases of sparse rewards.

Throughout this paper the impact of anxiety on adolescent athlete’s is highlighted through analyzing neurological and psychological aspects involved in sports performance. The neuroanatomy and neuropsychology of the human brain plays a major role in developing athlete’s that has only begun to be analyzed within recent years. Especially in young athletes, there are numerous factors that affect performance on the field that originate from the environment and training the adolescent athlete is involved in. Sports performance is viewed as a major physical factor such as training and lifting weights, however the psychological aspect of sports performance often goes unnoticed. Athletes can begin to develop mental fatigue and anxiety which inevitably can decrease their ability to perform. The athlete’s internal suffering begins to go unnoticed, as the only tangible factor is watching their performance decrease. This causes the athlete to push themselves further in training resulting in greater mental fatigue. This paper emphasizes the numerous factors neurologically that play a role in athletic performance which should be focused on in adolescent athletes as they navigate the challenges and competition that are apparent within athletics. It is critical to understand the processes and hormonal changes the body undergoes with training and playing a sport, this information can change the way adolescent sports are approached and advanced.

Improper lifting measures of long-span spatial steel structures during construction process may delay the construction period and even cause safety accidents. Few studies on long-span spatial steel structures considered time-varying mechanical characteristic during construction process. In order to achieve safe and efficient installation in a long-span spatial steel structure, this research presents a time-varying mechanical analysis of the synchronous and asynchronous integral lifting and a single and interlaced point’s asynchronous integral lifting analysis method of a long-span spatial steel structure. The results showed that in the case of asynchronous lifting of single point, the displacement influence on other members is related to the distance. The closer the distance from the lifting point, the greater the influence. In asynchronous integral lifting, the lifting point with large lifting force is first installed to the specified position, and the lifting point with small lifting force is installed to the specified position.

In our recent publications, based on strong and electroweak interactions, we have developed a completely new formula for estimating nuclear binding energy. With reference to currently believed Semi Empirical Mass Formula (SEMF), we call our formula as ‘Strong and Electroweak Mass Formula’ (SEWMF). Our formula constitutes 4 simple terms and only one energy coefficient of magnitude 10.1 MeV. First term is a volume term, second term seems to be a representation of free nucleons associated with electroweak interaction, third term is a radial term and fourth one is an asymmetry term about the mean stable mass number. In this paper, we make an attempt to understand and estimate the maximum binding energy associated with any mass number. It can be expressed as, for A > 4, $\left(BE\right)_A\cong \left[A-0.000935A^2-A^{1/3}-A^{-1/2}\right]$ MeV. We are working on refining the 4^th term with even-odd corrections, shell corrections and other microscopic corrections. Proceeding further, stable mass numbers and super heavy mass numbers can be understood with a relation of the form, $\left[\textrm{RoundOff}\left(\left(Z+2.9464\right)^{1.2}-1.7165\right)\right]\pm\left[0,1\right]\pm\left(2n\right)$ where $n \cong 0,1,2$. It needs a review with respect to even-odd proton numbers and other microscopic corrections.Very interesting point to be noted is that based on the concept of “binding energy per nucleon”, the most complicated Avogadro number and Unified atomic mass unit can be estimated in a unified approach.

This literature review investigates the morphological and behavioral significance of the horns of Carnotaurus sastrei, a theropod dinosaur from the Late Cretaceous period. Through a comprehensive analysis of fossil specimens, comparative anatomy, and biomechanical modeling, this review explores the evolutionary adaptations and potential functions of these distinctive cranial structures. The horns of Carnotaurus are hypothesized to serve multiple roles, including species recognition, thermoregulation, and intraspecific combat. Our findings suggest that its cranial horns likely served a primary role in species recognition, intimidating rivals and signaling mating readiness. It also could have been used occasionally in head-butting contests, potentially as a display of dominance, but likely did not play a role in thermoregulation. Additionally, we examine the ecological context in which Carnotaurus lived, assessing how environmental factors could have influenced the development of such features. The implications of this study extend to broader discussions on the functional morphology and adaptive strategies of theropod dinosaurs, highlighting the complex relationship between form and function in the evolutionary history of predatory dinosaurs like Carnotaurus sastrei.

In 1948, Skinner described the behavior of pigeons under response-independent schedules as “superstitious,” and proposed that the responses were reinforced by contiguous, adventitious food deliveries. Subsequently, response-independent schedules have been of interest to both basic and applied researchers, first to understand the mechanisms involved, and later, as “noncontingent reinforcement” (NCR) to reduce undesirable behavior. However, the potential superstitious effects produced by these schedules have been challenged, with some researchers arguing that antecedent variables play a significant role. This paper examines the evidence for adventitious reinforcement from both laboratory and applied research, the results of which suggest that antecedent, non-operant functions may be important in fully understanding the effects of NCR. We propose an applied-basic research synthesis, in which attention to potential non-operant functions could provide a more complete understanding of response-independent schedules. We conclude with a summary of the applied implications of the non-operant functions of NCR schedules.

With the proliferating of video surveillance system deployment and related applications, real-time video analysis is very critical to achieve intelligent monitoring, autonomous driving, etc. It is non-trivial to achieve high accuracy and low latency video stream analysis through the traditional cloud computing. In this paper, we propose a non-orthogonal multiple access (NOMA) based edge real-time video analysis framework with one edge server (ES) and multiple user equipments (UEs). A cost minimization problem composed of delay, energy and accuracy is formulated to improve the QoE of UEs. In order to efficiently solve this problem, we propose the joint video frame resolution scaling, task offloading, and resource allocation algorithm based on the Deep Q-Learning Network (JVFRS-TO-RA-DQN), which effectively overcomes the sparsity of the single-layer reward function and accelerates the training convergence speed. JVFRS-TO-RA-DQN consists of two DQN networks to reduce the curse of dimension, which respectively select the offloading and resource allocation action, the resolution scaling action. Experimental results show that JVFRS-TO-RA-DQN can effectively reduce the cost of transmission and computation, and have better performance in convergence compared to other baseline.

Ethnopharmacology experts face several challenges when identifying and retrieving documents and resources related to their scientific focus. The volume of sources that need to be monitored, the variety of formats utilized, the different quality of language use across sources, present some of what we call “big data” challenges in the analysis of this data. This study aims to understand if and how experts can be supported effectively through intelligent tools in the task of ethnopharmacological literature research. To this end, we utilize a real case study of ethnopharmacology research, aimed at the Southern Balkans and Coastal zone of Asia Minor. Thus, we propose a methodology for more efficient research in ethnopharmacology. Our work follows an “Expert-Apprentice” paradigm in an automatic URL extraction process, through crawling, where the apprentice is a Machine Learning (ML) algorithm, utilizing a combination of Active Learning (AL) and Reinforcement Learning (RL), and the Expert is the human researcher. ML-powered research improved 3.1 times the effectiveness and 5.14 times the efficiency of the domain expert, fetching a total number of 420 relevant ethnopharmacological documents in only 7 hours versus an estimated 36-hour human-expert effort. Therefore, utilizing Artificial Intelligence (AI) tools to support the researcher can boost the efficiency and effectiveness of the identification and retrieval of appropriate documents.

Event detection provides a crucial foundation for intelligent text analysis and supports numerous downstream applications. However, existing detection models often fail to meet the high accuracy requirements and typically require a human–machine collaborative detection mode involving “machine detection followed by manual audit and correction.” Analysis reveals that the recall rate of machine detection is a major factor limiting the efficiency of human–machine collaboration. To address this at its root and ensure high recall, an event trigger recommendation task is introduced, and a span-based recommendation model for Chinese, SpETR (Span-based Event Trigger Recommender), is developed. This model integrates confidence information from both trigger identification and event classification stages to recommend candidate triggers and their event types for each event. To enhance recall, a semantic boundary smoothing method is proposed, which assigns soft labels to spans around gold triggers based on semantic completeness and part-of-speech, combined with a modified focal loss function to make the model predictions smoother. Experiments on DuEE and a self-built dataset show that SpETR is most effective when the number of recommendations is set to 3, achieving recall rates of 97.76% and 98.70%, respectively, comprising improvements of 4.30% and 5.22% over the optimal baseline models. In terms of human–machine collaborative detection, SpETR saved 30.6% and 25.8% of time compared to traditional detection models on two datasets, and the average F1 score of the annotation results was also improved by 5.35% and 3.48%, respectively.

With the gradual development and integration of artificial intelligence into various industries, there is also a great range of integration in the financial industry. Therefore, this article focuses on the trend prediction model and financial risk management problems of deep reinforcement learning (DRL), one of the largest branches of artificial intelligence, in the cryptocurrency market. In addition, in the experimental part of this paper, the artificial intelligence machine learning Long short-term memory network (LSTM) model is used to make effective time series prediction and analysis on the relevant data of the cryptocurrency market, so as to make a large-scale analysis to improve the accuracy of market trend prediction and the effectiveness of risk management. In addition, in this experiment, technology-related indicators, emotional states of financial market customers and other content related to large language models are combined. While optimizing investment strategy by using deep reinforcement learning algorithm, machine learning prediction model is also used to capture the time dependence of financial market. The experimental results also show that the predicted results are consistent with the actual value. Therefore, the model has high practical application value in predicting the time series price trend of cryptocurrency in the financial market and indicates that the integrated DRL model framework can further optimize and manage the price and trading strategy of the financial market. Future research should focus on improving the LSTM model and incorporating more features to improve prediction accuracy and adapt to market changes.

The development of synthetic biology has enabled us to make massive progress on biotechnology and to approach research questions from a brand new perspective. In particular, the design and study of gene regulatory networks in vitro, in vivo and in silico, have played an increasingly indispensable role in understanding and controlling biological phenomena. Among them, it is of great interest to understand how associative learning is formed at the molecular circuit level. Noticeably, mathematical models have been increasingly used to predict the behaviors of molecular circuits. The Fernando’s model, which is thought to be one of the first works in this line of research using the Hill equation, attempted to design a synthetic circuit that mimics Hebbian learning in the neural network architecture. In this article, we carry out in-depth computational analysis of the model and demonstrate that the reinforcement effect can be achieved by choosing the proper parameter values. We also construct a novel circuit that can demonstrate forced dissociation, which was not observed in the Fernando’s model. Our work can be readily used as reference for synthetic biologists who consider implementing the circuits of this kind in biological systems.

In this paper, we consider a reconfigurable intelligent surface (RIS)-assisted integrated satellite-high altitude platform-terrestrial networks (IS-HAP-TNs) that can improve network performance by exploiting HAP's stability and RIS's reflection. Specifically, the reflector RIS is installed on the side of HAP to reflect signals from the multiple ground user equipments (UEs) to the satellite. To aim at maximising system sum rate, we jointly optimize the transmit beamforming matrix at the ground UEs and RIS phase shift matrix. Due to the limitation of the unit modulus of the RIS reflective elements constraint, the combinatorial optimization problem is difficult to tackle it effectively by traditional solving methods. Based on this, this paper studies deep reinforcement learning (DRL) algorithm to achieve online decision making for this joint optimization problem. In addition, it is verified through simulation experiments that the proposed DRL algorithm outperforms the standard scheme in terms of system performance and execution time, and higher computing speed, making real-time decision making truly feasible。

The Mining Industry has changed significantly in recent decades with the introduction of advanced technologies such as Artificial Intelligence (AI) and Machine Learning (ML). These innovations contribute to the creation of expert systems that help in optimizing processes, increasing the safety and sustainability of operations. This article is a literature review of modern AI and ML methods and technologies used in the Mining Industry. Discusses various intelligent and expert systems used to improve productivity, reduce operating costs, improve occupational safety, environmental sustainability, machine automation, predictive analytics, quality monitoring and control, and inventory and logistics management. The advantages and disadvantages of different approaches are analyzed, as well as their potential impact on the future of the Mining Industry. The review highlights the importance of integrating AI and ML into mining processes to achieve more efficient and safer solutions.

Network Slicing serves as an enabler for the new fifth generation (5G) of mobile networks era as it enhances connectivity and diverse network slice requirements across various user devices. Optimally orchestrating these 5G network slices to meet the varying demands of applications with diverse quality of service (QoS) requirements and stringent high performance metric remains a challenge today [1]. In this paper, a smart and novel machine learning (ML)-based framework, which dynamically sub-slice in real time and optimally orchestrate a 5G network slice is proposed. Our proposed framework incorporates series of supervised machine learning algorithms with the use of LazyPredict module to find and suggest the best-fit model and make intelligent decisions for the specific network data environment being monitored in real time or analysed with historical dataset. Additionally, the framework employs reinforcement-learning techniques to predict future network traffic data pattern thereby optimally allocating network resources and orchestrating alternative best fit bespoke network slice. This alternative network sub-slice is orchestrated as required to reduce the risk of denial of service and improve user experience. This ultimately forms the proposed new vertical Enterprise Network as a Service: Company Network as a Service (CNaaS) in this study. The proposed approach is extensively evaluated through simulations using python with MATLAB and Simulink. This identified the important features and performance metrics for the 5G network data and demonstrated the efficiency and improvement in key metrics such as latency, throughput, jitter, packet loss rate, and available network resource utilisation compared to conventional methods [1]. The results highlight the importance of leveraging the right machine-learning algorithm for each unique network as the demand requirements changes. The proposed Enhanced Sub-Slice (eSS) module with embedded series of machine learning algorithms using the Lazypredict shows that one size does not fit all and demonstrates the significance of leveraging advanced ML techniques and ML pipelines in the context of 5G network slice management and orchestration.

An experimental study was carried out to evaluate the ductility of reinforced concrete beams longitudinally reinforced with hybrid FRP-Steel bars. The specimens were fourteen reinforced concrete beams with and without hybrid reinforcement. The test variables were bars position, the ratio of longitudinal reinforcement, and the type of FRP bars. The beams were loaded up to failure using a four-point bending test. The performance of the tested beams was observed using the load-deflection curve obtained from the test. Numerical analysis using the fiber element model was used to examine the growth of neutral axis depth due to the effect of test variables. The neutral axis curves were then used to further estimate the neutral axis angle and neutral axis displacement index. The test results show that the position of the reinforcement greatly influences the flexural behavior of the beam with hybrid reinforcement. It was observed from the test that the flexural capacity of beams with hybrid reinforcement is 4% to 50% higher than that of the beams with conventional steel bars depending on bars position and the ratio of longitudinal reinforcement. The ductility decreases as the hybrid reinforcement ratio (Af/As) increases. This study also showed that a numerical model developed can predict the flexural behavior of beams with hybrid reinforcement with reasonable accuracy.

Adaptive bitrate (ABR) algorithms optimize the quality of streaming experiences for users in client-side video players especially in unreliable or slow mobile networks. Several rule-based heuristic algorithms can achieve stable performance, but they sometimes fail to adapt properly to changing network conditions. Fluctuating bandwidth may cause algorithms to default to behavior that creates a negative experience for the user. ABR algorithms can be generated with reinforcement learning, a decision-making paradigm in which an agent learns to make optimal choices through interactions with an environment. Training reinforcement learning algorithms for bitrate streaming requires building a simulator for an agent to experience interactions quickly; training an agent in the real environment is infeasible due to the long step times in real environments. This project explores using supervised learning to construct a world-model, or a learned simulator, from recorded interactions. A reinforcement learning agent trained inside of the learned model, rather than a simulator, can outperform rule-based heuristics. Furthermore, agents trained inside the learned world-model can outperform model-free agents in low sample regimes. This work highlights the potential for world-models to quickly learn simulators, and to be used to generate optimal policies.

This study investigates the latest advancements in utilizing machine learning techniques for predicting financial market movements. Financial markets are complex systems influenced by numerous factors, making accurate predictions challenging but highly valuable. The research explores various machine learning approaches, including supervised learning, unsupervised learning, reinforcement learning, and deep learning, to enhance the accuracy and reliability of financial market forecasts. The study provides a comprehensive review of current methodologies, highlighting key innovations and their applications in financial market prediction. Techniques such as neural networks, support vector machines, random forests, and ensemble methods are examined in the context of their ability to analyze vast amounts of financial data, identify patterns, and generate actionable insights. Additionally, the research delves into the integration of alternative data sources, such as social media sentiment, news articles, and macroeconomic indicators, to improve prediction models. Through empirical analysis and case studies, the study demonstrates the effectiveness of advanced machine learning models in predicting stock prices, currency exchange rates, and commodity prices. It also discusses the challenges associated with these techniques, including overfitting, data quality issues, and the need for interpretability in model outputs. Findings suggest that machine learning techniques can significantly enhance the predictive power of financial models, offering more accurate and timely insights for investors and financial analysts. The study concludes with recommendations for future research directions, emphasizing the importance of model transparency, the incorporation of diverse data sets, and the continuous adaptation of machine learning algorithms to evolving market conditions.

Recently, smart mobility intelligent traffic services have become a critical task in Intelligent Transportation Systems (ITS). This involves not only the use of advanced sensors and controllers but also the ability to respond to real-time traffic situations at intersections, alleviate congestion, and generate policies to prevent traffic jams. DRL (Deep Reinforcement Learning) provides a natural framework for processing tasks. In DRL, each intersection can control itself and coordinate with neighbors to achieve optimal network-wide policies. However, comparing approaches remains a challenging task due to the existence of numerous possible configurations. This research performs a critical comparison of various traffic controllers found in the literature. It demonstrates that using a nonlinear approximator for coordination mechanisms and enhancing observability at each intersection are key performance drivers.

In this paper, we study the performance of various deep reinforcement learning algorithms to enhance the energy management system of a microgrid. We propose a novel microgrid model that consists of a wind turbine generator, an energy storage system, a set of thermostatically controlled loads, a set of price-responsive loads, and a connection to the main grid. The proposed energy management system is designed to coordinate among the different flexible sources by defining the priority resources, direct demand control signals, and electricity prices. Seven deep reinforcement learning algorithms were implemented and are empirically compared in this paper. The numerical results show that the deep reinforcement learning algorithms differ widely in their ability to converge to optimal policies. By adding an experience replay and a semi-deterministic training phase to the well-known asynchronous advantage actor-critic algorithm, we achieved the highest model performance as well as convergence to near-optimal policies.

Gilbert's syndrome (GS) is a common hereditary condition characterized by mild increases in serum bilirubin levels due to inherited defects in bilirubin metabolism. This review, based on data from peer-reviewed articles and journals spanning from 1977 to January 2024, provides a comprehensive overview of over four decades of research on GS. Early studies primarily focused on defining the clinical and genetic characteristics of the syndrome, laying the foundation for subsequent investigations. More recent research has delved into the genetic mechanisms underlying the reduced expression of bilirubin UDP-glucuronosyltransferase, which has significantly enhanced our understanding of the pathogenesis of GS. Furthermore, recent studies have shed light on the clinical implications of GS, including its association with non-alcoholic fatty liver disease and mortality risk, highlighting the complex interplay between genetic factors, bilirubin metabolism, and clinical outcomes. Despite its generally benign nature, thorough research is essential to fully understand the impact of GS on patients' lives. This review emphasizes the importance of ongoing research to deepen our understanding of GS and its broader implications.

This article focuses on developing a continuous authentication system using behavioral biometrics to recognize users accessing computing devices. The user’s distinct behavioral biometric is captured through keystroke dynamics, and reward-based reinforcement learning (RL) ideas are applied to recognize them throughout the session. The suggested system adds an extra layer of security to traditional authentication methods, forming a robust continuous authentication system that can be added to static authentication systems. The methodology involves training a RL model to detect unusual user typing patterns and flag suspicious activity. Each user has an agent trained on their historical data, which is preprocessed and used to create episodes for the agent to learn from. The environment involves fetching observations and randomly corrupting them to learn out-of-order behavior. The observation vector includes both running features and summary features. The re-ward function is binary and minimalistic. The Principal Component Analysis (PCA) model is used to encode the running features, and the Double Deep Q-Network (DDQN) algorithm with a fully connected neural network is used as the policy net. The evaluation achieved an average training accuracy and EER (equal error rate) of 94.7% and 0.0126 and test accuracy and ERR of 81.06% and 0.0323 for all users when the number of encoder features was increased. Therefore, it is concluded that by continuously learning and adapting to changing behavior patterns, this approach can provide more secure and personalized authentication, lowering the possibility of unauthorized access and cyberattacks. Overall, the use of reinforcement learning and behavioral biometrics for continuous authentication has the potential to significantly enhance security in the digital age and are effective in identifying each user.

Ballastless track has high smoothness, but the laying requirements are strict. Therefore, the maximum span of cable-stayed bridges laying ballastless tracks is 392m. For laying ballastless track structures over larger spans, the deformation characteristics of long-span cable-stayed Bridges under complex loads are incomplete, and the interaction between long-span bridge-track structures is unclear. The influence of wavelength of cosine wave on the track- bridge mapping of different orbital structures was explored. The wavelength characteristics of vertical deformation under complex loads were investigated. The track-bridge integrated model for the cable-stayed bridge was establish to analyze the mapping relationship between rail and bridge and wavelength characteristics of deformation. Based on mapping relationships and wavelength characteristics of deformation, the Train-Track-Bridge dynamic simulation model was simplified. The results show that when the minimum wavelength of bridge deformation surpasses 6 m, 10 m, and 16 m, the rail deformation in the ballasted track, the longitudinal-connected track, and the unit slab-type ballastless track accurately mirrors the deformation of the bridge. For the span of bridge ranging from 200 m to 600 m, the wavelength of vertical deformation ranges from 21 to 1270 m under complex loads. During local loads, the vertical deformation below the 200 m wavelength constitutes a significant proportion near the pie. Considering the influence of the deformation on train vibration response, the Train-Bridge dynamic coupling model can be employed to treat the track structure as a load to reduce complexity of model and enhance the calculation efficiency.

A large body of evidence reports about the positive effects of physical activity in pathophysiological conditions associated with aging. Physical exercise, alone or in combination with other medical therapies, unquestionably causes reduction of symptoms in chronic non-transmissible diseases often leading to significant amelioration or complete healing. The molecular basis of this exciting outcome, however, remain largely obscure. Epigenetics, exploring at the interface between environmental signals and the remodelling of chromatin structure, promises to shed light on this intriguing matter possibly contributing to the identification of novel therapeutic targets. In this review, we shall focalize on the role of sirtuins (Sirts) a class III histone deacetylases (HDACs) which function has been frequently associated, often with a controversial role, to the pathogenesis of aging-associated pathophysiological conditions including cancer, cardiovascular, muscular, neurodegenerative, bones and respiratory diseases. Numerous studies, in fact, demonstrate that Sirt-dependent pathways are activated upon physical and cognitive exercises linking mitochondrial function, DNA structure remodelling and gene expression regulation to designed medical therapies leading to tangible beneficial outcomes. However, in similar conditions, other studies assign to sirtuins a negative pathophysiological role. In spite of this controversial effect, it is doubtless that studying sirtuins in chronic diseases might lead to an unprecedented improvement of life quality in the elderly.

This case report demonstrates the rare but potentially serious complication of pulmonary vein stenosis (PVS) diagnosed several years after radiofrequency ablation for atrial fibrillation. Pulmonary vein stenosis can remain asymptomatic until significant occlusion occurs, after which it can present with a variety of symptoms, mimicking numerous cardiovascular and pulmonary pathologies. Early diagnosis and treatment rely on consistent follow-up using appropriate diagnostic imaging modalities, which are paramount in preventing severe complications.

The document sets out three differentiated proposals for the transport of medicines through RPAS with their respective regulations, adapted to different operational categories: open category, special category and military scope. In the open category, an economical and efficient solution is prioritized for urban and rural areas, complying with weight and height regulations, in addition to guaranteeing the stability and quality of medications during transportation. The special category allows more complex operations and in more adverse conditions, such as densely populated areas and long routes, incorporating advanced security measures and training for personnel. In the military field, RPAS are used in conflict zones and hard-to-reach areas, ensuring the safe and rapid delivery of medical supplies under extreme conditions. Success stories in different countries are highlighted, demonstrating significant improvements in delivery times and risk reduction for health personnel.

We delve into the axion-dilaton model within the supergravity framework, with a specific focus on the intricacies of domain wall construction and stability. Employing holographic vitrification, we unravel the dynamics of domain wall formation in gauge theories featuring periodic vacuum structures. Our model, incorporating a QCD-like axion term and a stabilizing dilaton, undergoes scrutiny for conductivity variations under weak disorder. The investigation reveals the model's resilience, manifesting near-perfect conductivity under mild disorder conditions. However, the rigorous mathematical motivation for the holographic setup demands further elucidation. The scattered nature of our results prompts the necessity for a more systematic interpretation of QCD phenomena and conductivity transitions. This study contributes to the mathematical understanding of the axion-dilaton model's behavior, highlighting the imperative for a refined holographic framework and a more coherent interpretation of observed phenomena.

The problem addressed in this article consists of the motion control of a quadrotor, comprising model disturbances and uncertainties. In order to tackle model uncertainty, adaptive control based on reinforcement learning is used. The distinctive feature of this article, in comparison with other works on quadrotor control using reinforcement learning is the exploration of the underlying optimal control problem in which a quadratic cost and a linear dynamics allow an algorithm that runs in real time. Instead of identifying a plant model, adaptation is obtained by approximating the performance index given by the Q-function using directional forgetting recursive least squares that rely on a linear regressor build from quadratic functions of input/output data. The adaptive algorithm proposed is tested in simulation in a cascade control structure that drives a quadrotor. Simulations show the improvement in performance that results when the proposed algorithm is turned on.

Malaria is a long-standing disease and one of the top life-threatening diseases, yet its treatment has not changed, while the world has already embraced the Fourth Industrial Revolution (4IR). A wave of research on digitizing monitoring mechanisms of such a deadly disease has surfaced. Automated malaria screening is one of the detection processes which are gaining popularity in the research domain. However, the process needs to be coupled with other processes aiming a nationally or regionally contextualised malaria monitoring system. This paper proposes a digital malaria monitoring system in the context of an African country or region. One advantage of such a digital system is that is enables a novel disease spread forecasting model based on the dynamics of different malaria types. The architecture of the diagnosis system is described, and the disease spread model is mathematically modelled in terms of a SPITR (Susceptible- Protected- Infected-Treated- Recovered) epidemic model which is further analysed. The forecasting model is expressed and analysed whereas experiments are conducted using a Monte Carlo simulation method. The design of the monitoring system has inspired how predictions can be made in the complex cases such as mixed infections. Results show that reinforcing the model parameter makes a significant improvement on the disease prediction.

The need to develop novel lightweight materials and their manufacturing processes is sets out to meet the new aerospace, automotive and construction requirements. Within this context, this research work is proposed to develop a novel thermoplastic composite material with high mechanical properties. These composites will be based on thermoplastic matrixes made from polyamide and 35% short glass fiber filled-polyamide reinforced with different types of fabrics. As reinforcement, glass fiber fabrics will be used as the base. They will be treated with different processes, both chemical and physical, to promote adherence to the matrix. Textile overmoulding technology was selected for manufacturing these composites. This technology was primarily developed to manufacture aesthetic lined components and has achieved a great implantation. Once these new composites are manufactured, they will be submitted to different tests to evaluate their behavior regarding adhesion, impact strength and stiffness. It is expected an improvement on stiffness and impact absorption.

Most European cities are characterized by very large areas, often formed by buildings with low quality, from a series of point of view. The possibility of renovating them is strategic to improve both the quality of life and to the possibility of economic recovery for building companies. In the last decades, the attention of the scientific community has been addressed to the energy renovation, thanks to the strong activities of the European Community in this field. However, since a relevant part of the EC territory is at risk of earthquake, the possibility to combine both energy and seismic renovation actions may be strategic for many countries. In particular, Italy and Romania are linked by a common social tradition that springs from the Roman Empire. Nowadays, this link is stronger, thanks to common interests in social, cultural and business fields. Therefore, the investigation of possible synergies for seismic and energy renovation strategies may be really interesting for both countries. This paper represents the first step in this direction. After an overview of regulations and common practices for buildings with reinforced concrete structures, in both states, some key combined renovation interventions will be described and discussed, as well as advantages and perspectives of integrated renovation approaches.

The structural performance of reinforced concrete relies heavily on the bond between reinforcement and concrete. In nonlinear finite element analyses, bond is either modelled by merged, also called perfect bond, or coincident with slip, also called bond-slip, approaches. Here, the performance of these two approaches for the modelling of failure of reinforced concrete was investigated using a damage-plasticity constitutive model in LS-DYNA. Firstly, the influence of element size on the response of tension-stiffening analyses with the two modelling approaches was investigated. Then, the results of the two approaches were compared for plain and fibre reinforced tension stiffening and a drop weight impact test. It was shown that only the coincident with slip approach provided mesh insensitive results. However, both approaches were capable of reproducing the overall response of the experiments in the form of load and displacements satisfactorily for the meshes used.

As Chronic Non-Communicable Diseases (CNCD) increase, preventive approaches become more crucial. In this review, Calorie Restriction (CR) effects on human beings were evaluated, comparing benefits and risks of different CR diets: classic CR vs ketosis-inducing diets, including Intermittent Fasting (IF), Classic Ketogenic Diet (CKD), Fasting Mimicking Diet (FMD), Very-Low-Calorie Ketogenic Diet (VLCKD) and Spanish Ketogenic Mediterranean Diet (SKMD). Special emphasis on Insulin Resistance (IR) was placed, as it mediates Metabolic Syndrome (MS), a risk factor for CNCD, and predicts MS diagnosis. CR is the most robust intervention known to increase lifespan and health span, with high evidence and known biochemical mechanisms. CR improves cardiometabolic risk parameters, boosts exercise insulin sensitivity response, and there may be benefits of implementing moderate CR on healthy young and middle-aged individuals. However, there is insufficient evidence to support long-term CR. CKD is effective for weight and MS management, and may have additional benefits such as prevention of muscle loss and appetite control. SKMD has extreme significance benefits for all the metabolic parameters studied. Studies show inconsistent benefits of IF compared to classic CR. More studies are required to study biochemical parameters, reinforce evidence, identify risks, and seek effective and safe nutritional CR approaches.

Epoxy asphalt (EA) concrete is widely used in constructing long-span steel bridge pavements (SBDPs). This study aims to derive a fatigue damage evolution law, conducting an experimental investigation of SBDP. First, a general theoretical form of the fatigue damage evolution law of materials is established based on the thermal motion of atoms. Then, fatigue experiments demonstrate that this evolution law well represents the known damage–life relationships of SBDP. Taking into account the experimental relationships between damage and fatigue life under symmetrical cyclic loadings with different overload amplitudes and temperature variations, a detailed damage evolution law is deduced. Finally, the role of damage accumulation is discussed on the basis of the proposed damage evolution law for the extreme situation of heavy overload and severe environments. The results show that both heavy loading and falling temperatures increase the fatigue damage of SBDP considerably; therefore, SBDP should avoid heavy loading combined with winter temperatures. EA shows a fatigue life two to three times longer than that of modified matrix asphalt (SMA) or guss asphalt (GA). For the same thickness, EA pavement is demonstrated to be more suitable for an anti-fatigue design of large-span SBDP under high traffic flows and low temperatures.

In the evolving landscape of Portfolio Management (PM), the fusion of advanced machine 1 learning techniques with traditional financial methodologies has opened new avenues for innovation. 2 Our study introduces a cutting-edge model combining Deep Reinforcement Learning (DRL) with 3 a Non-stationary Transformer architecture. This model is specifically designed to decode complex 4 patterns in financial time series data, enhancing portfolio management strategies with deeper insights 5 and robustness. It effectively tackles the challenges of data heterogeneity and market uncertainty, 6 key obstacles in PM. Our approach integrates key macroeconomic indicators and targeted news 7 sentiment analysis into its framework, capturing a comprehensive picture of market dynamics. This 8 amalgamation of varied data types addresses the multifaceted nature of financial markets, enhancing 9 the model’s ability to navigate through the complexities of asset management. Rigorous testing 10 demonstrates the model’s efficacy, highlighting the benefits of blending diverse data sources and 11 sophisticated algorithmic approaches in mastering the nuances of PM.12

The affinity constant, also known as the equilibrium constant, binding constant, equilibrium association constant, or the reciprocal value, the equilibrium dissociation constant (K_d), can be considered as one of the most important characteristics for any antibody-antigen pair. Many methods based on different technologies have been proposed and used to determine this value. However, since a very large number of publications and commercial datasheets do not include this information, significant obstacles in performing such measurements seem to exist. In other cases where such data are reported, the results have often proved to be unreliable. This situation may indicate that most of the technologies available today require a high level of expertise that does not seem to be available in many laboratories. In this paper, we present a simple approach based on standard immunoassay technology that is easy and quick to perform. It relies on the effect that the molar IC₅₀ approaches the K_d value in the case of infinitely small concentrations of the reagent concentrations. A two-dimensional dilution of the reagents leads to an asymptotic convergence to K_d. The approach has some similarity to the well-known checkerboard titration used for the optimization of immunoassays. A well-known antibody against the FLAG peptide, clone M2, was used as a model system and the results were compared with other methods. This approach could be used in any case where a competitive assay is available or can be developed. The determination of an affinity constant should belong to the crucial parameters in any quality control of antibody-related products and assays and should be mandatory in papers using immunochemical protocols.

To further enhance the intelligence level of coal mining faces and achieve autonomous derivation, learning, and optimization of shearer navigation cutting, this paper proposes the methods of shearer digital twin navigation cutting motion planning based on the concept of shearer autonomous navigation cutting technology and intelligent coal mining face digital twins. The study includes the digital twin theory and the construction method of the shearer digital twin navigation cutting motion planning system based on this theory. Based on the digital twin theory, a shearer digital twin navigation cutting motion planning system was constructed. This system supports the service functions of shearer cutting digital twin, dynamic navigation map digital twin, reinforcement learning environment construction, and motion planning through the physical perception layer, comprehensive data layer, and digital-model fusion analysis layer. Finally, by comparing the effects of the DQN-NAF and DDPG deep reinforcement learning algorithms in the shearer motion planning task within the constructed digital twin environment, the results show that the DQN-NAF algorithm demonstrates better performance and stability in solving the shearer digital twin motion planning task.

In this work, we propose an approach for the autonomous navigation of mobile robots using fusion of sensor data by a Double Deep Q-Network with collision avoidance by detecting moving people via computer vision techniques. We evaluate two data fusion methods for the proposed autonomous navigation approach: Interactive and Late fusion strategy. Both are used to integrate mobile robot sensors through the following sensors: GPS, IMU and, an RGB-D camera. The proposed collision avoidance module is implemented along with the sensor fusion architecture in order to prevent the autonomous mobile robot from colliding with moving people. The simulation results indicate a significant impact on the success of completing the proposed mission by the mobile robot with the fusion of sensors, indicating a performance increase (success rate) of 27% in relation to navigation without sensor fusion. With the addition of moving people in the environment, deploying the people detection and collision avoidance security module has improved about 14% the success rate when compared to that of the autonomous navigation approach without the security module. Video was developed with robot navigation using the DDQN-Late Fusion https://www.loom.com/share/684afa6a5b0148afadc9a200ab9f3483.

We discuss the implementation of a suite of virtual diagnostics at the FACET-II facility currently under commissioning at SLAC National Accelerator Laboratory. The diagnostics will be used for prediction of the longitudinal phase space along the linac, spectral reconstruction of the bunch profile and non-destructive inference of transverse beam quality (emittance) using edge radiation at the injector dogleg and bunch compressor locations. These measurements will be folded in to adaptive feedbacks and ML-based reinforcement learning controls to improve the stability and optimize the performance of the machine for different experimental configurations. In this paper we describe each of these diagnostics with expected measurement results based on simulation data and discuss progress towards implementation in regular operations.

Natural fibers are being increasingly used in different areas of engineering, including as composite reinforcement. Among these fibers, carauá stands out for its good mechanical properties and adherence to resin. Nevertheless, little is known about the behavior of this material in the manufacture of a composite or whether classic failure theory can be used in this case. In this context, the present study assesses the mechanical properties of two laminas made of unidirectional curauá fiber with volumetric fiber percentages of 30 % and 22 %, and compares the results with the values obtained for four failure criteria reported in the literature, using analysis of variance (ANOVA). To that end, tensile tests were conducted in the direction of the fiber and at other loading angles, in addition to iosipescu shear tests. The results show that the maximum stress criterion does not represent the failure behavior of these materials and that the best was the Hashin criterion.

Composite fiber materials are superior materials due to their high strength and light weight. Composites reflect the properties of their constituents, which is proportional to the volume fraction of each phase. There are different fiber reinforcement types and each affects it’s flexural, tensile and compression strength. When selecting a composite for a specific application, the forces excreted on the composite must be known in order to determine the reinforcement type. Unidirectional fiber reinforcement will allow very strong load resistance but only in one direction where as a random orientated fiber reinforcement can resist less load but can maintain this quota in all directions. These materials are said to be anisotropic. Certain composite fibers, taking into consideration there weights, are physically stronger than conventional metals. This research deals with the analysis of three composite materials with different reinforcement types, volume fraction and phase content. It was found that material A (glass epoxy) was the strongest in the longitudinal direction with a flexural strength of 534 MPa in the longitudinal and 420 MPa in the transverse direction. The flexural stresses of material B (glass silicone) and material C (glass polyester) where both found in the 120 to 135 MPa range. Differences were due to their differences in matrix composition and reinforcement type.

Research question/problem: This comprehensive review and comparative analysis aims to investigate the significance of enhancing oxygen provision in the context of COVID-19, focusing on the implications for healthcare policies and services.Background/context: The study addresses the shift in the oxygen dissociation curve of blood in COVID-19 patients, emphasizing the impact on oxygen delivery to tissues and the challenges it poses for clinical management. It also explores the disruption of healthcare systems due to the pandemic and the integration of social determinants of health in clinical environments.Methodology: The research incorporates a thorough literature review, comparative analysis of diverse approaches to oxygen delivery, and examination of successful interventions through case studies and best practices. Data collection techniques include the review of scholarly articles, case studies, and policy documents.Results/Key findings: The study highlights the imperative for augmented oxygen supply and the significance of maintaining appropriate oxygen saturation levels, especially in the context of respiratory illnesses like COVID-19. It also emphasizes the positive correlation between increased oxygen saturation levels and life expectancy, indicating potential health benefits of enhanced oxygen availability.Conclusions/Implications: The findings underscore the need for a structured approach to oxygen therapy and the importance of equitable distribution of oxygen to underserved regions. The implications of the research extend to enhancing healthcare policies and services regarding oxygen provision during the COVID-19 pandemic, especially in rural and remote areas.

This study investigated how positive (e.g., incentives) and negative (e.g., restrictions) reinforcement used in vaccination promotional messages and trust in government impact parental vaccine perceptions and decisions on vaccinating their children during a large-scale infectious disease crisis. Two hundred and eighty-six parents of unvaccinated teens participated in a 2 (reinforcement: positive vs. negative) × 2 (trust in government: high vs. low) between-subjects experimental survey. Positive reinforcement (e.g., gift cards for the vaccinated) was more effective for parents with low trust in the government to improve their attitudes and intentions toward child vaccination, while parents with high trust in the government were unaffected by reinforcement types. To the best of our knowledge, this study is among the first to examine the effects of both positive and negative reinforcement on vaccination perceptions and behavioral intentions through a controlled experiment and is the first study to do so in a large-scale infectious disease crisis setting. This research is expected to help public health professionals better prepare for and manage virus-induced public health crises in the future.

The conducted investigation encompassed the comprehensive integration of mechanical, envi-ronmental, chemical, and microstructural evaluations of a composite amalgamating sandy soil and a synthetic polymer at two distinct concentrations (2.5% and %) across multiple curing in-tervals (0, 1, 2, 4, 7, 15, 30, and 45 days). The studied soil originates from an environmentally sig-nificant protected area in Brazil. The implementation of mechanisms for soil improvement in the region must adhere to technical criteria without causing environmental harm. Direct shear testing was conducted, permeability was assessed, and microstructure analysis and XRD and XRF/EDX studies of both the soil and composites were conducted. It was observed that longer curing times yielded improved results in shear stress, friction angle, and cohesive intercept, with SP_5% exhibiting the highest values compared to the soil and SP_2.5%. Adding the polymeric so-lution to the soil contributed to cementation and cohesion gains in the substrate. Through mi-crostructural characterization, the polymer's role as a cementing agent for the grains is evident, forming a film on the grains and binding them together. Based on the analyses and studies con-ducted in the research, it can be concluded that there is technical feasibility for applying the pol-ymeric solution at both dosages in geotechnical projects.

Machine learning offers advanced tools for efficient management of radio resources in modern wireless networks. In this study, we leverage a multi-agent deep reinforcement learning (DRL) approach, specifically the Parameterized Deep Q-Network (DQN), to address the challenging problem of power allocation and user association in massive multiple-input multiple-output (M-MIMO) communication networks. Our approach tackles a multi-objective optimization problem aiming to maximize network utility while meeting stringent quality of service requirements in M-MIMO networks. To address the non-convex and nonlinear nature of this problem, we introduce a novel multi-agent DQN framework. This framework defines a large action space, state space, and reward functions, enabling us to learn a near-optimal policy. Simulation results demonstrate the superiority of our Parameterized Deep DQN (PD-DQN) approach when compared to traditional DQN and RL methods. Specifically, we show that our approach outperforms traditional DQN methods in terms of convergence speed and final performance. Additionally, our approach shows 72.2 % and 108.5 % improvement over DQN methods and RL method respectively in handling large-scale multi-agent problems in M-MIMO networks.

Shape Memory Alloy (SMA) have emerged as a revolutionary material that holds the potential to change the design and construction methods of buildings and other structures. One of the most promising applications of SMAs in civil engineering is their use as reinforcements for steel structures. This paper presents the latest research progress, opportunities and challenges of SMA in the field of steel structural reinforcement, both in terms of basic components and applications. In terms of components, the construction forms and working mechanisms of Fe-SMA strips, SMA/CFRP composite patches and SMA dampers are introduced. On this basis, the application of SMA in steel structures reinforcement is introduced, and its effect is analyzed from three aspects: crack restoration, seismic retrofitting and structural strengthening. Finally, the results of the current research are summarized and the shortcomings are analyzed, hoping to provide a reference for the research of SMA in the field of steel structures reinforcement.

This report gives a comprehensive summary of two problems about graph convolutional networks (GCNs): over-smoothing and heterophily challenges, and outlines future directions to explore.

The demand for explainable and transparent models increases with the continued success of reinforcement learning. In this article, we explore the potential of generating shallow decision trees (DT) as simple and transparent surrogate models for opaque deep reinforcement learning (DRL) agents. We investigate three algorithms for generating training data for axis-parallel and oblique DTs with the help of DRL agents ("oracles") and evaluate these methods on classic control problems from OpenAI Gym. The results show that one of our newly developed algorithms, the iterative training, outperforms traditional sampling algorithms, resulting in well-performing DTs that often even surpass the oracle from which they were trained. Even higher dimensional problems can be solved with surprisingly shallow DTs. We discuss the advantages and disadvantages of different sampling methods and insights into the decision-making process made possible by the transparent nature of DTs. Our work contributes to the development of not only powerful but also explainable RL agents and highlights the potential of DTs as a simple and effective alternative to complex DRL models.

With the rapid development of the Internet of Things (IoT), there are dramatic increasing number of devices in the network, which causes the challenge that only using infrastructure such as based station cannot provide service with all devices with high quality. Therefore, due to their flexibility and economy, unmanned aerial vehicles (UAV) are widely used to increase the performance of IoT networks. UAVs can not only provide communication services for IoT devices in the absence of a network, but they can also perform video surveillance, cargo transportation, pesticide spraying, and other specialized tasks. However, due to the complexity of the scenario and the need for real-time decision making, it is challenging to schedule UAVs in the network using traditional optimization methods, and growing attention has focused on using AI to optimize UAVs in the network. In this paper, we focus on the AI-enabled UAV optimization method in IoT networks and give a comprehensive scope on what and how to use AI-enabled methods to increase the performance of UAV-assisted IoT networks. Moreover, a brief analysis of the challenges of using AI methods in IoT networks and some potential research directions are given.

The widespread adoption of electromobility constitutes one of the measures designed to reduce air pollution caused by traditional fossil fuels. However, several factors are currently impending this process, ranging from insufficient charging infrastructure, battery capacity, long queueing and charging time, to psychological factors. On top of range anxiety, the frustration of the EV drivers is further fueled by the lack the uncertainty of finding an available charging point on their route. To address this issue, we propose a solution that comes to bypass the limitations of the Reserve now function of the OCPP standard, enabling drivers to make charging reservations for the upcoming days, especially when planning a longer trip. We created an algorithm that generates reservation intervals based on the charging station's reservation and transaction history. Subsequently, we ran a series of test cases that yielded promising results, with no overlapping reservations.

This study delves into the intricate interplay between cosmology and gravity in dilaton-based models, enhancing our understanding. Exploring boundary theories coupled to gravity, we derive a four-dimensional effective theory from the cosmological constant. Dilaton gravity is then scrutinized, revealing elegant solutions and insights into brane embedding. The analysis extends to probe scalar fields, employing the WKB approximation in both conformal and non-conformal cases. In summary, this work contributes to a nuanced understanding of the complex relationship between cosmology and gravity in dilaton-based frameworks.

Effective communication in automated chat systems hinges on the ability to understand and respond to context. Traditional models often struggle with determining when additional context is necessary for generating appropriate responses. This paper introduces Context-Aware BERT (CA-BERT), a transformer-based model specifically fine-tuned to address this challenge. CA-BERT innovatively applies deep learning techniques to discern context necessity in multi-turn chat interactions, enhancing both the relevance and accuracy of responses. We describe the development of CA-BERT, which adapts the robust architecture of BERT with a novel training regimen focused on a specialized dataset of chat dialogues. The model is evaluated on its ability to classify context necessity, demonstrating superior performance over baseline BERT models in terms of accuracy and efficiency. Furthermore, CA-BERT's implementation showcases significant reductions in training time and resource usage, making it feasible for real-time applications. The results indicate that CA-BERT can effectively enhance the functionality of chatbots by providing a nuanced understanding of context, thereby improving user experience and interaction quality in automated systems. This study not only advances the field of NLP in chat applications but also provides a framework for future research into context-sensitive AI developments.

Crawling robots are the focus of intelligent inspection research, and the main feature of this type of robot is the flexibility of in-plane attitude adjustment. The crawling robot HIT_Spibot is a new type of steam generator heat transfer tube inspection robot with a unique mobility capability different from traditional quadrupedal robots. This paper introduces a hierachical motion planning approach for HIT_Spibot, aiming to achieve efficient and agile maneuverability. The proposed method integrates three distinct planners to handle complex motion tasks: a nonlinear optimization-based base motion planner, a TOPSIS-based base orientation planner, and a Mask-D3QN (MD3QN) algorithm-based gait motion planner. Initially, the robot's base and foot workspace are delineated through envelope analysis, followed by trajectory computation using Larangian methods. Subsequently, the TOPSIS algorithm is employed to establish an evaluation framework conducive to foundational turning planning. Finally, the MD3QN algorithm trains foot points to facilitate robot movement along predefined paths. Experimental results demonstrate the method's adaptability across diverse tube structures, showcasing robust performance even in environments with random obstacles. Compared to the D3QN algorithm, MD3QN achieves a 100% success rate, enhances average overall scores by 6.27%, reduces average stride lengths by 39.04%, and attains a stability rate of 58.02%. These results not only validate the effectiveness and practicality of the method but also showcase the significant potential of HIT_Spibot in the field of industrial inspection.

The shift towards autonomous excavators in construction and mining is a significant leap towards enhancing operational efficiency and ensuring worker safety. However, it presents challenges such as the need for sophisticated decision-making and environmental perception due to complex terrains and diverse conditions. Our study introduces the E-GTN framework, a novel approach tailored for autonomous excavation that leverages advanced multisensor fusion and a custom-designed convolutional neural network to address these challenges. Results demonstrate that GridNet effectively processes grid data, enabling the reinforcement learning algorithm to make informed decisions, thereby ensuring efficient and intelligent autonomous excavator performance. The study concludes that the E-GTN framework offers a robust solution for the challenges in unmanned excavator operations, providing a valuable platform for future advancements in the field.

Thin people who undergo implant insertion into the chest wall face a high risk of implant exposure to the external environment. Five months after an 8-year-old boy underwent implantable cardioverter-defibrillator (ICD) implantation in a subcutaneous pocket in the left anterolateral chest wall to manage long QT syndrome, ICD replacement became necessary owing to exposure risk from distal and lateral thinning of the ICD pocket. Pocket rupture and exposure would increase the risk of infection; therefore, we performed ICD removal and primary pocket closure. Two weeks later, a new suprafascial pocket was created, acellular dermal matrix (ADM) was attached to the inner wall to prevent ICD protrusion, and a new ICD was inserted. One year postoperatively, the ADM was engrafted, and no complications were observed. A thin subcutaneous layer increases the risk of ICD implantation complications. Inner wall strengthening with ADM can help prevent pocket rupture.