Change detection with heterogeneous remote sensing images (Hete-CD) plays an important role in practical applications, especially when homogenous remote sensing images are unavailable. However, bitemporal heterogeneous remote sensing images (HRSIs) cannot compare directly to measure change magnitude, and many deep learning methods require large amounts of samples to train the module. Moreover, labeling many samples for land cover change detection with HRSIs is time-consuming and labor-intensive. Therefore, acquiring satisfactory performance of Hete-CD remains a challenge for deep learning networks with very small training samples. In this study, we promote a novel deep-learning framework for Hete-CD to obtain satisfactory performance while the initial samples are very small. We initially design a multiscale network with select kernel-attention module to focus on capturing different change targets with various sizes and shapes. Then, a simple yet effective non-parametric sample-enhanced algorithm based on the Pearson correlation coeffi-cient is promoted to explore potential samples around each initial sample. Finally, the proposed network and sample-enhanced algorithm are fused into one iterative framework to improve the change detection performance with very small samples. Experimental results conducted on four pairs of actual HRSIs indicated that the proposed framework can achieve competitive accuracies with very small samples for initialization when compared with some state-of-the-art methods. For example, the improvement is approximately 3.38% and 1.99% when compared with the selected traditional methods and deep learning methods, respectively.

The used car market has a high global economic importance, with more than 35 million cars sold yearly. Accurately predicting prices is a crucial task for both buyers and sellers to facilitate informed decisions in terms of opportunities or potential problems. Although various Machine Learning techniques have been applied to create robust prediction models, a comprehensive approach has yet to be studied. This research introduces two datasets from different markets, one with over 300,000 entries from Germany to serve as a training base for deep prediction models and a second dataset from Romania containing more than 15,000 car quotes used mainly to observe local traits. As such, we include extensive cross-market analyses by comparing the emerging Romanian market versus one of the world’s largest and most developed car markets, Germany. Our study uses several neural network architectures that capture complex relationships between car model features, individual add-ons, and visual features to predict used car prices accurately. Our models achieved a high R2 score exceeding 0.95 on both datasets, indicating their effectiveness in estimating used car prices. Moreover, we experimented with advanced convolutional architectures to predict car prices based solely on visual features extracted from car images. This approach exhibited transfer-learning capabilities, leading to improved prediction accuracy, especially since the Romanian training dataset is limited. Our experiments highlight the most important factors influencing the price, while our findings have practical implications for buyers and sellers in assessing the value of vehicles. At the same time, the insights gained from this study enable informed decision-making and provide valuable guidance in the used car market.

The problem of optimal scheduling in a system with parallel queues and a single server has been extensively studied in the queueing theory. However, such systems have mostly been studied by assuming homogeneous attributes of arrival and service processes, or in heterogeneous case a Markov random process was usually assumed. In this paper, we explore the possibility of combining simulation and neural network paradigms for optimal scheduling in a heterogeneous system with an arbitrary inter-arrival and service time distributions. The service of any queue by the server is exhaustive. Further routing of the server after the queue has been emptied is as instructed by the trained neural network. Any change in the queue that currently has a server is accompanied by a switching cost. The simulated annealing optimization algorithm which is used to optimize the parameters of the neural network was adapted for numerically evaluated average cost function. A Markov decision problem was formulated for the corresponding markovian queueing system to verify the results of solving the optimization problem. In addition to comparing the actual values of the average cost, the parameter values of the neural network trained on the optimal control policy of the Markov model are also used to determine the domains of parameters required for the simulated annealing defined on a finite discrete domain. Numerical analysis shows the effectiveness of this approach. Moreover, a comparison of the results for different distributions illustrates the insensitivity of the optimal control policy to the shape of the distributions for the same two first moments. Thus, the optimal control policy for an exponential model can be used as a suboptimal one for the general system.

The exponential growth of digital media content has introduced new challenges in managing and classifying internet traffic. Digital media traffic is composed of various applications such as video, audio, social media, and search, and its data structure is complex, incorporating a vast array of features. The classification of traffic data is a crucial aspect of internet traffic management and network security, and it forms the basis for several scenarios, including content distribution, advertising recommendations, and data analysis. Traditional classification methods rely mainly on deep packet inspection and port-based techniques, which have become increasingly ineffective due to the rapid evolution of network traffic. To address this issue, this study proposes a machine learning-based traffic classification method aimed at enhancing the accuracy and efficiency of digital media traffic classification to meet the current needs of traffic management and network security. The paper also analyzes and evaluates the classification effect and prediction capability of various algorithms under different training set sizes to validate the feasibility and effectiveness of the proposed method. The result demonstrates that the neural network algorithm has superior classification and prediction capabilities compared to the decision tree and support vector machine algorithms. Furthermore, our proposed method achieves the highest accuracy of 96.88% with a large training sample of 40,000 data streams, proving its superiority in handling high-dimensional data and complex datasets. The research results are significant for the development of digital media traffic classification and prediction methods and are expected to be applied in practical scenarios.

Modern hyperspectral imaging technologies generate enormous datasets that could potentially transmit a wealth of information, but such a resource presents numerous difficulties for data analysis and interpretation. Deep learning techniques undoubtedly provide a wide range of potential for solving both traditional imaging tasks and exciting new problems in the spatial-spectral domain. This is true in the primary application area of remote sensing, where hyperspectral technology originated and has made the majority of its progress, but it may be even more true in the vast array of now existing and developing application areas that make use of these imaging technologies. The current review advances on two fronts: on the one hand, it is directed at domain experts who desire an updated overview of how deep learning architectures might work in conjunction with hyperspectral acquisition techniques to address specific tasks in various application sectors. On the other hand, by providing them with a picture of how deep learning technologies are applied to hyperspectral data from (near)real-time perspective. The contributions of this review include the existence of these two points of view and the inclusion of opportunities and important problems associated with the development of future CHIME mission to be launched by European Space Agency (ESA).

With the development of deep learning technology, various structures and research methods for super-resolution restoration of natural images and document images have been introduced. In particular, a number of recent studies have been conducted and developed in image restoration using generative adversarial network. Super-resolution restoration is ill-posed problem because of some complex restraints such as a lot of high-resolution images being restored for the same low-resolution image and also difficulty in restoring noises like edges, light smudging, and blurring. In this study, we utilized the spatially adaptive de-normalization (SPADE) structure for document image restoration to solve previous problems such as edge unclearness, hardness to catch features of texts, and the image color transition. Consequently, it can be confirmed that the edge of the character and the ambiguous stroke are restored more clearly when contrasting with the other previously suggested methods. Also, the proposed method’s PSNR and SSIM scores are geting 8% and 15% higher, respectively, compared to the previous methods.

Currently, skin cancer is the most commonly diagnosed form of cancer in humans and is one of the leading causes of death in patients with cancer. Biopsy methods are an invasive research method and are not always available for primary diagnosis. Imaging methods have low accuracy and depend on the experience of the dermatologist. Artificial intelligence technologies can match and surpass visual analysis methods in accuracy, but they have the risk of a false negative response when a malignant pigmented lesion can be recognized as benign. One possible way to improve accuracy and reduce the risk of false negatives is to analyze heterogeneous data, combine different preprocessing methods, and use modified loss functions to eliminate the negative impact of unbalanced dermatological data. The paper proposes a multimodal neural network system with a modified cross-entropy loss function that is sensitive to unbalanced heterogeneous dermatological data. The accuracy of recognition in 10 diagnostically significant categories for the proposed system was 85.19%. The novelty of the proposed system lies in the use of cross-entropy loss when training the modified function with the help of weight coefficients. The introduction of weighting factors has reduced the number of false negative forecasts, as well as improved accuracy by 1.02-4.03 percentage points compared to the original multimodal systems. The introduction of the proposed multimodal system as an auxiliary diagnostic tool can reduce the consumption of financial and labor resources involved in the medical industry, as well as increase the chance of early detection of skin cancer.

Automatic target detection of remote sensing images (RSI) plays an important role in military reconnaissance, disaster monitoring, and target rescue. The core task of remote sensing target detection is to judge the target categories and complete precise location. However, the existing target detection algorithms have limited accuracy and weak generalization capability for remote sensing images with complex backgrounds. To achieve accurate detection of different categories targets in remote sensing images, this study presents a novel feature enhancement single shot multibox detector (FESSD) algorithm for remote sensing target detection. The FESSD introduces feature enhancement module and attention mechanism into the convolution neural networks (CNN) model, which can effectively enhance the feature extraction ability and nonlinear relationship between different convolution features. Specifically, the feature enhancement module is used to extract the multi-scale feature information, and enhance the model nonlinear learning ability; the self-learning attention mechanism (SAM) is used to expand the convolution kernel local receptive field, which makes the model extract more valuable features. In addition, the nonlinear relationship between different convolution features is enhanced using the feature pyramid attention mechanism (PAM). The advantage of FESSD over other state-of-the-art target detection methods is validated by experiments on the presented seven-class target detection dataset (SD-RSI) and the public DIOR dataset.

Blind deconvolution (BD) is one of the effective methods that help pre-process vibration signals and assist in bearing fault diagnosis. Currently, most BD methods design an optimization criterion and use frequency or time domain information independently to optimize a deconvolution filter. It recovers weak periodic impulses related to incipient faults. However, the random noise interference may cause the optimizer to overfit. The time-domain-based BD methods tend to extract fault-unrelated single peak impulse, and the frequency-domain-based BD methods tend to retain the maximum energy frequency component, which will lose the fault-related harmonics frequency components. To solve the above issue, we propose a hybrid criterion that combines the kurtosis for time domain optimization and the $G-l_1/l_2$ norm for the frequency domain. These two criteria are monotonically increasing and decreasing, so they mutually constrain to avoid overfitting. After that, we design a multi-task one-dimensional convolutional neural network with time and frequency branches to achieve an optimal solution for this hybrid criterion. The multi-task neural network realizes the simultaneous optimization of two domains. Experimental results show that our proposed method outperforms other state-of-the-art methods.

In recent years, the demand for coffee has increased tremendously. During the production process, green coffee beans are traditionally screened manually for defective beans before they are packed into coffee bean packages; however, this method is not only time-consuming but also increases the rate of human error due to fatigue. Therefore, this paper proposed a lightweight deep convolutional neural network (LDCNN) for the quality detection system of green coffee beans, which combined depthwise separable convolution (DSC), squeeze-and-excite block (SE block), skip block, and other frameworks. To avoid the influence of low parameters of the lightweight model caused by the model training process, rectified Adam (RA), lookahead (LA), and gradient centralization (GC) were included to improve efficiency; the model was also put into the embedded system. Finally, the local interpretable model-agnostic explanations (LIME) model was employed to explain the predictions of the model. The experimental results indicated that the accuracy rate of the model could reach up to 98.38% and the F1 score could be as high as 98.24% when detecting the quality of green coffee beans. Hence, it can obtain higher accuracy, lower computing time, and lower parameters. Moreover, the interpretable model verified that the lightweight model in this work is reliable, providing the basis for screening personnel to understand the judgment through its interpretability, thereby improving the classification and prediction of the model.

The computer network world is changing and the NetDevOps approach has brought the dynamics of applications and systems into the field of communication infrastructure. Businesses are changing and businesses are faced with difficulties related to the diversity of hardware and software that make up those infrastructures. The "Intent-Based Networking - Concepts and Definitions" document describes the different parts of the ecosystem that could be involved in NetDevOps. The recognize, generate intent, translate and refine features need a new way to implement algorithms. This is where artificial intelligence comes in.

Cancer stem cells (CSCs) are a tumor cell subpopulation that drives tumor progression and metastasis, leading to poor overall survival of patients. In colorectal cancer (CRC), hyper-activation of Wnt/β-catenin signaling by mutation of both Adenomatous polyposis coli (APC) and K-Ras increases the size of the CSC population. We previously showed that the CPD0857 inactivates Wnt/β-catenin signaling by promoting ubiquitin-dependent proteasomal degradation of β-catenin and Ras proteins, thereby decreasing proliferation and increasing apoptosis of CRC lines. CPD0857 also decreased growth and invasiveness of CRC cells harboring mutant K-Ras resistant to EGFR mAb therapy. Here, we show that CPD0857 treatment decreases proliferation and increases neuronal differentiation of neural progenitor cells (NPCs). CDP0857 effectively reduced expression of CSC markers and suppressed self-renewal capacity. CPD0857 treatment also inhibited proliferation and expression of CSC markers in D-K-Ras MT cells carrying K-Ras, APC and PI3K mutations, indicating inhibition of PI3K/AKT signaling. Moreover, CPD0857-treated xenograft mice showed regression of tumor growth and decreased numbers of CSCs in tumors. We conclude that CPD0857 could serve as the basis of a drug development strategy targeting CSCs activated through Wnt/β-catenin-Ras MAPK-PI3K/AKT signaling in CRCs.

Deep learning techniques, such as convolutional neural networks (CNN), generative adversarial networks (GAN), and graph neural networks (GNN), have over the past decade changed the ac-curacy of prediction in many diverse fields. In recent years, the application of deep learning tech-niques in computer vision tasks in pathology demonstrated extraordinary potential in assisting clinicians, automating diagnosis, and reducing costs for patients. Formerly unknown pathologi-cal evidence, such as morphological features related to specific biomarkers, copy number varia-tions, and other molecular features, were also able to be captured by deep learning models. In this paper, we review popular deep learning methods and some recent publications about their appli-cations in pathology.

Damage detection based on modal parameter changes becomes popular in the last decades. Nowadays are available robust and reliable mathematical relations to predict the natural frequency changes if damage parameters are known. Using these relations, it is possible to create databases containing a large variety of damage scenarios. Damage can be thus assessed by applying an inverse method. The problem is the complexity of the database, especially for structures with more cracks. In this paper, we propose two machine learning methods, namely the random forest (RF) and the artificial neural network (ANN) as search tools. The databases we developed contain damage scenarios for a prismatic cantilever beam with one crack and ideal and non-ideal boundary conditions. The crack assessment is made in two steps. First, a coarse damage location is found from the networks trained for scenarios comprising the whole beam. Afterward, the assessment is made involving a particular network trained for the segment of the beam on which the crack is previously found. Using the two machine learning methods, we succeed to estimate the crack location and severity with high accuracy for both simulation and laboratory experiments. Regarding the location of the crack, which is the main goal of the practitioners, the errors are less than 0.6%. Based on these achievements, we concluded that the damage assessment we propose, in conjunction with the machine learning methods, is robust and reliable.

This paper provides a methodology for the production of synthetic images for training neural networks to recognise shapes and objects. There are many scenarios in which it is difficult, expensive and even dangerous to produce a set of images that is satisfactory for the training of a neural network. The development of 3D modelling software has nowadays reached such a level of realism and ease of use that it seemed natural to explore this innovative path and to give an answer regarding the reliability of this method that bases the training of the neural network on synthetic images. The results obtained in the two proposed use cases, that of the recognition of a pictorial style and that of the recognition of migrants at sea, leads us to support the validity of the approach, provided that the work is conducted in a very scrupulous and rigorous manner, exploiting the full potential of the modelling software. The code produced, which automatically generates the transformations necessary for the data augmentation of each image, and the generation of random environmental conditions in the case of Blender and Unity3D software, is available under the GPL licence on GitHub. The results obtained lead us to affirm that through the good practices presented in the article, we have defined a simple, reliable, economic and safe method to feed the training phase of a neural network dedicated to the recognition of objects and features, to be applied to various contexts.

Decision tree algorithms have been among the most popular algorithms for interpretable (transparent) machine learning since the early 1980s. On the other hand, deep learning methods have boosted the capacity of machine learning algorithms and are now being used for non-trivial applications in various applied domains. But training a fully-connected deep feed-forward network by gradient-descent backpropagation is slow and requires arbitrary choices regarding the number of hidden units and layers. In this paper, we propose near-optimal neural regression trees, intending to make it much faster than deep feed-forward networks and for which it is not essential to specify the number of hidden units in the hidden layers of the neural network in advance. The key idea is to construct a decision tree and then simulate the decision tree with a neural network. This work aims to build a mathematical formulation of neural trees and gain the complementary benefits of both sparse optimal decision trees and neural trees. We propose near-optimal sparse neural trees (NSNT) that is shown to be asymptotically consistent and robust in nature. Additionally, the proposed NSNT model obtain a fast rate of convergence which is near-optimal up to some logarithmic factor. We comprehensively benchmark the proposed method on a sample of 80 datasets (40 classification datasets and 40 regression datasets) from the UCI machine learning repository. We establish that the proposed method is likely to outperform the current state-of-the-art methods (random forest, XGBoost, optimal classification tree, and near-optimal nonlinear trees) for the majority of the datasets.

The aim of this research is to expand recent developments in the mapping of pasture yield with remotely piloted aircraft systems to that of satellite-borne imagery. Up to date, spatially explicit and accurate information of the pasture resource base is needed for improved climate-adapted livestock rangeland grazing. This study developed deep learning predictive models of pasture yield, as total standing dry matter in tonnes per hectare (TSDM(tha−1)), from field measurements and both remotely piloted aircraft systems and satellite imagery. Repeated remotely piloted aircraft system structure measurements derived from structure from motion photogrammetry, provided measures of pasture biomass from many overlapping high-resolution images. Repeated remotely piloted aircraft system measures throughout a growing season, were modelled with persistent photosynthetic pasture responses from various Planet Dove high spatial resolution satellite image-derived vegetation indices. Pasture height modelling as an input to the modelling of yield was assessed against terrestrial laser scanning and reported correlation coefficients (R2) from 0.3 to 0.8 for both a coastal grassland and inland woodland pasture. Accuracy of the predictive modelling from both the remotely piloted aircraft system and the Planet Dove satellite image estimates of pasture yield ranged from 0.8 to 1.8 TSDM(tha−1). These results indicated that the practical application of repeated remotely piloted aircraft system derived measures of pasture yield can, with some limitations, be scaled-up to satellite-borne imagery to provide more temporally and spatially explicit measures of the pasture resource base.

Objectives: In order to solve the problem that most of the existing research focuses on the binary tactile attributes of objects,which ignores the tactile attribute strength and category recognition,an attribute strength and category recognition method based on convolutional neural network matrix-label is proposed. Methods:Firstly,in the data preparation stage,we preprocess the raw data and determine the matrix labels to build the haptic dataset.Secondly,in the feature extraction stage,we fuse the haptic data of two fingers and use the convolutional neural network to extract the attribute strength features.Finally,in the attribute strength and category recognition stage,all channel haptic data is fused to predict the attribute strength and category.Results:We compared with the multi-label convolutional neural network method in terms of elastic strength,hardness strength and category,and compared the attribute strength recognition capabilities of the two methods using novel objects outside the haptic dataset.The results show that the accuracy of the last 20 iterations of the matrix-label method has an average elastic strength of 96.73%,hardness strength of 97.34%,and category of 96.67%.The performance is better.When the Euclidean distance between the prediction of the novel object and the real label is less than 1,the accuracy of the elastic strength is best to reach 100%,and the hardness strength is best to reach 100%.The performance is better. Conclusions:The effectiveness of the method has been verified.Comparing with the convolutional neural network method,our method can effectively recognize the attribute strength and category of objects.

Intelligent systems are transforming the world, as well as our healthcare system. We propose a deep learning-based cough sound classification model that can distinguish between children with healthy versus pathological coughs such as asthma, upper respiratory tract infection (URTI), and lower respiratory tract infection (LRTI). In order to train a deep neural network model, we collected a new dataset of cough sounds, labelled with clinician's diagnosis. The chosen model is a bidirectional long-short term memory network (BiLSTM) based on Mel Frequency Cepstral Coefficients (MFCCs) features. The resulting trained model when trained for classifying two classes of coughs -- healthy or pathology (in general or belonging to a specific respiratory pathology), reaches accuracy exceeding 84\% when classifying cough to the label provided by the physicians' diagnosis. In order to classify subject's respiratory pathology condition, results of multiple cough epochs per subject were combined. The resulting prediction accuracy exceeds 91\% for all three respiratory pathologies. However, when the model is trained to classify and discriminate among the four classes of coughs, overall accuracy dropped: one class of pathological coughs are often misclassified as other. However, if one consider the healthy cough classified as healthy and pathological cough classified to have some kind of pathologies, then the overall accuracy of four class model is above 84\%. A longitudinal study of MFCC feature space when comparing pathologicial and recovered coughs collected from the same subjects revealed the fact that pathological cough irrespective of the underlying conditions occupy the same feature space making it harder to differentiate only using MFCC features.

We propose a chemical language processing model to predict polymers’ glass transition temperature (Tg) through a polymer language (SMILES, Simplified Molecular Input Line Entry System) embedding and recurrent neural network. This model only receives the SMILES strings of polymer’s repeat units as inputs and considers the SMILES strings as sequential data at the character level. Using this method, there is no need to calculate any additional molecular descriptors or fingerprints of polymers, and thereby, being very computationally efficient and simple. More importantly, it avoids the difficulties to generate molecular descriptors for repeat units containing polymerization point `*’. Results show that the trained model demonstrates reasonable prediction accuracy on unseen polymer’s Tg. Besides, this model is further applied for high-throughput screening on an unlabeled polymer database to identify high-temperature polymers that are desired for applications in extreme environments. Our work demonstrates that the SMILES strings of polymer’s repeat units can be used as an effective feature representation to develop a chemical language processing model for predictions of Tg. The framework of this model is general and can be used to construct structure-property relationships for other polymer’s properties.

Autonomous systems require identifying the environment and it has a long way to go before putting it safely into practice. In autonomous driving systems, the detection of obstacles and traffic lights are of importance as well as lane tracking. In this study, an autonomous driving system is developed and tested in the experimental environment designed for this purpose. In this system, a model vehicle having a camera is used to trace the lanes and avoid obstacles to experimentally study autonomous driving behavior. Convolutional Neural Network models were trained for Lane tracking. For the vehicle to avoid obstacles, corner detection, optical flow, focus of expansion, time to collision, balance calculation, and decision mechanism were created, respectively.

The paper presents the identification and classification of steel cord failures in the conveyor belt core based on an analysis of a two-dimensional image of magnetic field changes recorded using the Diagbelt system around scanned failures in the test belt. The obtained set of identified changes in images obtained for numerous devices parameters settings were the base for statistical analysis. It makes it possible to determine the Pearson’s linear correlation coefficient between the parameters being changed and the image of the failures. In the second stage of the research, artificial intelligence methods were applied to construct a multilayer neural network (MLP) and to teach its appropriate identification of damage. In both methods were used the same data sets, which made it possible to compare methods.

This study addresses the causal links between external factors and the main hydro-climatic variables. There is a gap in the literature on the description of a complete chain in addressing the structures of direct causal links of solar activity on the terrestrial variables. This is why, the present study uses the extensive facilities of the application of information theory in view of recent advances in different fields. Also, by other methods (e.g. neural networks) first are tested the existence non-linear links of solar-terrestrial influences on hydro-climate system. The results are promising related to the solar impact on terrestrial phenomena which is discriminant in space-time domain. The implications prove robust for determining the causal measure of climate variables under direct solar impact which makes it easier to consider solar activity in climate models, by appropriate parametrizations. This study found that hydro-climatic variables are sensitive to solar impact only for certain frequencies (periods) and these have a coherence with the Solar-Flux only for some lags of the Solar-Flux (in advance).

Extra virgin olive oil (EVOO) is the highest quality of olive oil and is characterized by highly beneficial nutritional properties. The large increase in both consumption and fraud, for example through adulteration, creates new challenges and an increasing demand for developing new quality assessment methodologies that are easier and cheaper to perform. As of today, the determination of olive oil quality is performed by producers through chemical analysis and organoleptic evaluation. The chemical analysis requires the advanced equipment and chemical knowledge of certified laboratories, and has therefore a limited accessibility. In this work a minimalist, portable and low-cost sensor is presented, which can perform olive oil quality assessment using fluorescence spectroscopy. The potential of the proposed technology is explored by analyzing several olive oils of different quality levels, EVOO, virgin olive oil (VOO), and lampante olive oil (LOO). The spectral data were analyzed using a large number of machine learning methods, including artificial neural networks. The analysis performed in this work demonstrates the possibility of performing classification of olive oil in the three mentioned classes with an accuracy of 100%. These results confirm that this minimalist low-cost sensor has the potential of substituting expensive and complex chemical analysis.

Different machine learning models (multiple linear regression, vector support machines, artificial neural networks and random forests) are applied to predict the monthly global irradiation (MGI) from different input variables (latitude, longitude and altitude of meteorological station, month, average temperatures, among others) of different areas of Galicia (Spain). The models were trained, validated and queried using data from three stations, and each best machine model was checked in two independent stations. The results obtained confirmed that the best ML methodology is the ANN model which presents the lowest RMSE value in the validation and querying phases 122.6·10kJ/(m2∙day) and 113.6·10kJ/(m2∙day), respectively, and predict conveniently for independent stations, 201.3·10kJ/(m2∙day) and 209.4·10kJ/(m2∙day), respectively. Given the good results obtained, it is convenient to continue with the design of artificial neural networks applied to the analysis of monthly global irradiation.

The question of molecular similarity is core in cheminformatics, and is usually assessed via a pairwise comparison based on vectors of properties or molecular fingerprints. We recently exploited variational autoencoders to embed 6M molecules in a chemical space, such that their (Euclidean) distance within the latent space so formed could be assessed within the framework of the entire molecular set. However, the standard objective function used did not seek to manipulate the latent space so as to cluster the molecules based on any perceived similarity. Using a set of some 160,000 molecules of biological relevance, we here bring together three modern elements of deep learning to create a novel and disentangled latent space, viz transformers, contrastive learning, and an embedded autoencoder. The effective dimensionality of the latent space was varied such that clear separation of individual types of molecules could be observed within individual dimensions of the latent space. The capacity of the network was such that many dimensions were not populated at all. As before, we assessed the utility of the representation by comparing clozapine with its near neighbours, and did the same for various antibiotics related to flucloxacillin. Transformers, especially when as here coupled with contrastive learning, effectively provide one-shot learning, and lead to a successful and disentangled representation of molecular latent spaces that at once uses the entire training set in their construction while allowing ‘similar’ molecules to cluster together in an effective and interpretable way.

This article describes techniques employed in the production of a synthetic dataset of driver telematics emulated from a similar real insurance dataset. The synthetic dataset generated has 100,000 policies that included observations about driver’s claims experience together with associated classical risk variables and telematics-related variables. This work is aimed to produce a resource that can be used to advance models to assess risks for usage-based insurance. It follows a three-stage process using machine learning algorithms. The first stage is simulating values for the number of claims as multiple binary classifications applying feedforward neural networks. The second stage is simulating values for aggregated amount of claims as regression using feedforward neural networks, with number of claims included in the set of feature variables. In the final stage, a synthetic portfolio of the space of feature variables is generated applying an extended SMOTE algorithm. The resulting dataset is evaluated by comparing the synthetic and real datasets when Poisson and gamma regression models are fitted to the respective data. Other visualization and data summarization produce remarkable similar statistics between the two datasets. We hope that researchers interested in obtaining telematics datasets to calibrate models or learning algorithms will find our work valuable.

This paper inquires on the options pricing modeling using Artificial Neural Networks to price Apple(AAPL) European Call Options. Our model is based on the premise that Artificial Neural Networks can be used as functional approximators and can be used as an alternative to the numerical methods to some extent, for a faster and an efficient solution. This paper provides a neural network solution for two financial models, the BlackScholes-Merton model, and the calibrated-Heston Stochastic Volatility Model, we evaluate our predictions using the existing numerical solutions for the same, the analytic solution for the Black-Scholes equation, COS-Model for Heston’s Stochastic Volatility Model and Standard Heston-Quasi analytic formula. The aim of this study is to find a viable time-efficient alternative to existing quantitative models for option pricing.

With the rapid expansion of intelligent resource-constrained devices and high-speed communication technologies, Internet of Things (IoT) has earned a wide recognition as the primary standard for low-power lossy networks (LLNs). Nevertheless, IoT infrastructures are vulnerable to cyber-attacks due to the constraints in computation, storage, and communication capacity of the endpoint devices. From one side, the majority of newly developed cyber-attacks are formed by slightly mutating formerly established cyber-attacks to produce a new attack tending to be treated as a normal traffic through the IoT network. From the other side, the influence of coupling the deep learning techniques with cybersecurity field has become a recent inclination of many security applications due to their impressive performance. In this paper, we provide a comprehensive development of a new intelligent and autonomous deep learning-based detection and classification system for cyber-attacks in IoT communication networks leveraging the power of convolutional neural networks, abbreviated as (IoT-IDCS-CNN). The proposed IoT-IDCS-CNN makes use of the high-performance computing employing the robust CUDA based Nvidia GPUs and the parallel processing employing the high-speed I9-Cores based Intel CPUs. In particular, the proposed system is composed of three subsystems: Feature Engineering subsystem, Feature Learning subsystem and Traffic classification subsystem. All subsystems are developed, verified, integrated, and validated in this research. To evaluate the developed system, we employed the NSL-KDD dataset which includes all the key attacks in the IoT computing. The simulation results demonstrated more than 99.3% and 98.2% of cyber-attacks’ classification accuracy for the binary-class classifier (normal vs anomaly) and the multi-class classifier (five categories) respectively. The proposed system was validated using k-fold cross validation method and was evaluated using the confusion matrix parameters (i.e., TN, TP, FN, FP) along with other classification performance metrics including precision, recall, F1-score, and false alarm rate. The test and evaluation results of the IoT-IDCS-CNN system outperformed many recent machine-learning based IDCS systems in the same area of study.

The calculation of echo intensity involved in the SAR image simulation is usually based on the electromagnetic formula or approximate formula derived under certain assumptions. However, the parameters used in these formulas are often difficult to obtain, and the formulas have errors with the actual situation. In this paper, a method of embedding deep neural network (DNN) into the simulation process based on ray tracing is proposed, so that the DNN model can be directly used to fit the calculation formula of echo intensity from real SAR images. Simulation results show that this method can obtain SAR images with high similarity.

Artificial Intelligence is dominated by Artificial Neural Networks (ANNs). Currently, the Batch Gradient Descent (BGD) is the only solution to train ANN weights when dealing with large datasets. In this article, a modification to the BGD is proposed which significantly reduces the training time and improves the convergence. The modification, called Instance Eliminating Back Propagation (IEBP), eliminates correctly-predicted-instances from the Back Propagation. The speedup is due to the elimination of unnecessary matrix multiplication operations from the Back Propagation. The proposed modification does not add any training hyperparameter to the existing ones and reduces the memory consumption during the training.

Radio Frequency Interference (RFI) detection and characterization play a critical role to in ensuring the security of all wireless communication networks. Advances in Machine Learning (ML) have led to the deployment of many robust techniques dealing with various types of RFI. To sidestep an unavoidable complicated feature extraction step in ML, this paper proposes an efficient end-to-end method using the latest advances in deep learning to extract the appropriate features of the RFI signal. Moreover, this study utilizes the benefits of transfer learning to determine both the type of received RFI signals and their modulation types. To this end, the scalogram of the received signals is used as the input of the pre-trained convolutional neural networks (CNN), followed by a fully-connected classifier. This study considers a digital video stream as the signal of interest (SoI), transmitted in a real-time satellite-to-ground communication using DVB-S2 standards. To create the RFI dataset, the SoI is combined with three well-known jammers namely, continuous-wave interference (CWI), multi- continuous-wave interference (MCWI), and chirp interference (CI). This study investigated four well-known pre-trained CNN architectures, namely, AlexNet, VGG-16, GoogleNet, and ResNet-18, for the feature extraction to recognize the visual RFI patterns directly from pixel images with minimal preprocessing. Moreover, the robustness of the proposed classifiers is evaluated by the data generated at different signal to noise ratios (SNR).

The paper demonstrates the analysis of Corona Virus Disease based on a CNN probabilistic model. It involves a technique for classification and prediction by recognizing typical and diagnostically most important CT images features relating to Corona Virus. The main contributions of the research include predicting the probability of recurrences in no recurrence (first time detection) cases at applying our proposed Convolution neural network structure. The Study is validated on 2002 chest X-ray images with 60 confirmed positive covid19 cases and (650 bacterial – 412 viral -880 normal) x-ray images. The proposed CNN compared with traditional classifiers with proposed CHFS feature extraction model. The experimental study has done with real data demonstrates the feasibility and potential of the proposed approach for the said cause. The result of proposed CNN structure has been successfully done to achieve 98.20% accuracy of covid19 potential cases with comparable of traditional classifiers.

The COVID-19 is a highly contagious viral infection which played havoc on everyone's life in many different ways. According to the world health organization and scientists, more testing potentially helps governments and disease control organizations in containing the spread of the virus. The use of chest radiographs is one of the early screening tests to determine the onset of disease, as the infection affects the lungs severely. This study will investigate and automate the process of testing by using state-of-the-art CNN classifiers to detect the COVID19 infection. However, the viral could of many different types; therefore, we only regard for COVID19 while the other viral infection types are treated as non-COVID19 in the radiographs of various viral infections. The classification task is challenging due to the limited number of scans available for COVID19 and the minute variations in the viral infections. We aim to employ current state-of-the-art CNN architectures, compare their results, and determine whether deep learning algorithms can handle the crisis appropriately. All trained models are available at https://github.com/saeed-anwar/COVID19-Baselines

Owing to the time-varying characteristics and nonlinearities of industrial processes, control has higher difficulties and results in challenges for advanced technology. In this paper, a self-tuning controller that includes a nonlinear proportional-integral-derivative (NPID) control function as well as a self-tuning function is proposed for first-order plus time delay (FOPTD) process control. The NPID control function is implemented using the nonlinear PID controller whose optimum parameters are adapted by a neural network (NN). The self-tuning function is able to identify the process dynamics using a short period of process behavior and tune NPID parameters based on the identified parameters. The advantage of the proposed method is validated with a set of simulation works on three processes and the comparison results are presented.

Exploiting the linguistic knowledge of the source language for neural machine translation (NMT) has recently achieved impressive performance on many large-scale language pairs. However, since the Turkish→English machine translation task is low-resource and the source-side Turkish is morphologically-rich, there are limited resources of bilingual corpora and linguistic information available to further improve the NMT performance. Focusing on the above issues, we propose a multi-source NMT approach that models the word feature in parallel to external linguistic features by using two separate encoders to explicitly incorporate linguistic knowledge into the NMT model. We extend the word embedding layer of the knowledge-based encoder to accommodate for each word’s linguistic annotations in the context. Moreover, we share all parameters across encoders to enhance the representation ability of the NMT model on the source language. Experimental results show that our proposed approach achieves substantial improvements of up to 2.4 and 1.1 BLEU scores in Turkish→English and English→Turkish machine translation tasks, respectively, which points to a promising way to utilize the source-side linguistic knowledge for the low-resource NMT.

Recently natural gas (NG) global market attracted much attention in case it is cleaner than oil, and simultaneously in most regions is cheaper than renewable energy sources. However, price fluctuations, environmental concerns, technological development, emerging unconventional resources, energy security challenges, and shipment are some of the forces that made the NG market more dynamic and complex. From a policy-making perspective, it is vital to uncover demand-side future trends. This paper proposed an intelligent forecasting model to forecast NG global demand, however investigating a multi-dimensional purified input vector. The model starts with a data mining (DM) step to purify input features, identify the best time lags, and to pre-process selected input vector. Then a hybrid artificial neural network (ANN) which equipped with genetic optimizer is applied to set up ANN’s characteristics. Among 13 available input features, six features (e.g. Alternative and Nuclear Energy, CO2 Emissions, GDP per Capita, Urban Population, Natural Gas Production, Oil Consumption) selected as the most critical feature via the DM step. Then, the hybrid prediction model is designed to extrapolate the consumption of future trends. The proposed model overcomes competitive models refer to different error based evaluation statistics. Besides, as the model proposed the best input feature set, results compared to the model which used the raw input set, with no DM purification process.

We first review the literature on machine behavior which may contain the interaction between machines, and then discuss the methodology and the related case involving collaboration between robots to instantiate the concept with AI empowerment. The navigation case aims to a security application with autonomous control and roadside agent embedded in a mobile edge computing structure. In the studied navigation based on the artificial potential field method, robots need to use position information to calculate the moving direction frequently. In the case of high motion speed as well as GPS positioning error, the path trajectory may show a sharp change of direction. In order to mitigate the trajectory oscillation, this paper proposes a path planning design where training process and motion direction prediction are integrated by using artificial neural network. The auxiliary navigation agent near multiple obstacles can first extract the past movement information of the robot and then determines whether there is a serious path jitter event. Computer simulation analysis shows that the combination of autonomous control and cooperative behavior can effectively reduce the path jitter so as to achieve a fast and safe path planning.

G protein-coupled receptors (GPCRs) play a key role in many cellular signaling mechanisms, and must select among multiple coupling possibilities in a ligand-specific manner in order to carry out a myriad of functions in diverse cellular contexts. Much has been learned about the molecular mechanisms of ligand-GPCR complexes from Molecular Dynamics (MD) simulations. However, to explore ligand-specific differences in the response of a GPCR to diverse ligands, as is required to understand ligand bias and functional selectivity, necessitates creating very large amounts of data from the needed large-scale simulations. This becomes a Big Data problem for the high dimensionality analysis of the accumulated trajectories. Here we describe a new machine learning (ML) approach to the problem that is based on transforming the analysis of GPCR function-related, ligand-specific differences encoded in the MD simulation trajectories into a representation recognizable by state-of-the-art deep learning object recognition technology. We illustrate this method by applying it to recognize the pharmacological classification of ligands bound to the 5-HT2A and D2 subtypes of class A GPCRs from the serotonin and dopamine families. The ML-based approach is shown to perform the classification task with high accuracy, and we identify the molecular determinants of the classifications in the context of GPCR structure and function. This study builds a framework for the efficient computational analysis of MD Big Data collected for the purpose of understanding ligand-specific GPCR activity.

In this work we present the results obtained with an artificial neural network (ANN) which we trained to predict the expected output of high-order harmonic generation (HHG) process, while exploring a multi-dimensional parameter space. We argue on the utility and efficiency of the ANN model and demonstrate its ability to predict the outcome of HHG simulations. In this case study we present the results for a loose focusing HHG beamline, where the changing parameters are: the laser pulse energy, gas pressure, gas cell position relative to focus and gas cell length. The physical quantity which we predict here using ANN is directly related to the total harmonic yield in a specified spectral domain (20-40 eV). We discuss the versatility and adaptability of the presented method.

Advanced electrode designs have made single-unit neural recordings commonplace among modern neuroscience research. However, single-unit resolution remains out of reach for the intrinsic neurons of the gastrointestinal system. Single-unit recordings of the enteric (gut) nervous system have been conducted in anesthetized animal models and excised tissue, but there is a large physiological gap between awake and anesthetized animals, particularly for the enteric nervous system. Here, we describe the opportunity for advancing enteric neuroscience offered by single-unit recording capabilities in awake animals. We highlight the primary challenges to microelectrodes in the gastrointestinal system including structural, physiological, and signal quality challenges.

The selection of the best mine production technical indicators is crucial to increasing a mine’s economic benefit and saving resources for sustainability. Therefore, this research proposes a ‘multi-objective optimization model’ based on a ‘fast and elitist Non-dominated Sorting Genetic Algorithm’ (NSGA-II) and ‘Artificial Neural Networks’ (ANN) for the optimization of production technical indicators in the entire geology, mining and beneficiation metal mine production processes. The multi-objective optimization model has decision variables including ‘cut-off grade,’ ‘industrial grade’ and ‘loss rate,’ with objectives being ‘economic benefit (profit)’ and ‘resource benefit (metal volume).’ First, the relationship between the technical indicators of mine production is studied. The REG model, MATLAB’s own ksdensity function and the BP neural network are used to calculate the ore weight, the probability density of grade distribution, the dilution rate, the concentration ratio and the concentrate grade, and to further calculate geological reserves, profit and metal volume. Then, the NSGA-II is applied to maximize profit and metal volume simultaneously. Finally, the model is applied to the Huogeqi copper mine. The optimization result is a set of multiple optimal solutions called Pareto optimal solutions. Compared with the plan data, the profit and metal volume of partial optimization results increased by 2.89% and 2.64% simultaneously. These Pareto optimal solutions can help decision makers in bettering the actual process of metal mine production.

Singing melody extraction is the task that identifies the melody pitch contour of singing voice from polyphonic music. Most of the traditional melody extraction algorithms are based on calculating salient pitch candidates or separating the melody source from the mixture. Recently, classification-based approach based on deep learning has drawn much attentions. In this paper, we present a classification-based singing melody extraction model using deep convolutional neural networks. The proposed model consists of a singing pitch extractor (SPE) and a singing voice activity detector (SVAD). The SPE is trained to predict a high-resolution pitch label of singing voice from a short segment of spectrogram. This allows the model to predict highly continuous curves. The melody contour is smoothed further by post-processing the output of the melody extractor. The SVAD is trained to determine if a long segment of mel-spectrogram contains a singing voice. This often produces voice false alarm errors around the boundary of singing segments. We reduced them by exploiting the output of the SPE. Finally, we evaluate the proposed melody extraction model on several public datasets. The results show that the proposed model is comparable to state-of-the-art algorithms.

Multidisciplinary Design Optimization (MDO) is the most active field in the design of current complex system engineering, which is possessed with such two difficulties as subsystem information exchange and analytical and computational complexity of systems. Therefore, an improved collaborative optimization algorithm based on ANN (artificial neural network) response surface was proposed dependent on the consistency constraint algorithm and concurrent subspace algorithm. As an optimization method with secondary structure, it satisfied only local constraints in discipline layer, but provided a coordinated mechanism for interdisciplinary conflict in system layer. Finally, it was applied in the multidisciplinary design optimization of autonomous underwater vehicle (AUV). As shown from the result, the MDO convergence stability and reliability of low resistance, low noise and high maneuvering performance of the AUV shape can be ensured by the improved collaborative optimization algorithm, thus verifying the effectiveness of the algorithm.

X-band marine radar captures signal reflected from the sea surface. Theoretical studies indicate that the initial unfiltered signal contains meaningful information including parameters related to wind waves. Traditional methods for estimating significant wave height (SWH) rely on the physical laws describing signal reflection from rough surfaces. However, recent studies suggest the feasibility of employing artificial neural networks (ANNs) for SWH approximation. Both classical and ANN-based approaches necessitate costly in situ data. In this study, we propose generating synthetic radar images with specified wind wave parameters using Fourier-based approach and Pierson–Moskowitz wave spectrum as a viable alternative. We generate synthetic images and use them for unsupervised learning approach to train a convolutional component of an ANN. After that, we train a regression ANN based on the previous convolutional component to obtain SWH back from synthetic images. Then, we apply preliminary trained weights for the same regression model to train SWH approximation on the dataset of real sea clutter images. In this study, we demonstrate the increase in the accuracy of SWH estimation from radar images with the preliminary training on synthetic data.

Abstract

The morphological diversity of New World monkeys (Platyrrhini) across Brazilian biomes represents a remarkable example of adaptive radiation in Neotropical primates. This review synthesizes current knowledge about morphological adaptations and distribution patterns of platyrrhine primates across Brazil's major biomes, focusing on the relationship between form and function in different ecological contexts. Through analysis of recent geometric morphometric studies, ecological surveys, and comparative anatomical research, we identify several key patterns of morphological variation, including: a continuous gradient of postcranial adaptations rather than discrete locomotor categories, unexpected variations in temporal bone architecture across forest strata, and rapid morphological responses to local dietary conditions as evidenced by dental morphology. Our synthesis reveals that morphological adaptation in Brazilian platyrrhines involves complex trade-offs between competing functional demands, with different anatomical systems evolving at varying rates. The relationship between morphological specialization and ecological resilience emerges as particularly significant, with implications for conservation strategies in the face of habitat modification and climate change. We demonstrate that traditional views of morphological adaptation as a gradual process require revision, as populations show capacity for relatively rapid morphological responses to environmental changes. These findings have important implications for both evolutionary theory and conservation practice, suggesting the need for approaches that preserve not just species, but the full spectrum of morphological diversity and the ecological contexts that generate and maintain it.

The document sets out three differentiated proposals for the transport of medicines through RPAS with their respective regulations, adapted to different operational categories: open category, special category and military scope. In the open category, an economical and efficient solution is prioritized for urban and rural areas, complying with weight and height regulations, in addition to guaranteeing the stability and quality of medications during transportation. The special category allows more complex operations and in more adverse conditions, such as densely populated areas and long routes, incorporating advanced security measures and training for personnel. In the military field, RPAS are used in conflict zones and hard-to-reach areas, ensuring the safe and rapid delivery of medical supplies under extreme conditions. Success stories in different countries are highlighted, demonstrating significant improvements in delivery times and risk reduction for health personnel.

We delve into the axion-dilaton model within the supergravity framework, with a specific focus on the intricacies of domain wall construction and stability. Employing holographic vitrification, we unravel the dynamics of domain wall formation in gauge theories featuring periodic vacuum structures. Our model, incorporating a QCD-like axion term and a stabilizing dilaton, undergoes scrutiny for conductivity variations under weak disorder. The investigation reveals the model's resilience, manifesting near-perfect conductivity under mild disorder conditions. However, the rigorous mathematical motivation for the holographic setup demands further elucidation. The scattered nature of our results prompts the necessity for a more systematic interpretation of QCD phenomena and conductivity transitions. This study contributes to the mathematical understanding of the axion-dilaton model's behavior, highlighting the imperative for a refined holographic framework and a more coherent interpretation of observed phenomena.

This research delves into the realm of predictive modeling within artificial intelligence (AI) and introduces an innovative ensemble approach designed to enhance prognostic precision. The proposed methodology, characterized by its ebullient nature, leverages the collective wisdom of diverse predictive models, synthesizing a comprehensive and robust framework. The pursuit of parsimony are paramount, underscoring the need for efficient model architectures that balance predictive acuracy and computational efficiency. Our study meticulously explores the amalgamation of diverse algorithms, encapsulating the intrinsic complexity of real-world phenomena. We navigate the intricate landscape of feature selection, algorithmic diversity, and model combination, fostering a holistic understanding of the prognostic process. The ebullient ensemble emerges as a dynamic solution, harnessing the synergy of constituent models while mitigating the risk of overfitting. To validate the efficacy of our approach, extensive experiments are conducted on diverse datasets, spanning domains such as healthcare, finance, and natural language processing. Comparative analyses demonstrate the superior predictive performance of our ensemble model in comparison to individual algorithms and traditional ensemble methods. Additionally, we delve into interpretability aspects, elucidating the underlying mechanisms that contribute to the model's prognostic prowess. Our research advances the frontier of predictive modeling in AI, offering a nuanced and sophisticated paradigm for prognostication. The ebullient ensemble approach stands as a testament to the synergy achievable through the amalgamation of diverse predictive models, providing a pragmatic and parsimonious avenue for enhanced prognostic acuracy in artificial intelligence applications.

Deep learning (DL) algorithms have recently emerged from machine learning and soft computing techniques. Since then, several deep learning (DL) algorithms have been recently introduced to scientific communities and are applied in various application domains. Today the usage of DL has become essential due to their intelligence, efficient learning, accuracy and robustness in model building. However, in the scientific literature, a comprehensive list of DL algorithms has not been introduced yet. This paper provides a list of the most popular DL algorithms, along with their applications domains.

In today’s expanding cities, pipeline networks are becoming an essential part of the industrial infrastructure. Monitoring these pipelines autonomously is becoming increasingly important. Inspecting pipelines for cracks is one specific task that poses a huge burden on humans. Undetected cracks may pose multi-dimensional risks. In this paper, we introduce the Pipeline Leak Identification Emergency Robot Swarm (PLIERS) system; an industrial system that deploys Internet of Things (IoT), robotics, and Neural Network technologies to detect cracks in emptied water and sewage pipelines. In PLIERS, a swarm of robots inspect emptied pipelines from the inside to detect cracks, collect images of them, and register their locations. When the images are taken, they are fed into a cloud-based module for analysis by a convolutional neural network (CNN). CNN is used to detect cracks and identify their severity. Through extensive training and testing, the CNN model performance showed promising scores for accuracy (between 80% and 90%), recall (at least 95%), precision (at least 95%), and F1 (at least 96%). Additionally, through the careful design of a prototype for a water/sewage pipeline structure with several types of cracks, the robots used managed to exchange information among themselves and convey crack images to the cloud-based server for further analysis. PLIERS is a system that deploys trendy technologies to detect and recognize cracks in pipeline grids. It adds to the efforts of improving instrumentation and measurement approaches by using robots, sensory, IoT principles, and the efficient analysis of CNN’s.

This study aims to address the inefficiencies and time-consuming nature of traditional hot-air anti-icing system designs by introducing reduced order models (ROM) and machine learning techniques to predict anti-icing surface temperature distributions. This study compares several classic neural networks, ultimately proposing two models: POD-AlexNet and multi-CNNs with GRU (MCG). Design variables of the hot-air anti-icing cavity are used as inputs, and the corresponding surface temperature distribution data serve as outputs. The performance of these models is evaluated on the test set. The POD-AlexNet model achieves a mean prediction accuracy of over 95%, while the MCG model reaches 96.97%. Both models significantly outperform traditional numerical simulation methods, delivering faster predictions. The proposed models achieve fast prediction of anti-icing surface temperature distribution while ensuring acceptable prediction accuracy. These models greatly enhance the prediction efficiency over existing numerical simulation approaches, contributing to the design of aircraft hot-air anti-icing systems based on optimization methods such as genetic algorithms.

Credit scoring is a cornerstone of financial risk management, enabling financial institutions to assess the likelihood of loan default. However, widely recognized contemporary credit risk metrics, like FICO or Vantage scores, remain proprietary and inaccessible to the public. This study aims to devise an alternative credit scoring metric that mirrors the FICO score, using an extensive dataset from Lending Club. The challenge lies in the limited insights available on both the precise analytical formula and the comprehensive suite of credit-specific attributes integral to the FICO score's calculation. Our proposed metric leverages basic information provided by potential borrowers, eliminating the need for extensive historical credit data. We aim to articulate this credit risk metric in a closed analytical form with variable complexity. To achieve this, we employ a symbolic regression method anchored in Genetic Programming (GP). Here, Occam's razor principle guides evolutionary bias towards simpler, more interpretable models. To ascertain our method's efficacy, we juxtapose the approximation capabilities of GP-based symbolic regression with established machine learning regression models, such as Gaussian Support Vector Machines (GSVMs), Multi-Layer Perceptrons (MLPs), Regression Trees and Radial Basis Function Networks (RBFNs). Our experiments indicate that GP-based symbolic regression offers comparable accuracy with these benchmark methodologies. Moreover, the resultant analytical model offers invaluable insights into credit risk evaluation mechanisms, enabling stakeholders to make informed credit risk assessments. This study contributes to the growing demand for transparent machine learning models by demonstrating the value of interpretable, data-driven credit scoring models.

Non-convex optimization is a critical tool in advancing machine learning, especially for complex models like deep neural networks and support vector machines. Despite challenges such as multiple local minima and saddle points, non-convex techniques offer various pathways to reduce computational costs. These include promoting sparsity through regularization, efficiently escaping saddle points, and employing subsampling and approximation strategies like stochastic gradient descent. Additionally, non-convex methods enable model pruning and compression, which reduce the size of models while maintaining performance. By focusing on good local minima instead of exact global minima, non-convex optimization ensures competitive accuracy with faster convergence and lower computational overhead. This paper examines the key methods and applications of non-convex optimization in machine learning, exploring how it can lower computation costs while enhancing model performance. Furthermore, it outlines future research directions and challenges, including scalability and generalization, that will shape the next phase of non-convex optimization in machine learning.

At the beginning of any electric arc welding process, a regime called transient regime will always occur, characterized by the continuous variation of the energy flow, also resulting in the variation of the cooling rate of the welded joint. Consequently, the weld bead hardness varies from a maximum value to an approximately constant value, and this phenomenon can be represented by an asymptotic exponential function. For several welded specimens, it would be possible to fit a non-linear regression for each specimen, however, it would be a difficult task to combine different regressions in a single analysis. The objective of this study is to evaluate the applicability of mixed-effects models and artificial neural networks to predict the longitudinal hardness of the weld bead in the initial transient regime. To this end, linearized and nonlinear mixed-effects models and artificial neural networks with different architectures were fitted. Finally, valid models were obtained with all approaches, with artificial neural network model slightly better fitted. However, the explainability capacity of the linearized mixed-effects model was superior, showing that the relative thickness, one of the predictor variables, was responsible for 48% of the variation in longitudinal hardness of weld bead.

Objective: Scaphoid fractures, particularly occult and non-displaced fractures, are difficult to detect using traditional X-ray methods due to their subtle appearance and variability in bone density. The study proposes a two-stage CNN approach to detect and classify scaphoid fractures using anterior-posterior (AP) and lateral (LA) X-ray views for more accurate diagnosis. Methods: The study emphasizes the use of multi-view X-ray images (AP and LA views) to improve fracture detection and classification. The multi-view fusion module helps integrate information from both views to enhance detection accuracy, particularly for occult fractures that may not be visible in a single view. The proposed method includes two stages that are stage 1: Detect the scaphoid bone using Faster R-CNN and Feature Pyramid Network (FPN) for region proposal and small object detection. The detection accuracy for scaphoid localization is 100% with Intersection over Union (IoU) scores of 0.8662 for AP views and 0.8478 for LA views. And stage 2: Perform fracture classification using a ResNet backbone and FPN combined with a multi-view fusion module to combine features from both AP and LA views. This stage achieves a classification accuracy of 89.94%, recall of 87.33%, and precision of 90.36%. Results: The proposed model performs well in both scaphoid bone detection and fracture classification. The multi-view fusion approach significantly improves recall and accuracy in detecting fractures compared to single-view approaches. In scaphoid detection, both AP and LA views achieved 100% detection accuracy. In fracture detection, using multi-view fusion, the accuracy for AP views reached 87.16%, and for LA views, it reached 83.83%. Conclusion: The multi-view fusion model effectively improves the detection of scaphoid fractures, particularly in cases of occult and non-displaced fractures. The model provides a reliable, automated approach to assist clinicians in detecting and diagnosing scaphoid fractures more efficiently.

Deep convolutional neural networks (CNNs) have revolutionised computer vision technology by automatically learning hierarchical representations of image data, leading to state-of-the-art performance in visual recognition tasks. This article presents a comprehensive review of deep CNNs, from their evolution and architectures to the current state-of-the-art research. The review also provides the core concepts and building blocks of CNNs and their concise mathematical representations. Furthermore, it explores applications of deep CNNs in three popular domains, including medical diagnosis, remote sensing, and facial recognition. This review will benefit researchers and practitioners in computer vision and artificial intelligence, as well as industry professionals seeking to leverage the latest advancements in deep learning technologies.

In Brain-Computer Interface Motor Imagery (BCI-MI) systems, Convolutional Neural Networks (CNNs) have traditionally dominated as the deep learning method of choice, demonstrating significant advancements in state-of-the-art studies. Recently, Transformer models with attention mechanisms have emerged as a sophisticated technique, enhancing the capture of long-term dependencies and intricate feature relationships in BCI-MI. This research investigates the performance of EEG-TCNet and EEG-Conformer models, which are trained and validated using various hyperparameters and bandpass filters during preprocessing to assess improvements in model accuracy. Additionally, this study introduces the EEG-TCNTransformer, a novel model that integrates the convolutional architecture of EEG-TCNet with a series of self-attention blocks employing a multi-head structure. The EEG-TCNTransformer achieves an accuracy of 82.97% without the application of bandpass filtering. The source code of EEG-TCNTransformer is available on GitHub.

Emergency departments (EDs) are critical in urgent care, where accurate triage optimizes patient flow and resource allocation. Manual triage faces challenges due to increasing volume and com-plexity. This study compares logistic regression, gradient boosting, neural network, and random forest models in predicting Emergency Severity Index (ESI) triage levels for non-traumatic pa-tients at Lampang Hospital's ED. Using data from January 1, 2023, to April 30, 2024, we ana-lyzed 45,245 complete records. The gradient boosting model achieved the highest accuracy (0.81), significantly outperforming logistic regression (accuracy 0.64). Pain scale, sex, and mean arterial pressure were key predictors. This is the first comparison of these models for ESI triage in a Thai hospital, highlighting their potential to improve triage accuracy and efficiency. Implementing these models could enhance patient outcomes and resource management in ED.

In this paper, the neural network is developed for solving the inverse photoacoustic problem aiming to estimate thermo-elastic properties and thickness of a semiconductor sample. The idea was that these sample properties be estimated from the phase characteristic because the phase measurements are more sensitive. The neural network has been trained on a large basis of photoacoustic phases calculated from a theoretical Si n-type model in the range of 20Hz to 20kHz, on which random Gaussian noise has been applied. It is shown that high accuracy and precision could be reached in solving of inverse photoacoustic problem using only phase measurement.

The success of near-infrared spectroscopy (NIRS) analysis hinges on the precision and robustness of the calibration model. Shallow learning (SL) algorithms like partial least squares discriminant analysis (PLS-DA) often fall short in capturing interrelationships between adjacent spectral variables, and the analysis results are easily affected by spectral noise, which dramatically limits the breadth and depth application of NIRS. Deep learning (DL) methods, with their capacity to discern intricate features from limited samples, are progressively integrated into NIRS. In this paper, two discriminant analysis problems including wheat kernels and Yali pears were used as examples, and several representative calibration models to research the robustness and effectiveness of the model. Additionally, this article proposed a near-infrared calibration model, which is based on gramian angular difference field and coordinate attention convolutional neural networks (G-CACNN). The research results show that, compared with SL, spectral preprocessing has a smaller impact on the analysis accuracy of consensus learning (CL) and DL, and the latter has the highest analysis accuracy in the modeling results of using the original spectrum. The accuracy of G-CACNN in two discrimination tasks is 98.48% and 99.39%, respectively. Finally, This research compared the performance of various models under noise to evaluate the robustness and noise resistance of the proposed method.

Property valuation services using artificial intelligence (AI) have been developed, with more than 20 services available in Japan. However, since their algorithms and training data are not publicly available, the extent of differences in the AI property valuations among these services is not clear. This study focuses on the five services handling a lot of properties and uses a sample of 4,295 property valuations for 859 condominium units in six popular residential areas in Tokyo. (1) Multiple comparison tests of the AI property valuations among each service are conducted to confirm their statistical significance and to examine the degree of the difference and variation. (2) The business models of each service are compared to examine the factors contributing to these differences and variations. The results showed that there was a variation of approximately 0.15 in the coefficient of variation between the high and low valuation groups, and the service providers in the high valuation group guided the users to their sales, while those in the low valuation group focused on attracting users as their business models. The contribution of this study is to indicate the possibility that the business models may lead to the differences in the AI property valuations.

This paper presents machine learning methods for approximate solutions of reaction-diffusion equations with multivalued interaction functions. This approach addresses the challenge of finding all possible solutions for such equations, which often lack uniqueness. The proposed method utilizes physics-informed neural networks (PINNs) to approximate generalized solutions.

A suitable waste management strategy is crucial for a sustainable and efficient circular economy in the construction sector, and requires precise data on the volume of demolition waste (DW) gen-erated. Therefore, we developed an optimal machine learning (ML) model to forecast the quantity of recycling and landfill waste based on the characteristics of DW. A dataset comprising infor-mation on the characteristics of 150 buildings, demolition equipment utilized, and volume of five waste types generated (i.e., recyclable mineral, recyclable combustible, landfill specified, and landfill mix waste, and recyclable minerals) was constructed. ML models were developed to predict the quantities of such waste. Artificial neural network, decision tree, gradient boosting machine, k-nearest neighbors, linear regression, random forest (RF), and support vector regression were applied, and optimal models were derived via hyperparameter tuning. The RF model demonstrated superior performance. In both validation and test phases, the “recyclable mineral waste” and “recyclable combustible waste” models achieved accuracies of of 0.987 and 0.972, re-spectively. The “recyclable metals” and “landfill specified waste” models achieved accuracies of 0.953 and 0.858 or higher, respectively. Moreover, the “landfill mix waste” model exhibited an accuracy of 0.984 or higher. SHapley Additive exPlanations analysis highlighted floor area as the primary input variable influencing model performance. The type of equipment employed in demolition emerged as another crucial input variable impacting the volume of recycling and landfill wastes generated. The developed model can provide precise data on waste management, thereby facilitating the decision-making process for industry professionals.

This research project aimed to address the growing concern about methane emissions from seaweed by developing a Convolutional Neural Network (CNN) model capable of accurately predicting these emissions. The study used PANDAS to read and analyze the dataset, incorporating statistical measures like mean, median, and standard deviation to understand the dataset. The CNN model was trained using the ReLU activation function and mean absolute error as the loss function. Model performance was evaluated through MAPE graphs, comparing the mean absolute percentage error (MAPE) between training and validation sets and between true and predicted emissions, and analyzing trends in yearly greenhouse gas emissions. The results demonstrated that the CNN model achieved a high level of accuracy in predicting methane emissions, with a low MAPE between the expected and actual values. This approach should enhance our understanding of methane emissions from Sargassum, contributing to more accurate environmental impact assessments and effective mitigation strategies.

Photovoltaic modules should pass series of tests and examinations, verifying the electrical and thermal characteristics of the module, before being released to the market, examining the properties of the modules should undergo appropriate durability and efficiency tests under real outdoor conditions with exposure to climatic conditions and under laboratory conditions. The performance of photovoltaic cells depends on many factors, such as solar irradiance, module operating temperature, installation location, weather conditions and module shading. In this paper, three selected photovoltaic modules were examined using a large-scale steady-state solar simulator. The current-voltage (I-V) characteristics of the photovoltaic modules were experimentally tested and analyzed. The next step was to implement a three-layer artificial neural network model (MLP). The experimental data obtained in the previous step coupled with output data from MLP were then used in global sensitivity analysis (GSA). Experiments carried out on a large-scale stationary solar simulator showed differences between the values declared by the manufacturer and the values obtained from measurements of PV modules. The first module tested achieved the maximum power point greater than that specified by the manufacturer, while the other two showed power drops; it was 85-87% for the second module, and 95-98% for the third, respectively. The performed global sensitivity analysis (GSA) for the MLP model showed that the parameters: eff (22.9) and Voc/V (14.19) have the largest effect on the power-voltage relationship, while U (7.29) has the smallest effect. The usefulness of machine learning (ML) methods in the comparative analysis of PV modules has been proved.

Convolutional Neural Networks (CNNs) are often criticized for their lack of transparency, acting as 'black boxes' in decision-making, a challenge compounded by class imbalance in ECG datasets, which limits their clinical application. This study introduces a CNN-based ECG signal classification model that enhances interpretability and addresses class imbalance through the Synthetic Minority Over-sampling Technique (SMOTE). The model also integrates Uniform Manifold Approximation and Projection (UMAP) for dimensionality reduction and SHAP value analysis, facilitating the visualization of decision boundaries and the assessment of feature contributions. Our evaluation using the MIT-BIH Arrhythmia Database highlights the model's high performance, with accuracy and precision nearing 1.00 for Normal ECG (NOR), Left Bundle Branch Block (LBBB), Right Bundle Branch Block (RBBB), and Ventricular Premature Beat (PVC) in the six-class task. In the ten-class task, the model demonstrated robustness, particularly with an accuracy of 0.9846, precision of 0.9783, recall of 0.9736, and F1 score of 0.9760 in Pacemaker Fusion Beat (PFHB), supported by an AUC of 0.9999 and AP of 0.9885. These results underscore the model's efficacy in cardiac rhythm recognition and resilience to class imbalance. Future research will explore sophisticated model architectures and feature extraction methods to enhance the model's generalization and clinical applicability for early heart disease diagnosis and personalized treatment.

The development of new turbomachinery designs requires numerous time-consuming and computationally intensive computational fluid dynamics (CFD) calculations. However, most of the generated high spatial resolution data remains unused at later development steps. That is also the case with automated optimization processes that use only a few integral values to determine objectives and constraints. Therefore the development of a data-driven AI model was initiated to ensure the potential of further use of the CFD data which is also used to train the AI model. The presented method subsequently provides a fast approximation of the 3D flow for new designs. In this paper, the structure of the developed AI model is presented and the approximation quality is analysed using a complex, state-of-the-art compressor test case. It is shown that the AI model can reproduce many characteristics of the 3D flow of new designs, and performance measures such as efficiency can be derived from these flow predictions. In addition, the complex test case revealed that greater design variation reduces the AI approximation quality which can lead to undesirable exploratory behaviour in an optimisation setup. Overall, the test case has shown promising results and has provided hints for further improvements of the AI model.

Microscopic vision plays an important role in automated micro-assembly. However, some uncertain factors in the assembly process, such as occlusion and stains can lead to the mistakes of feature extraction. Herein, to solve the problem, the deep learning techniques are introduced into the feature recognition tasks, focusing on the attention mechanism and visualizing CNNs for DL-based microscopic vision. The main contributions are summarized as follows: The CBAM attention mechanism is combined with the YOLOv5 algorithm to improve the accuracy and robustness of feature extraction. The micropart feature occlusion experiment results show that at 70% occlusion degree, YOLOV5-CBAM can reach 97.9% mAP@0.5, which is 4.6% higher than the original one. Visualization analysis of DL-based model is conducted using Grad-CAM to make the decision result more transparent and avoid potential visual detection risks during assembly. The heatmap matching degree between GT area and high-light area is increased by 27.81% on average, which further verify the effectiveness of attention mechanism in micropart feature localization. Additionally, micropart surface stain and droplet quality classification models based on ResNet50 are trained to replace the manual sorting. The visual results are consistent with human eye discernment and judgement, confirming the reliability of parts and droplets sorting.

Facial recognition systems are widely used systems around the globe. Their applications in real world scenarios like law enforcement, surveillance, ID card identification, information security, access control and many more has made them very important in today’s world. These systems have shown promising results and excellent performance; however, these systems need tremendous number of facial images for training before they are implemented in the real-world scenarios. The problem arises when we face lack of training images due to which the performance of such systems decreases exponentially. The problem becomes even worse when there is only one image per subject for training our facial recognition system, the probe image might contain variation like illumination, expression and disguise which are hard to be accurately recognized. Generative models have achieved great success for handling the lack of training data by generating synthetic data that resembles the original data. We have created a new method by fine tuning the basic Conditional Generative Adversarial Network (CGAN) to generate synthetic facial images provided only single image per subject. These synthetic facial images contain six variations that are smile, anger, stare, illumination and disguise. We increased the size of the dataset and then applied a convolutional neural network for facial recognition, which improved the accuracy of our system from 0.76 to 0.99.

In this paper major machine learning (ML) tools and the most important applications developed elsewhere for numerical weather and climate modeling systems (NWCMS) are reviewed. NWCMSs are briefly introduced. The most important papers published in this field in recent years are reviewed. The advantages and limitations of the ML approach in applications to NWCMS are briefly discussed. Currently, this field is experiencing explosive growth. Several important papers are published every week. Thus, this paper should be considered a simple introduction to the problem.

This study introduces a neural network-based approach to predict dust emission, specifically PM2.5 particles, during almond harvesting in California. Using a feedforward neural network (FNN), the research predicts PM2.5 emissions by analyzing key operational parameters of an advanced almond harvester. The model is trained on extensive field data from the almond pickup system, including variables like brush speed, angular velocity, and harvester forward speed. The results demonstrate a notable predictive accuracy of the FNN model, with a Mean Squared Error (MSE) of 0.02 and a Mean Absolute Error (MAE) of 0.01, indicating a high degree of precision in forecasting PM2.5 levels. The study also finds a strong correlation between certain operational parameters and PM2.5 emissions, highlighting specific areas for optimization in harvesting techniques. By integrating machine learning with agricultural practices, this research provides a significant tool for environmental management in almond production, offering a method to reduce harmful emissions while maintaining operational efficiency. This model presents a solution for the almond industry and sets a precedent for applying predictive analytics in sustainable agriculture.

Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges such as low imaging quality and high variability. There is a critical need to develop advanced automatic US image analysis methods to enhance diagnostic accuracy and objectivity. Vision transformer, a recent innovation in machine learning, has demonstrated significant potential in various research fields, including general image analysis and computer vision, due to its capacity to process large datasets and learn complex patterns. Its suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformer and discusses its applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in its application in medical US image analysis. Vision transformer has shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and is expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses.

This paper introduces a novel ensemble adjustment Kalman filter (EAKF) that integrates a machine-learning approach. The approach employs nonlinear variable relationships established by a deep neural network (DNN) during the analysis stage of the EAKF. This process nonlinearly projects observation increments into the state variable space. The newly developed DNN-EAKF algorithm can be applied to coupled data assimilation using coupled ocean-atmosphere models. It enhances cross-component updates in strongly coupled data assimilation (SCDA) by diminishing errors in estimating cross-component error covariance arising from insufficient ensemble members, thereby improving the SCDA analysis. This paper employs a conceptual model to conduct twin experiments, validating the DNN-EAKF’s capability to utilize cross-component observation information in SCDA effectively. The approach is anticipated to offer insights for future methodological integrations of machine learning and data assimilation and provide methods for SCDA applications in coupled general circulation models.

Memristive spiking neural networks are a promising emerging technology in the field of deep learning. These models can be implemented using neuromorphic hardware and therefore offer low power consumption and low latency, crucial to edge-computing applications. These systems, however, pose several challenges, including on-chip training and connectivity reduction. The latter is essential to ease the manufacturing complexity of the system. On-chip training can be realized using synaptic plasticity enabled by memristors. In this work, we compare two methods of connectivity reduction applicable to memristive spiking networks: an ensemble-based approach and a probabilistic sparse connectivity approach. We evaluate both of these methods in conjunction with a three-layer spiking neural network on the handwritten and spoken digits classification tasks using two memristive plasticity models and a classical time-dependent plasticity rule. On the handwritten digits recognition task, both methods achieve the F1-score of 0.89–0.93, and yield the 0.80–0.96 F1-score on the spoken digits recognition task.

Background: The development and application of machine learning (ML) methods became so fast that almost nobody can follow their developments in every detail. There is no wonder that numerous errors and inconsistencies in their usage have also spread with a similar speed independently from the tasks: regression and classification.. This work summarizes frequent errors committed by certain authors with the aims of helping scientists to avoid them. Methods: The principle of parsimony governs the train of thought. Fair method comparison methods can be completed with multicriteria decision making techniques, preferably sum of ranking differences (SRD). Its coupling with analysis of variance (ANOVA) decomposes the effect of several factors. Earlier findings are summarized in a review-like manner: the abuse of the correlation coefficient and proper practices for model discrimination are also outlined. Results: Using an illustrative example, the correct practice and the methodology is summarized as guidelines for model discrimination, and for minimizing the prediction errors. The following factors are all prerequisites for successful modeling: proper data preprocessing, statistical tests, suitable performance parameters, appropriate degrees of freedom, fair comparison of models, outlier detection, just to name a few. A checklist is provided on how to present ML modeling properly. The advocated practices are reviewed in the discussion. Conclusions: Many of the errors can easily be filtered out with careful reviewing. The authors’ responsibility is to adhere to the rules of modeling and validation.

To protect power lines, distance relays are used which can rely on a modified distance resistance. But usually the operating range of these relays changes as the network conditions change (network topology, load value, output value, etc.) and leads to false trips, By using methods that can process information and recognize patterns, for example, the use of micro and intelligent processor algorithms can use the new relays with high precision and thus provide adequate protection. In this study, a distance relay was modeled using a neural network, and it was observed that the neural relay had higher accuracy than the conventional relay. In addition to detecting the fault and its location, type and phase of the fault, three-stage simultaneous protection can be performed. As a result, the number of linear relays can be reduced by using relays based on neural technologies. An MLP (multilayer perceptron) neural network is used to model a sequence of distances.

Photoelectric smoke detectors are the most cost-effective devices for very early fire alarms. However, due to different light intensity response values for different fire smoke and interference from interferential aerosols, they have a high false alarm rate, which limits their popularity in Chinese homes. To address these issues, an embedded spatial-temporal convolutional neural network (EST-CNN) model is proposed for real fire smokes identification and aerosols (fire smokes and interferential aerosol) classification. EST-CNN consists of three modules including information fusion, scattering feature extraction, and aerosol classification. Moreover, a two dimensional spatial-temporal scattering (2D-TS) matrix is designed to fuse the scattered light intensities in different channels and adjacent time slices, which is the output of the information fusion module and the input of the scattering feature extraction module. EST-CNN is trained and tested with experimental data measured on the established fire test platform using the developed dual-wavelength dual-angle photoelectric smoke detector. The optimal network parameters are selected through extensive experiments resulting in an average classification accuracy of 95.6% for different aerosols with only 66 kB network parameters. The experimental results demonstrate the feasibility of the designed EST-CNN model to be directly installed in existing commercial photoelectric smoke detectors to realize aerosol classification.

The accurate classification of landfill waste diversion plays a critical role in efficient waste management practices. Traditional approaches, such as visual inspection, weighing and volume measurement, and manual sorting, have been widely used but suffer from subjectivity, scalability, and labour requirements. In contrast, machine learning approaches, particularly Convolutional Neural Networks (CNN), have emerged as powerful deep learning models for waste detection and classification. This paper analyses VGG-16, InceptionResNetV2, DenseNet121, Inception V3, and MobileNetV2 models to classify real-life waste when trained on pristine and unadulterated materials, versus samples collected at a landfill site. When training on DiversionNet, the unadulterated material dataset with labels required for landfill modelling, classification accuracy was limited to 49.69% in the real environment. Using real-world samples in the newly formed RealWaste dataset showed that practical applications for deep learning in waste classification is possible, with Inception V3 reaching 89.19% classification accuracy on the full spectrum of labels required for accurate modelling.

Facial expression recognition is crucial for understanding human emotions and nonverbal communication. With the growing prevalence of facial recognition technology and its various applications, accurate and efficient facial expression recognition has become a significant research area. However, most previous methods have focused on designing unique deep-learning architectures while overlooking the loss function. This study presents a new loss function that allows simultaneous consideration of inter- and intra-class variations to be applied to CNN architecture for facial expression recognition. More concretely, this loss function reduces the intra-class variations by minimizing the distances between the deep features and their corresponding class centers. It also increases the inter-class variations by maximizing the distances between deep features and their non-corresponding class centers, and the distances between different class centers. Numerical results from several benchmark facial expression databases, such as Cohn-Kanade Plus, Oulu-Casia, MMI, and FER2013, are provided to prove the capability of the proposed loss function compared with existing ones.

Eutrophication is a major environmental issue with many negative consequences, such as hypoxia and harmful cyanotoxins production. Monitoring coastal eutrophication is a crucial, especially for island countries like the Republic of Cyprus, which are economically dependent on the touristic sector. Additionally, the open-sea aquaculture industry in Cyprus has been exhibiting an increase in the last decades and environmental monitoring to identify possible signs of eutrophication is mandatory according to the legislation. Therefore, in this modelling study, two different types of Artificial Neural Networks (ANNs) are developed based on in situ-data collected from stations located in the coastal waters of Cyprus. Theses ANNs aim to model the eutrophication phenomenon based on two different data-driven modelling procedures. Firstly, the self-organizing map (SOM) ANN examines several water quality parameters (specifically water temperature, salinity, nitrogen species, ortho-phosphates, dissolved oxygen and electrical conductivity) interactions with the Chlorophyll-a parameter. The SOM model enables us to visualize the monitored parameters relationships and to comprehend complex biological mechanisms related to Chlorophyll-a production. A second feed-forward ANN model is also developed for predicting the Chlorophyll-a levels. Based on this ANN model, several scenarios associated to the eutrophication-related water quality parameters can be extracted. The combination of these two ANNs models is considered a holistic modelling approximation for the identification of eutrophication scenarios, since it enables not only the prediction of the Chlorophyll-a parameter levels, but also the “capturing” of hidden biological mechanisms associated with algal production.

Waste heat recovery stands out as a promising technique to tackle both energy shortages and environmental pollution. Currently, this valuable resource, generated from processes like fuel combustion or chemical reactions, is often dissipated into the environment, despite its potential to significantly contribute to the economy. To harness this untapped potential, a travelling-wave thermo-acoustic generator has been designed and subjected to comprehensive experimental analysis. Fifty-two data corresponding to different working conditions of the system were extracted to build ANN, ANFIS and ANN-PSO models. Evaluation of performance metrics reveals that the ANN-PSO model demonstrates the highest predictive accuracy (R^2=0.9959), particularly in relation to output voltage. This research demonstrates the potential of machine learning techniques for the analysis of thermo-acoustic systems. By doing so, it is possible to get an insight into nonlinearities inherent to thermo-acoustic systems. This advancement empowers researchers to forecast the performance characteristics of alternative configurations with a heightened level of precision.

The semantic segmentation of high-resolution remote sensing images (HR-RSI) is crucial for a wide range of applications, such as precision agriculture, urban planning, natural resource assessment, and ecological monitoring. However, accurately classifying pixels in HR-RSI faces challenges due to densely distributed small objects and scale variations. Existing techniques, including Convolutional Neural Networks (CNNs) and other methods for hierarchical feature extraction and fusion of remote sensing image, often do not achieve the desired accuracy. In this paper, we propose a novel approach called the Hierarchical Rich-scale Fusion Network (HRFNet) to address these challenges. HRFNet utilizes advanced information rating and image partition techniques to extract rich-scale features within image layers. This allows for the adaptive exploration of both local and global contextual information. Moreover, we introduce a structured intra-layer to inter-layer feature aggregation module, which enables the adaptive extraction of fine-grained details and high-level semantic information from multi-layer feature maps in a highly flexible manner. Extensive experimentation has been conducted to validate the effectiveness of our proposed method. Our results demonstrate that HRFNet outperforms existing techniques, achieving state-of-the-art (SOTA) results on benchmark datasets, specifically the ISPRS Potsdam and Vaihingen datasets.

Background: Proteins targeted by the Ubiquitin Proteasome System (UPS) are identified for degradation by the proteasome, which has been implicated in the development of neuro-degenerative diseases. Major histocompatibility complex (MHC) molecules present peptides broken down by the proteasome and are involved in neuronal plasticity, regulating synapse number and axon regeneration in the central or peripheral nervous system during develop-ment and in brain diseases. The mechanisms governing these effects are mostly unknown, but evidence from different compartments of the cerebral cortex indicates the presence of im-mune-like MHC receptors in the central nervous system. Methods: We have used human induced pluripotent stem cells (iPSC) differentiated to neural stem cells and then to motor neurons as a developmental model to better understand the structure of the proteasome in developing motor neurons. We perform a proteomic analysis of starting human skin fibroblasts, their matching iPSC, differentiated neural stem cells and motor neurons that highlighted significant differences in the constitutive proteasome and immunoproteasome subunits during development towards motor neurons from iPSC. Results: Proteomic analysis showed that the catalytic proteasome subunits expressed in fi-broblasts differ to those in neural stem cells and motor neurons. Western blot analysis con-firmed the proteomic data, particularly the decreased expression of Beta5i (PSMB8) subunit immunoproteasome. Conclusion: The constitutive proteasome subunits are upregulated in iPSC from HFF and the immunoproteasome subunit beta 5i expression is higher in MN than NSC suggesting a im-munoproteasome phenotype in MN. The immunoproteasome may have implications on motor neuron development and neurodevelopmental diseases that warrants further investi-gation.

Plasma technology shows tremendous potential for revolutionizing oncology research and treatment. Reactive oxygen and nitrogen species, electromagnetic emissions generated through gas plasma jets, have attracted significant attention due to their selective cytotoxicity towards cancer cells. To leverage the full potential of plasma medicine, researchers have explored the use of mathematical models and various subsets of machine learning, such as reinforcement learning, and deep learning. This review emphasizes the significant application of AI algorithms in the adaptive plasma system, paving the way for precision and dynamic cancer treatment. Realizing the full potential of AI in plasma medicine, requires research efforts, data sharing and interdisciplinary collaborations. Unravelling the complex mechanisms, developing real-time diagnostics, and optimizing AI models will be crucial to harness the true power of plasma technology in oncology. The integration of personalized and dynamic plasma therapies, alongside AI and diagnostic sensors, presents a transformative approach to cancer treatment with the potential to improve outcomes globally.

In order to solve the problems of complicated steering control of unmanned vehicles in the field and difficult steering on complex roads, we designed a wheel-track composite vehicle equipped with a novel power differential steering mechanism with dual driving, which drove the steering of the vehicle through the differential rotation of the rear two wheels. The unmanned vehicle was simple to control, small in size, and able to work under the conditions of complex roads, such as hills, mountains, and muddy land. Firstly, a steering mechanism with both differential speed and force was designed to prevent the vehicle from skidding into muddy land and stopping motionless. Secondly, the kinematics model and dynamics model of the two drive shafts and the two output shafts (wheel shafts) were established. Thirdly, according to the relationship between the rotational speed of the two output shafts and the steering radius of the vehicle, the kinematic model of the rotational speed of the two input shafts and the steering radius of the wheel-track composite vehicle was obtained. Finally, according to the test data, the mathematical model of rotational speeds of the two input shafts and the actual steering radius of the vehicle was obtained by neural network fitting, and the maximum relative error between the model results and the actual steering radius value was 3.53%. The combination of power differential steering mechanism and wheel-track composite unmanned vehicle increased the adhesion with the ground and could better adapt to the complex road environment. In conclusion, the unmanned vehicle had the advantages of continuous radius steering, deceleration and torsion increase, differential lock, etc. It was suitable for all-terrain military and civilian vehicles and various special equipment mobile platforms of the walking device, and the research results could provide theoretical references for the steering control and structural optimization of the wheel-track composite vehicle under the environment of complex road surface.

Bone growth plate abnormalities and skull shape defects are seen in hypophosphatasia, a heritable disorder in humans that occurs due to deficiency of Tissue Nonspecific Alkaline Phosphatase (TNAP, Alpl) enzyme activity. Abnormal development of the cranial base growth plates (synchondroses) and abnormal skull shapes have also been demonstrated in global TNAP-/- mice. To distinguish local vs. systemic effects of TNAP on skull development, we utilized P0-Cre to knockout TNAP only in cranial neural crest derived tissues using TNAP flox mice. Here we show that TNAP deficiency in cranial neural crest leads to leads to skull shape defects and deficient growth of the intersphenoid synchondrosis (ISS). ISS chondrocyte abnormalities included increased proliferation in resting and proliferative zones with decreased apoptosis in hypertrophic zones. ColX expression is increased, indicative of premature differentiation in the absence of TNAP. Sox9 expression is increased in both resting and prehypertrophic zones of mutant mice. Expression of PTHrP and IHH were also increased. Finally, cranial base organ culture revealed that inorganic phosphate (Pi) and pyrophosphate (PPi) have specific effects on cell signaling and phenotype changes in the ISS. Together, these results demonstrate that TNAP expression in growth plate chondrocytes is essential for normal development, and that the mechanism likely involves Sox9, PTHrP, IHH and PPi.

Arrhythmia is a cardiac condition characterized by an irregular heart rhythm that hinders the proper circulation of blood, posing a severe risk to individuals’ lives. Globally, arrhythmias are recognized as a significant health concern, accounting for nearly 12 percent of all deaths. As a result, there has been a growing focus on utilizing artificial intelligence for the detection and classification of abnormal heartbeats. In recent years, self-operated heartbeat detection research has gained popularity due to its cost-effectiveness and potential for expediting therapy for individuals at risk of arrhythmias. However, building an efficient automatic heartbeat monitoring approach for arrhythmia identification and classification comes with several significant challenges. These challenges include addressing issues related to data quality, determining the range for heart rate segmentation, managing data imbalance difficulties, handling intra- and inter-patient variations, distinguishing supraventricular irregular heartbeats from regular heartbeats, and ensuring model interpretability. In this study, we propose the Reseek-Arrhythmia model, which leverages deep learning techniques to automatically detect and classify heart arrhythmia diseases. The model combines different convolutional blocks and identity blocks, along with essential components such as convolution layers, batch normalization layers, and activation layers. To train and evaluate the model, we utilized the MIT-BIH and PTB datasets. Remarkably, the proposed model achieves outstanding performance with an accuracy of 99.35% and 93.50% and an acceptable loss of 0.688 and 0.2564, respectively.

Light Detection and Ranging (LiDAR) technology, a cutting-edge advancement in mobile applications, presents a myriad of compelling use cases, including enhancing low-light photography, capturing and sharing 3D images of fascinating objects, and elevating the overall augmented reality (AR) experience. However, its widespread adoption has been hindered by the prohibitive costs and substantial power consumption associated with its implementation. To surmount these obstacles, this paper proposes a low-power, low-cost, SPAD-based system-on-chip (SoC) which packages the microlens arrays (MLA) and incorporates with a light-weight RGB-guided sparse depth imaging completion neural network for 3D LiDAR imaging. The proposed SoC integrates an 8x8 Single-Photon Avalanche Detectors (SPADs) macro pixel array with time-to-digital converters (TDC) and charge pump, fabricated using a 180nm bipolar-CMOS-DMOS (BCD) process. A random MLA-based homogenizing diffuser efficiently transforms Gaussian beams into flat-topped beams with a 45° field of view (FOV), enabling flash projection at the transmitter. To further enhance resolution and broaden application possibilities, a lightweight neural network employing RGB-guided sparse depth complementation is proposed, enabling a substantial expansion of image resolution from 8x8 to quarter video graphics array level (QVGA; 256x256). Experimental results demonstrate the effectiveness and stability of the hardware encompassing the SoC and optical system, as well as the lightweight features and accuracy of the algorithmic neural network. This integrated state-of-the-art hardware-software solution offers a promising and inspiring foundation for developing consumer-level 3D imaging applications.

Epilepsy is a chronic neurological disorder characterized by abnormal electrical activity in the brain, often leading to recurrent seizures. With approximately 50 million people worldwide affected by epilepsy, there is a pressing need for efficient and accurate methods to detect and diagnose seizures. Electroencephalogram (EEG) signals, which record the brain’s electrical activity, have emerged as a valuable tool in detecting epilepsy and other neurological disorders. Traditionally, the process of analyzing EEG signals for seizure detection has relied on manual inspection by experts, which is time-consuming, labor-intensive, and susceptible to human error. To address these limitations, researchers have turned to machine learning and deep learning techniques to automate the seizure detection process. In this work, we propose a novel method for epileptic seizure detection, leveraging the power of 1-D Convolutional layers in combination with Bidirectional Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) and Average pooling Layer as a unit. The performance of the proposed model is verified on the Bonn dataset. Our proposed model offers significant advancements over existing approaches for the detection of epileptic seizures, particularly for 2, 3, 4, and 5-class classification tasks. To assess the robustness and generalizability of our proposed architecture, we employ 5-fold cross-validation. By dividing the dataset into five subsets and iteratively training and testing the model on different combinations of these subsets, we obtain robust performance measures, including accuracy, sensitivity, and specificity. We demonstrate the effectiveness of the proposed architecture in accurately detecting epileptic seizures from EEG signals by using EEG signals of varying lengths. The results indicate its potential as a reliable and efficient tool for automated seizure detection, paving the way for improved diagnosis and management of epilepsy.

Efficient management of basal cell carcinoma (BCC) requires reliable assessments of both tumors and post-treatment scars. We aimed to estimate image similarity metrics that account for BCC's perceptual color and texture deviation from perilesional skin. 176 clinical photographs of BCC were assessed by six physicians using a visual deviation scale. Internal consistency and inter-rater agreement were estimated using Cronbach's α, weighted Gwet's AC2, and quadratic Cohen's Kappa. The mean visual scores were used to validate a range of similarity metrics employing different color spaces, distances, and image embeddings from pre-trained VGG16 neural network. The calculated similarities were transformed into discrete values using ordinal logistic regression models. The Bray-Curtis distance in the YIQ color model and rectified embeddings from the 'fc6' layer minimized the mean squared error and demonstrated strong performance in representing perceptual similarities. Box plot analysis and Wilcoxon rank-sum test were used to visualize and compare the levels of agreement, conducted on a random validation round between the two groups: ‘Human-System’ and ‘Human-Human’. The proposed metrics were comparable in terms of internal consistency and agreement with human raters. The findings suggest that the proposed metrics offer a robust, and cost-effective approach to monitoring BCC treatment outcomes in clinical settings.

Lithium-ion batteries serve as the primary sources of power for electric vehicles (EVs) and hybrid electric vehicles (HEVs). For vehicle applications, battery management systems (BMSs) are nec-essary to protect lithium-ion batteries from overheating and to ensure optimum vehicle perfor-mance. Our approach to developing a BMS was based on recent advances in the application of phase field models for lithium-ion batteries. In particular, our reduced-order model (ROM) uti-lized a dataset generated from the COMSOL® Multiphysics simulation of the Cahn–Hilliard equation for a single particle of a lithium iron phosphate (LiFePO4) cathode: an example of using a reduced-order model (ROM) based on a single-particle model (SPM). The main innovation of our ROM is that the SPM is fully coupled to a heat transfer model at the battery cell level. We utilized principal component analysis to identify a lower-order model that could reproduce the battery’s voltage and temperature response for ambient temperatures ranging from 253 to 298 K and for discharge rates ranging from 1 C to 20.5 C. The reduced-order dataset was then fitted to the ex-perimental data for an A123 Systems 26650 2.3 Ah cylindrical battery using deep neural network (DNN) regression.

With the exponential growth of machine learning and development of Artificial Neural Network (ANNs) in recent years, there is great opportunity to leverage this approach and accelarate bio-logical discoveries through applications on the analysis of bioinformatics data. Various types of datasets including for example protein or gene interaction networks, molecular structures and cellular signalling pathways, have already been used for machine learning by training ANNs for inference and pattern classification. However, unlike regular data structures that are commonly used in the computer science and engineering fields, bioinformatics datasets present challenges that require unique algorithmic approaches. The recent development of the geometric and deep learning approach within the machine learning field, is very promising towards accelerating analysis complex bioinformatics datasets. The principles of ANNs and their importance for bio-informatics machine learning is demonstrated herein, through presentation of the undelying mathematical and statistical foundations from group theory, symmetry, linear algebra. Further-more, the structure and functions of ANN algorithms that form the core principles of artificial intelligence are explained, in relation to the bioinformatics data domain. Overall, the manuscript provides guidance for researchers to understand the principles required for practicing machine learning and artificial intelligence, with the special considerations towards bioinformatics applications.

Physics-informed machine learning (PIML) refers to the emerging area of extracting physically relevant solutions to complex multiscale modeling problems and has enjoyed significant interest from the research community. This paper discusses the recent critical advancements in the PIML domain. Novel methods and applications of domain decomposition in physics-informed neural networks (PINN) in particular are highlighted. Additionally, we explore recent Works toward utilizing neural operator learning to intuit relationships in physics systems traditionally modeled by sets of complex governing equations and solved with expensive differentiation techniques. Finally, expansive applications of traditional physics-informed machine learning and potential limitations are discussed. In addition to summarizing recent work, we propose a novel taxonomic structure to catalog physics-informed machine learning based on how the physics-information is derived and injected into the machine learning process. The taxonomy assumes the explicit objectives of facilitating interdisciplinary collaboration in methodology, thereby promoting a wider characterization of what types of physics-problems are served by the physics-informed learning machines, and assisting in identifying apt targets for future work. To summarize, the major twofold goal of this work is to summarize recent advancements and introduce a taxonomic catalog for applications of physics-informed machine learning.

Accurate evaluation of health status of lithium-ion batteries must be deemed as of great significance, insofar as utility and safety of batteries are of concern. Lithium plating, in particular, is notoriously known to be a chemical reaction that can cause deterioration in, or even fatal hazards to, the health of lithium-ion batteries. Electrochemical impedance spectroscopy (EIS), which has distinct advantages such as fast or non-destructive over its competitors, suffices in detecting lithium plating and thus has been attracting increasing attention in the field of battery management, but its ability of assessing quantitatively the degree of lithium plating remains largely unexplored hitherto. On this point, this work seeks to narrow that gap, by proposing an EIS-based method that can quantify the degree of lithium plating. The core conception is to eventually circumvent the reliance on state-of-health measurement, and use instead the impedance spectrum, to acquire an estimate on battery capacity loss. To do so, the effects of solid electrolyte interphase formation and lithium plating on battery capacity must be first decoupled, so that the mass of lithium plating can be quantified. Then, based on an impedance spectrum measurement, the parameters of the fractional equivalent circuit model (ECM) of the battery can be identified. These fractional ECM parameters are received as inputs by an artificial neural network, which is tasked to establish a correspondence between the model parameters and the mass of lithium plating. The empirical part of the work revolves around the data collected from an aging experiment, and the validity of the proposed method is truthfully attested by dismantling the batteries, which is otherwise not needed during the actual uptake of the method.

A block-diagonal fuzzy neural network for short-term load forecasting is proposed. DBD-FELF consists of fuzzy rules with consequent parts that are neural networks with internal recurrence. These networks have a hidden layer which consists of pairs of neurons with feedback connections between them. The overall fuzzy model partitions the input space in partially overlapping fuzzy regions, where the recurrent neural networks of the respective rules operate. The partition of the input space and determination of the fuzzy rule base is performed by use of Fuzzy C-Means clustering algorithm and the RENNCOM constrained optimization method is applied for consequent parameter tuning. The electric load time-series of the Greek power system is examined, and hourly-based forecasting for the whole year is performed. The performance of DBD-FELF is tested via extensive experimental analysis and the results are promising, since an average percentage error of 1.18% is attained, along with an average yearly absolute error of 76.2 MW. Moreover, DBD-FELF is compared with Deep Learning, fuzzy and neurofuzzy rivals, such that its particular attributes are highlighted.

The current neural networks for tomato leaf disease recognition have problems such as large model parameters, long training time, and low model accuracy. To solve these problems, a lightweight convolutional neural network LBFNet is proposed in this paper.First, a lightweight convolutional neural network LBFNet is established as the base model. Secondly, a three-channel attention mechanism module is introduced to learn the disease features in tomato leaf disease images and reduce the interference of redundant features. Finally, a cascade module is introduced to increase the depth of the model, solve the gradient descent problem, and reduce the loss caused by increasing the depth of the model. The quantized pruning technique is also used to further compress the model parameters and optimize the model performance. The results show that the LBFNet model achieves 99.06% accuracy on the LBFtomato dataset, with a training time of 996s and a single classification accuracy of over 94%. Further training using the saved weight file after quantized pruning makes the model accuracy reach 97.66%. Compared with the base model, the model accuracy was improved by 28%, and the model parameters were reduced by 96.7% compared with the traditional Resnet50. It was found that LBFNet can quickly and accurately identify tomato leaf diseases in complex environments, providing effective assistance to agricultural producers.

The difficulty in detecting tumors in earlier stages is the major cause of mortalities of patients, despite the advancements in treatment and research regarding ovarian cancer. Deep Learning algorithms are applied to serve the purpose of a diagnostic tool by applying them on CT scan images of the ovarian region. The images go through a series of pre-processing techniques and further the tumor is segmented using the UNet model. Instances are then classified into two categories – benign and malignant tumors. Classification is performed using Deep Learning models like CNN, ResNet, DenseNet, Inception-ResNet, VGG16 and Xception along with Machine Learning models such as Random Forest, Gradient Boosting, AdaBoosting, XGBoosting. DenseNet 121 emerges as the best model on this dataset even after applying optimization on the Machine Learning models by obtaining an accuracy of 95.7%. The current work demonstrates the comparison of multiple CNN architectures among themselves and with common Machine Learning algorithms, with and without optimization techniques applied.

The deep energy method (DEM), a type of physics-informed neural network, is evolving as an alternative to finite element analysis. This method employs the principle of minimum potential energy to predict deformations under static loading conditions. However, the model’s accuracy is contingent upon choosing the appropriate architecture for the model, which can be challenging due to the high interactions between hyperparameters, large search space, difficulty in identifying objective functions, and non-convex relationships with the objective functions. To improve DEM’s accuracy, we first introduce random Fourier feature (RFF) mapping. RFF mapping helps with the training of the model by reducing bias towards high frequencies. The effects of six hyperparameters are then studied under compression, tension, and bending loads in planar linear elasticity. Based on this study, a systematic automated hyperparameter optimization approach is proposed. Due to the high interaction between hyperparameters and the non-convex nature of the optimization problem, Bayesian optimization algorithms are used. The models trained using optimized hyperparameters and having Fourier feature mapping can accurately predict deflections compared to finite element analysis. Additionally, the deflections obtained for tension and compression load cases are more sensitive to variations in hyperparameters than bending.

Nowadays, there is extensive production and use of plastic materials for different industrial activities. These plastics, either from their primary production sources or through degradation processes of the plastics themselves, can contaminate the ecosystem with micro and nanoplastics. Once in the aquatic environment, these microplastics can be the basis for the adsorption of chemical pollutants, favoring that these chemical pollutants disperse more quickly in the environment and can affect living beings. Due to the lack of information on adsorption, three machine learning models (random forest, support vector machine, and artificial neural network) to predict different microplastic/water partition coefficients (log Kd) were developed using two different approximations (based on the number of input variables). The best-selected machine learning models present, in general, correlation coefficients upper than 0.92 in the query phase, which indicate that these type of models could be used for a rapid estimation of the absorption of organic contaminants on microplastics.