The neural crest, a unique cell population originating from the primitive neural field, has a multi-systemic and structural contribution to vertebrate development. At the cephalic level, the neural crest generates most of the skeletal tissues encasing the developing forebrain and provides the prosencephalon with functional vasculature and meninges. Over the last decade, we have demonstrated that the cephalic neural crest (CNC) exerts an autonomous and prominent control on forebrain and sense organs development. The present paper reviews the primary mechanisms by which CNC can orchestrate vertebrate encephalization. Demonstrating the role of the CNC as an exogenous source of patterning for the forebrain provides a novel conceptual framework with profound implications for understanding neurodevelopment. From a biomedical standpoint, these data suggest that the spectrum of neurocristopathies is broader than expected and that some neurological disorders may stem from CNC dysfunctions.

A recent study from our lab revealed that inhibition of cyclooxygenase-2 exclusively reduces the level of PGE2 amongst the prostanoids and hamper the normal development of several structures, strikingly the cranial vault, in chick embryos. In order to unearth the mechanism behind the deviant development of cranial features, the expression pattern of various factors that are known to influence the cranial neural crest cell (CNCC) migration were checked in chick embryo after inhibiting the COX-2 activity using etoricoxib. The compromised level of cell adhesion molecules and their upstream regulators, namely CDH1, CDH2, MSX1, and TGF-β, observed in the etoricoxib treated embryos indicate that COX-2, through its downstream effector PGE2, regulates the expression of these factors perhaps to aid the migration of CNCC. The histological features and levels of FoxD3 as well as PCNA further consolidates the role of COX-2 in migration and survival of CNCC in developing embryo. The results of the current study indicate that the COX-2 plays a pivotal role in orchestrating the proliferation and migration of CNCC during embryonic development of chick.

Periodontitis is microbial infection affecting periodontium, the tooth supporting structure and affects >743 million people worldwide. Neural crest-derived stem cells (NCSCs) hold the promise to regenerate the damaged periodontium. These cells have been identified within adult adipose tissue, periodontal ligament, and palatal tissue. Typical enzymatic isolation protocols are expensive, time consuming and often not clinically compliant. Enzyme-free, mechanical dissociation has been suggested as an alternative method of generating cell suspensions required for cell separation and subsequent expansion ex vivo. In our study, samples of rat skeletal muscle tissue were used to appraise the suitability of a novel mincing method of mechanical dissociation against enzymatic digestion with collagenase and dispase. Skeletal muscle is readily available and has been shown to contain NCSC populations. We used a Rigenera-Human Brain Wave^® prototype mincer to produce a suspension of skeletal muscle-derived cells modeling NCSCs. We have compared the resulting cell cultures produced via mechanical dissociation and enzymatic dissociation, producing single cell suspensions suitable for Magnetic Cell Sorting (MACs) and Fluorescence-activated cell sorting (FACS). Despite the Countess Automated Cytometry data demonstrating that cell suspensions produced by mechanical dissociation (n=24) contain on average 26.8 times as many viable cells as enzymatic cell suspensions (n=18), enzymatic suspensions produced more successful cell cultures. Spheroids and subsequently adherent cells formed from 4 enzymatic cell suspensions (44.4%) vs. 1 mechanical cell suspension (8.3%). Enzymatic digestion protocols formed spheroids faster and more plentifully than mechanical cell suspensions. Adherent cells and spheroids isolated via both methods appear morphologically similarly to NCSCs from our previous studies.

Bone growth plate abnormalities and skull shape defects are seen in hypophosphatasia, a heritable disorder in humans that occurs due to deficiency of Tissue Nonspecific Alkaline Phosphatase (TNAP, Alpl) enzyme activity. Abnormal development of the cranial base growth plates (synchondroses) and abnormal skull shapes have also been demonstrated in global TNAP-/- mice. To distinguish local vs. systemic effects of TNAP on skull development, we utilized P0-Cre to knockout TNAP only in cranial neural crest derived tissues using TNAP flox mice. Here we show that TNAP deficiency in cranial neural crest leads to leads to skull shape defects and deficient growth of the intersphenoid synchondrosis (ISS). ISS chondrocyte abnormalities included increased proliferation in resting and proliferative zones with decreased apoptosis in hypertrophic zones. ColX expression is increased, indicative of premature differentiation in the absence of TNAP. Sox9 expression is increased in both resting and prehypertrophic zones of mutant mice. Expression of PTHrP and IHH were also increased. Finally, cranial base organ culture revealed that inorganic phosphate (Pi) and pyrophosphate (PPi) have specific effects on cell signaling and phenotype changes in the ISS. Together, these results demonstrate that TNAP expression in growth plate chondrocytes is essential for normal development, and that the mechanism likely involves Sox9, PTHrP, IHH and PPi.

The evolution of jaws represents one of the most transformative innovations in vertebrate history, fundamentally reshaping both animal anatomy and ecological dynamics. At the core of this innovation lies the neural crest, a unique vertebrate cell population whose molecular and developmental properties enabled the formation of articulated jaws. This review explores how neural crest cells, as multipotent stem cells, contributed to jaw development through their extraordinary capacity to form cartilage, bone, and associated tissues. Beyond the developmental perspective, we examine how the emergence of jawed vertebrates dramatically altered marine ecosystems, leading to new predator-prey relationships and driving an unprecedented diversification of feeding strategies. This innovation triggered an evolutionary arms race that shaped marine biodiversity and established new trophic relationships that persist in modern oceans. Understanding this ancient innovation provides crucial insights into both developmental mechanisms and ecological dynamics that defined vertebrate evolution.

One of the most effective image processing techniques is the use of convolutional neural networks, where we combine intensity values at grid points in the vicinity of each point. To speed up computations, researchers have developed a dilated version of this technique, in which only some points are processed. It turns out that the most efficient case is when we select points from a sub-grid. In this paper, we explain this empirical efficiency proving that the sub-grid is indeed optimal – in some reasonable sense. To be more precise, we prove that all reasonable optimality criteria, the optimal subset of the original grid is either a sub-grid, or a sub-grid-like set.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a new approach to ONNs based on integrated Kerr micro-combs that is programmable, highly scalable and capable of reaching ultra-high speeds, demonstrating the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve a single-unit throughput of 11.9 Giga-OPS at 8 bits per OP, or 95.2 Gbps. We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

Machine unlearning, the process of removing the influence of specific data points from trained machine learning models, has become increasingly important in light of modern data privacy regulations, such as the GDPR and the "right to be forgotten." This paper explores the challenges and solutions associated with implementing machine unlearning in three widely used neural network architectures: Convolutional Neural Networks (CNNs), Deep Neural Networks (DNNs), and Graph Neural Networks (GNNs). Each of these networks presents unique challenges due to their distinct architectures and learning processes. Techniques such as selective retraining, influence functions, and knowledge distillation have been proposed to address these challenges. The paper also introduces the concept of full parameter unlearning, which adjusts all trainable parameters using two key techniques: gradient ascent based on first-order information and Fisher information based on second-order information. These methods ensure comprehensive unlearning, but also introduce computational complexity. We discuss examples, potential solutions, and future research directions to make full parameter unlearning more scalable and efficient, thus providing a framework for balancing data privacy with model performance.

The informational synthesis of neural structures, processes, parameters and characteristics that allow a unified description and modeling as neural machines of natural and artificial neural systems is presented. The general informational parameters as the global quantitative measure of the neural systems computing potential as absolute and relative neural power were proposed. Neural information organizing and processing follows the way in which nature manages neural information by developing functions, functionalities and circuits related to different internal or peripheral components and also to the whole system through a non-deterministic memorization, fragmentation and aggregation of afferent and efferent information, deep neural information processing representing multiple alternations of fragmentation and aggregation stages. The relevant neural characteristics were integrated into a neural machine type model that incorporates unitary also peripheral or interface components as the central ones. The proposed approach allows overcoming the technical constraints in artificial computational implementations of neural information processes and also provides a more relevant description of natural ones.

The intracranial pressure (ICP) signal, as monitored on patients in intensive care units, contains pulses of cardiac origin where P1 and P2 subpeaks can often be observed. When calculable, the ratio of their relative amplitudes is an indicator of the patient’s cerebral compliance. This characterization is particularly informative on the overall state of the cerebrospinal system. We developed a recurrent neural network-based framework for P2/P1 ratio computation that only takes a raw ICP signal as an input. Two tasks are performed, namely pulse classification and subpeak designation. Pulse classification was achieved with an area under the curve of 0.90 on a 4,344-pulse testing dataset, while the peak designation algorithm identified pulses with a P2/P1 ratio > 1 with a 97.3% accuracy. Although it still needs to be evaluated on a larger number of labeled recordings, our automated P2/P1 ratio calculation framework appears to be a promising tool for improving bedside monitoring devices.

This paper takes a novel approach to simulating fractional stride decreases in binary artificial neural networks (ANNs), addressing the limitations of fixed integer strides typically used in traditional convolutional neural networks (CNNs). By introducing a dynamic stride adjustment mechanism that reduces stride over time using either exponential or logarithmic decay functions, the network can simulate fractional strides, enabling the capture of increasingly fine-grained spatial details at deeper layers. This method improves output resolution, feature extraction without increasing network complexity, and possible solutions for binary ANN architectures where computational efficiency is critical. Using synthetic data, we analyze how both decay functions impact output resolution in a comparative experiment. Exponential decay produces a rapid initial stride decrease, leading to early significant resolution gains, while logarithmic decay offers a more gradual and controlled resolution increase. Both methods are enhanced by an attention mechanism that selectively emphasizes important regions in feature maps, improving spatial feature extraction efficiency. Our findings demonstrate that dynamic, asymptotic stride adjustment provides an effective method for simulating fractional strides in binary ANNs, with potential applications in areas such as object detection, segmentation, super-resolution, and autonomous systems, including spacecraft navigation and hazard detection. We also examine the trade-offs between the two decay methods in terms of resolution growth and computational overhead, contributing to ongoing research in adaptive ANN architectures and binary neural network design.

An earthquake is a natural event by its general definition. This natural event is a disaster that causes significant damage, loss of life, and other economic effects that will damage the state. The possibility of predicting a natural event such as an earthquake will minimize the reasons mentioned. Data collection, data processing, and data evaluation were carried out in this study. Earthquake forecasting was performed using the data and the RNN (Recurrent Neural Network) method. The study was carried out on seismic data with a magnitude of 3.0 and above belonging to Düzce Province between 1990 and 2022. In order to increase the learning potential of the method, the b and d values of the earthquake were calculated and included in the data set, except for the earthquake magnitude. The determination of earthquakes in a specific time interval in regions of Turkey, the classification of earthquake-related seismic data using artificial neural networks, and the production of predictions for the future reveal the importance of this study.

This paper investigates the dynamic properties of artificial neural networks using differential equations and explores the influence of parameters on stability and neural oscillations. By analyzing the equilibrium point of the differential equations, we identify conditions for asymptotic stability and criteria for oscillation in artificial neural networks. Furthermore, we demonstrate how adjusting synaptic weights between neurons can effectively control stability and oscillation. The proposed model offers potential insights into the malfunctioning mechanisms of biological neural networks implicated in neurological disorders like Parkinson's disease tremors and epilepsy seizures, which are characterized by abnormal oscillations.

Multiply-accumulate calculations using a memristor crossbar array is an important method to realize neuromorphic computing. However, the memristor array fabrication technology is still immature, and it is difficult to fabricate large-scale arrays with high-yield, which restricts the development of memristor-based neuromorphic computing technology. Therefore, cascading small-scale arrays to achieve the neuromorphic computational ability that can be achieved by large-scale arrays, which is of great significance for promoting the application of memristor-based neuromorphic computing. To address this issue, we present a memristor-based cascaded framework with some basic computation units, several neural network processing units can be cascaded by this means to improve the processing capability of the dataset. Besides, we introduce a split method to reduce pressure of input terminal. Compared with VGGNet and GoogLeNet, the proposed cascaded framework can achieve 93.54% Fashion-MNIST accuracy under the 4.15M parameters. Extensive experiments with Ti/AlOx/TaOx/Pt we fabricated are conducted to show that the circuit simulation results can still provide a high recognition accuracy, and the recognition accuracy loss after circuit simulation can be controlled at around 0.26%.

Tumorigenesis is a complex biological phenomenon that includes extensive genetic and phenotypic heterogeneities and complicated regulatory mechanisms. In the recent few years, our studies demonstrate that tumor-initiating cells are similar to neural stem/progenitor cells in regulatory networks, tumorigenicity and pluripotent differentiation potential. In the review, I will make further discussion on these observations and propose a rule of cell biology by integrating these findings with evidence from developmental biology, tumor biology and evolution, which suggests that neural stemness underlies two coupled cell properties, tumorigenicity and pluripotent differentiation potential. Tumorigenicity and phenotypic heterogeneity in tumor is a result of acquirement of neural stemness in cells. The neural stemness property of tumor-initiating cells can hopefully integrate different concepts/hypotheses underlying tumorigenesis. Neural stem cells/neural progenitors and tumor-initiating cells share regulatory networks; both exhibit neural stemness, tumorigenicity and differentiation potential; both are dependent on expression or activation of ancestral genes (the atavistic effect); both rely primarily on aerobic glycolytic metabolism; both can differentiate into various cells or tissues that are derived from three germ layers, resembling severely disorganized or more severely degenerated process of embryonic development; both are enriched in long genes with more splice variants that provide more plastic scaffolds for cell differentiation, etc. The property of neural stemness might be a key point to understand tumorigenesis and pluripotent differentiation potential, and possibly explain certain pathological observations in tumors that have been inexplicable. Therefore, behind the complexity of tumorigenesis might be a general rule of cell biology, i.e., neural stemness represents the ground state of cell tumorigenicity and pluripotent differentiation potential.

This research investigates the application of deep learning in sentiment analysis of Canadian maritime case law. It offers a framework for improving maritime law and legal analytics policy-making procedures. The automation of legal document extraction takes center stage, underscoring the vital role sentiment analysis plays at the document level. Therefore, this study introduces a novel strategy for sentiment analysis in Canadian maritime case law, combining sentiment case law approaches with state-of-the-art deep learning techniques. The overarching goal is to systematically unearth hidden biases within case law and investigate their impact on legal outcomes. Employing Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based models, this research achieves a remarkable accuracy of 98.05% for categorizing instances. In contrast, Conventional Machine Learning techniques such as Support Vector Machines (SVM) yield an accuracy rate of 52.57%, Naive Bayes at 57.44 %, and Logistic Regression at 61.86%. The superior accuracy of the CNN and STM model combination underscores its usefulness in legal sentiment analysis, offering promising future applications in diverse fields like legal analytics and policy design. These findings mark a significant choice for AI-powered legal tools, presenting more sophisticated and sentiment-aware options for the legal profession.

A chronically stable electrode material with low impedance for recording neural activity and high charge-injection capacity for functional electro-stimulation is desirable for the fabrication of implantable microelectrode arrays that aims to restore impaired or lost neurological functions in humans. For this purpose, we have investigated the electrochemical properties of sputtered ruthenium oxide (RuOx) electrode coatings, deposited on planar microelectrode arrays, using an inorganic model of interstitial fluid (model-ISF) at 37o C as the electrolyte. Through a combination of cyclic voltammetry (CV) and electrochemical impedance spectroscopy (EIS) modelling study we have established the contribution of faradaic reaction as the major charge-injection contributor within the safe neural stimulation potential window of 0.6 V vs Ag|AgCl. We have also established the reversibility of the charge-injection process for sputtered RuOx film by applying constant charge-per-phase current stimulations at different pulse widths and by comparing the magnitudes of leading and trailing access voltages during voltage transient measurements. Finally, the impedance of the sputtered RuOx film was found to be reasonably comparable in both its oxidized and reduced states, although electronic contribution from capacitive double-layer was found to be slightly higher for the completely oxidized film around 0.6 V than its reduced counterpart around -0.6V.

Our objective in this work is to demonstrate how Physics-Informed Neural Networks, a type of deep learning technology, can be utilized to examine the mechanical properties of a helicopter blade. The blade is regarded as a prismatic cantilever beam that is exposed to triangular loading, and comprehending its mechanical behavior is of utmost importance in the aerospace field. PINNs utilize the physical information, including differential equations and boundary conditions, within the loss function of the neural network to approximate the solution. Our approach determines the overall loss by aggregating the losses from the differential equation, boundary conditions, and data. We employed a Physics-Informed Neural Network (PINN) and an Artificial Neural Network (ANN) with equivalent hyperparameters to solve a fourth-order differential equation. By comparing the performance of the PINN model against the analytical solution of the equation and the results obtained from the ANN model, we have conclusively shown that the PINN model exhibits superior accuracy, robustness, and computational efficiency when addressing high-order differential equations that govern physics-based problems. In conclusion, the study demonstrates that PINN offers a superior alternative for addressing solid mechanics problems with applications in the aerospace industry.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-OPS (TOPS - operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

This research investigates the implementation of complex exponential-based neurons in FPGA, which can pave the way for implementing bio-inspired spiking neural networks to compensate for the existing computational constraints in conventional artificial neural networks. The increasing use of extensive neural networks and the complexity of models in handling big data lead to higher power consumption and delays. Hence, finding solutions to reduce computational complexity is crucial for addressing power consumption challenges. The complex exponential form effectively encodes oscillating features like frequency, amplitude, and phase shift, streamlining the demanding calculations typical of conventional artificial neurons through levering simple phase addition of complex exponential functions. The article implements such a two-neuron and a multi-neuron neural model using Xilinx system generator and Vivado design suite, employing 8-bit, 16-bit, and 32-bit fixed-point data format representations. The study evaluates the accuracy of the proposed neuron model across different FPGA implementations while also providing a detailed analysis of operating frequency, power consumption, and resource usage for the hardware implementations. BRAM-based Vivado designs outperformed Simulink regarding speed, power, and resource efficiency. Specifically, the Vivado BRAM-based approach supported up to 128 neurons, showcasing optimal LUT and FF resource utilization. Such outcomes accommodate choosing the optimal design procedure for implementing spiking neural networks on FPGAs.

Neural stem cell (NSC) based therapies are at the forefront of regenerative medicine strategies to combat illness and injury of the central nervous system (CNS). In addition to their ability to produce new cells, NSCs secrete a variety of products, known as the NSC secretome, that have been shown to ameliorate CNS disease pathology and promote recovery. As pre-clinical and clinical research to harness the NSC secretome for therapeutic purposes advances, a more thorough understanding of the endogenous NSC secretome can provide useful insight into the functional capabilities of NSCs. In this review, we focus on research investigating the autocrine and paracrine functions of the endogenous NSC secretome across life. We also compare the NSC secretome across species, finding signs of conserved parallels between rodent, human and zebrafish NSC secretomes. Throughout development and adulthood, we find evidence that the NSC secretome is a critical component of how endogenous NSCs regulate themselves and their niche. We also find gaps in current literature, most notably in the clinically relevant domain of endogenous NSC paracrine function in the injured CNS. Future investigations to further define the endogenous NSC secretome and its role in CNS tissue regulation are necessary to bolster our understanding of NSC-niche interactions and to aid in the generation of safe and effective NSC-based therapies.

Biological neural network (BNN) is the core brain for creature to accomplish its intelligent behaviors through unique network structure and mechanisms. It also inspires controlling human-designed autonomous agents including robots to generate more advanced intelligent behaviors by mimicking the neural control mechanism of creatures at a deeper and more interactive level. Here, we constructed a whole-brain neural network model of Caenorhabditis elegans (C. elegans), which characterizes the electrochemical processes at the level of cellular synapse. The neural network simulation integrates computational programming and visualization of neurons and synapse connections of C. elegans, containing the specific controllable circuits summarization with their dynamic characteristics within the whole network. To illustrate its particular intelligent control capability in terms of robotics, we introduce the first innovative methodology for applying the BNN model to a 12-legged robot’s movement control based on the established numerical simulation platforms. We accomplished and designed two methods and corresponding encoding processes, one involving orientation control and the other involving locomotion generation, to demonstrate the intelligent control performance of BNN. Both simulation and experiment results indicate that the robot exhibits stronger autonomy and more intelligent moving performance under the control of BNN. We then summarized the contributions for digitalizing the C. elegans’ whole-brain neural network in real-time and utilizing it to control robot in a closed loop to validate its advanced intelligence control ability in terms of a scientific robot.

Speech recognition, or automatic speech recognition (ASR), is a technology designed to convert spoken language into text using software. However, conventional ASR methods involve several distinct components, including language, acoustic, and pronunciation models with dictionaries. This modular approach can be time-consuming and may influence performance. In this study, we propose a method that streamlines the speech recognition process by incorporating a unified recurrent neural network (RNN) architecture. Our architecture integrates a convolutional neural network (CNN) with an RNN and employs a connectionist temporal classification (CTC) loss function. Key experiments were carried out using a dataset comprising 576,656 valid sentences, using erosion techniques. Evaluation of the model performance, measured by the word error rate (WER) metric, demonstrated remarkable results, achieving a WER of 2%. This approach has significant implications for the realm of speech recognition, as it alleviates the need for labor-intensive dictionary creation, enhancing the efficiency and accuracy of ASR systems, and making them more applicable to real-world scenarios. For future enhancements, we recommend the inclusion of dialectal and spontaneous data in the dataset to broaden the model's adaptability. Additionally, fine-tuning the model for specific tasks can optimize its performance for targeted objectives or domains, further enhancing its effectiveness in those areas.

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have been utilised to forecast the foreign exchange market (Forex). However, such models usually exhibit unstable behaviour, as data perturbations often downgrade the functionality of the entire network due to their monolithic architecture. Hence, this study proposes a novel neuroscience-informed modular network applied to the closing prices and the sentiments retrieved from Yahoo Finance and Twitter APIs, aiming to anticipate better price fluctuations in Euro to British Pound Sterling (EUR/GBP) rather than monolithic methods. The proposed model is based on a new modular CNN, replacing pooling layers with orthogonal kernel initialisation RNNs coupled with Monte Carlo Dropout (MCD), namely MCoRNNMCD. It combines two modules: i) a convolutional simple RNN and ii) a convolutional Gated Recurrent Unit (GRU), where orthogonality and MCD are added to reduce the overfitting, assessing each module's uncertainty. These parallel feature extraction modules concatenate their outputs to a final three-layer Artificial Neural Network (ANN) decision-making module. A comprehensive comparison viewing objective evaluation metrics such as the Mean Square Error (MSE) proved that the proposed MCoRNNMCD-ANN outperformed single CNN, LSTM, GRU, and the state-of-the-art hybrid BiCuDNNLSTM, CLSTM, CNN-LSTM, and LSTM-GRU in forecasting hourly EUR/GBP closing price fluctuations.

In recent years, Neural networks are increasingly deployed in various fields to learn complex patterns and make accurate predictions. However, designing an effective neural network model is a challenging task that requires careful consideration of various factors, including architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different learning approaches, including convolutional neural networks (CNN), deep neural networks (DNN), and recurrent neural networks (RNN) in image recognition, natural language processing, and time series analysis. Besides, it discusses the benefits of choosing the ideal values for the different components of ANN, such as the number of Input/output layers, hidden layers number, activation function type, epochs number, and model type selection, which help improve the model performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

Computer-aided image diagnostics (CAD) have been used in many fields of diagnostic medicine. It relies heavily on classical computer vision and artificial intelligence. Quantum neural network (QNN) has been introduced by many researchers around the world and presented recently by research corporations such as Microsoft, Google, and IBM. In this paper, the investigation of the validity of using the QNN algorithm for machine-based breast cancer detection was performed. To validate the learnability of the QNN, a series of learnability tests were performed alongside with classical convolutional neural network (CCNN). QNN is built using the Cirq library to perform the assimilation of quantum computation on classical computers. Series of investigations were performed to study the learnability characteristics of QNN and CCNN under the same computational conditions. The comparison was performed for real Mammogram data sets. The investigations showed success in terms of recognizing the data and training. Our work shows better performance of QNN in terms of successfully training and producing a valid model for smaller data set compared to CCNN.

In this study, a computational method is proposed that employs Recurrent Convolutional 1 Neural Networks, utilizing meteorological radar images to forecast storm movement and intensity up 2 to 3 hours ahead, a process known as nowcasting. For this purpose, images from a radar situated in 3 southern Brazil were used. These data are publicly accessible on the website of the National Institute 4 for Space Research (INPE) in Brazil. The approach involves evaluating a spatiotemporal learning 5 recurrent convolutional neural network called PredRNN++. The results were validated through 6 case studies of storms within the radar’s coverage area. To evaluate the performance of the neural 7 network, both visual assessments and metrics such as RMSE and SSIM were employed. The findings 8 indicate that PredRNN++ was effective in simulating the shape and location of the meteorological 9 system.

Optical artificial neural networks (ONNs) — analog computing hardware tailored for machine learning [1,2] — have significant potential for ultra-high computing speed and energy efficiency [3]. We propose a new approach to architectures for ONNs based on integrated Kerr micro-comb sources [4] that is programmable, highly scalable and capable of reaching ultra-high speeds. We experimentally demonstrate the building block of the ONN — a single neuron perceptron — by mapping synapses onto 49 wavelengths of a micro-comb to achieve a high single-unit throughput of 11.9 Giga-FLOPS at 8 bits per FLOP, corresponding to 95.2 Gbps. We test the perceptron on simple standard benchmark datasets — handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. This performance is a direct result of the record small wavelength spacing (49GHz) for a coherent integrated microcomb source, which results in an unprecedented number of wavelengths for neuromorphic optics. Finally, we propose an approach to scaling the perceptron to a deep learning network using the same single micro-comb device and standard off-the-shelf telecommunications technology, for high-throughput operation involving full matrix multiplication for applications such as real-time massive data processing for unmanned vehicle and aircraft tracking.

Accurate prediction of investment returns is crucial for financial decision-making. Traditional models often struggle with the complexity and variability inherent in financial data. This article presents an advanced predictive model integrating three deep neural networks, pretraining, and ensemble learning to enhance prediction accuracy. By leveraging a combination of DNNs for handling different complexities of data, and a CNN for efficient feature extraction, our approach significantly outperforms traditional methods. Our model incorporates sophisticated data handling techniques, including extensive preprocessing and data augmentation, and benefits from pretraining and ensemble methods to improve robustness and performance. This comprehensive framework offers a robust solution for investment return prediction, addressing the shortcomings of existing models and providing enhanced accuracy and reliability.

Astronomy is entering an unprecedented era of Big Data science, driven by missions like the ESA's Gaia telescope, which aims to map the Milky Way in three dimensions. Gaia's vast dataset presents a monumental challenge for traditional analysis methods. The sheer scale of this data exceeds the capabilities of manual exploration, necessitating the utilization of advanced computational techniques. In response to this challenge, we developed a novel approach leveraging deep learning to estimate the metallicity of fundamental mode (ab-type) RR Lyrae stars from their light curves in the Gaia optical G-band. Our study explores applying deep learning techniques, particularly advanced neural network architectures, in predicting photometric metallicity from time-series data. Moreover, our deep learning models demonstrated notable predictive performance over experimentation and analysis. Through GRU-based models we achieved a low mean absolute error of 0.056 and a high $R^2$ regression performance of 0.9401 measured by cross-validation, showcasing the effectiveness of our approach in accurately estimating metallicity values. Our work underscores the importance of deep learning in astronomical research, particularly with large datasets from missions like Gaia. By harnessing the power of deep learning methods, we can provide precision in analyzing vast datasets, contributing to more precise and comprehensive insights into complex astronomical phenomena.

This paper investigates the use of Physics-Informed Neural Networks (PINNs) in active learning cycles. We defined two scenarios: one semi-supervised and the other fully supervised. PINNs emphasize the integration of physical laws into neural networks to improve the predictive performance of vanilla neural networks and to enhance the efficiency of traditional methods for solving partial differential equations (PDEs). Key contributions include adapting existing computational frameworks to enable the use of Graph Neural Networks for solving problems that require the calculation of gradients on unstructured triangle meshes, a query strategy focusing on the physical loss, and a comparative analysis of this strategy against random sampling across both defined scenarios. This work establishes a foundation for future research aimed at expanding the application of Physics-Informed Graph Neural Networks (PIGNN) using active learning and addressing real-world problems in fluid dynamics and electrodynamics.

This article presents a detailed examination of the methodology and modeling tools utilized to analyze gas flow in pipelines, rooted in the fundamental principles of gas dynamics. The methodology integrates numerical simulations with modern neural network techniques, particularly focusing on the Physics-Informed Neural Network (PINN) method. This innovative approach combines artificial neural networks (ANNs) with physical equations, offering a more efficient and accurate way to model various complex processes and phenomena. The proposed mathematical model, based on the Euler equation, has been meticulously implemented using the Python language. Verification with analytical solutions ensures the accuracy and reliability of the computations. In the research, a comprehensive comparative analysis was conducted between results obtained using the PINN method and those from conventional Computational Fluid Dynamics (CFD) approaches. The analysis highlighted the advantages of the PINN method, which produced smoother pressure and velocity fluctuation profiles while reducing computation time, demonstrating its potential as a transformative modeling tool. The data derived from this study are of paramount importance for ensuring ongoing energy supply reliability and can also be used to create predictive models related to gas behavior in pipelines. The application of modeling techniques for gas flow simulation has the potential to revolutionize the integrity of our energy infrastructure and utilization of gas resources. However, it is crucial to emphasize that the effectiveness of such models relies on continuous monitoring and frequent updates to ensure alignment with real-world conditions. This research not only contributes to a deeper understanding of compressible gas flows but also underscores the crucial role of advanced modeling methodologies in the sustainable management of gas resources for both current and future generations.

The application of artificial intelligence to point-of-care testing (POCT) disease detection has become a hot research field, in which breath detection which detects the patient's exhaled VOCs combined with sensor arrays of Convolutional Neural Network (CNN) algorithms as a new lung cancer detection is attracting more researchers' attention. However, due to low accuracy, high complexity computation and large number of parameters make the CNN algorithms difficult transplant to the embedded system of POCT devices. A lightweight neural network (LTNet) in this work is proposed to deal with this problem, and meanwhile, achieve high-precision classification of acetone and ethanol gases, which are respiratory markers for lung cancer patients. Compared to currently popular lightweight CNN models such as EfficientNet, LTNet has fewer parameters (32K) and its training weight size is only 0.155 MB. LTNet achieved an overall classification accuracy of 99.06% and 99.14% in the own mixed gas dataset and the University of California (UCI) dataset, which are both higher than that of the five existing models’, and it also offers the shortest training (844.38s and 584.67s) and inference time (23s and 14s) in the same validation set. Compared to the existing CNN models, LTNet is more suitable for resource-limited POCT devices.

Dynamic neural networks (DNN) are types of artificial neural networks (ANN) that are designed to work with sequential data where context in time is important. In contrast to traditional static neural networks that process data in a fixed order, dynamic neural networks use information about past inputs, which is important if dynamic of a certain process is emphasized. They are widely used in natural language processing, speech recognition and time series prediction. In industrial processes, their use is interesting for the prediction of difficult-to-measure process variables. In an industrial process of isomerization, it is crucial to measure the quality attributes affecting the octane number of gasoline. Process analyzers that are commonly used for this purpose are expensive and subject to failures, therefore, in order to achieve continuous production in case of malfunction, mathematical models for estimating the product quality attributes are imposed as a solution. In this paper, mathematical models were developed using dynamic recurrent neural networks (RNN), i.e., their subtype of a long short-term memory (LSTM) architecture. The results of the developed models were compared with the results of several types of other data-driven models developed for an isomerization process, such as multilayer perceptron (MLP) artificial neural networks, support vector machines (SVM) and dynamic polynomial models. The obtained results are satisfactory, which suggests a good possibility of application.

This study aimed at exploring what role artificial intelligence techniques could play in the futural numerical analysis. In this paper, a convolutional neural network techniques based on modified loss function is proposed as a surrogate of finite element method(FEM). Several surrogate-based physics-informed neural networks(PINNs) are developed to solve representative boundary value problems based on elliptic partial differential equations (PDEs). Results from the proposed surrogate-based approach are in good agreement with ones from conventional FEM. It is found that modification of the loss function could improve the prediction accuracy of the neural network. It is indicated that to some extent the artificial intelligence technique could replace conventional numerical analysis as a great surrogate model.

Forest fire identification is important for forest resource protection. Effective monitoring of forest fires requires the deployment of multiple monitors with different viewpoints, while most traditional recognition models can only recognize images from a single source. By ignoring the information from images with different viewpoints, these models produce high rates of missed and false alarms. In this paper, we propose a graph neural network model based on the similarity of dynamic features of multi-view images to improve the accuracy of forest fire recognition. The input features of the nodes on the graph are converted into relational features of different gallery pairs by establishing pairs (nodes) representing different viewpoint images and gallery images. The new feature library relationship is used to update the image gallery with dynamic features in order to achieve the estimation of similarity between images and improve the image recognition rate of the model. In addition, to reduce the complexity of image pre-processing process and extract key features in images effectively, this paper also proposes a dynamic feature extraction method for fire regions based on image segment ability. By setting the threshold value of HSV color space, the fire region is segmented from the image, and the dynamic features of successive frames of the fire region are extracted. The experimental results show that, compared with the baseline method Resnet, this paper's method is more effective in identifying forest fires, and its recognition accuracy is improved by 2%. And the scheme of this paper can adapt to different forest fire scenes, with better generalization ability and anti-interference ability.

Structural Health Monitoring (SHM) plays a vital role in ensuring the health status of a wide range of structures, such as bridges, buildings and large infrastructure in general. The advantages of this process can be further enhanced by incorporating more numerical and statistical approaches into traditional methods, such as Finite Element Analysis and Machine Learning. In this study a bridge structure is examined, and neural networks are trained with data derived from finite element analyses under static loads and dynamic excitations. Initially, a binary classification problem is addressed, where numerically trained classifiers are tasked with identifying whether the structure is in a healthy state or not. This category is further divided into two subcategories, depending on the extent of the damage present in the structure. Subsequently, a multi-class classification problem is defined, where three different damage classes of the same extent are considered, and the trained network is required to distinguish between them. Finally, conclusions are drawn from the results of the study regarding the model error parameter, the impact of the damage size, as well as the types of neural networks and training data used.

Characterization of cancer cells and neural stem cells indicates that tumorigenicity and pluripotency are coupled cell properties determined by neural stemness, and tumorigenesis represents a process of progressive loss of original cell identity and gain of neural stemness. This reminds of a most fundamental process required for the development of the nervous system and body axis during embryogenesis, i.e., embryonic neural induction. Neural induction is that, in response to extracellular signals that are secreted by the Spemann-Mangold organizer in amphibians or the node in mammals and inhibit epidermal fate in ectoderm, the ectodermal cells lose their epidermal fate and assume the neural default fate and consequently, turn into neuroectodermal cells. They further differentiate into the nervous system and also some non-neural cells via interaction with adjacent tissues. Failure in neural induction leads to failure of embryogenesis, and ectopic neural induction due to ectopic organizer or node activity or activation of embryonic neural genes causes a formation of secondary body axis or a conjoined twin. During tumorigenesis, cells progressively lose their original cell identity and gain of neural stemness, and consequently, gain of tumorigenicity and pluripotency, due to various intra-/extracellular insults in cells of a postnatal animal. Tumorigenic cells can be induced to differentiation into normal cells and integrate into normal embryonic development within an embryo. However, they form tumors and cannot integrate into animal tissues/organs in a postnatal animal because of lack of embryonic inducing signals. Combination of studies of developmental and cancer biology indicates that neural induction drives embryogenesis in gastrulating embryos but a similar process drives tumorigenesis in a postnatal animal. Tumorigenicity is the manifestation of aberrant occurrence of pluripotent state in a postnatal animal. Pluripotency and tumorigenicity are both but different manifestations of neural stemness in pre- and postnatal stage, respectively, of animal life. The unique property of neural stemness is derived from the evolutionary advantage of neural genes and the neural-biased state of the last common unicellular ancestors of metazoan. Based on these findings, I discuss about some confusion in cancer research, propose to distinguish the causality and associations and discriminate causal and supporting factors involved in tumorigenesis, and suggest revisiting the focus of cancer research.

This paper introduces a novel architecture—bidirectional optical neural network (BONN)—for providing backward connections alongside forward connections in artificial neural networks (ANNs). BONN incorporates laser diodes and photodiodes and exploits the properties of Köhler illumination to establish optical channels for backward directions. Thus, it has bidirectional functionality that is crucial for algorithms such as the backpropagation algorithm. BONN has a scaling limit of 96 × 96 for input and output arrays, and a throughput of 8.5 × 1015 MAC/s. While BONN's throughput may rise with additional layers for continuous input, limitations emerge in the backpropagation algorithm as its throughput doesn't scale with layer count. Successful BONN-based implementation of the backpropagation algorithm requires the development of fast spatial light modulator to accommodate frequent data flow changes. A two-mirror-like BONN and its cascaded extension are alternatives for multilayer emulation, and they help save hardware space and increase the parallel throughput for inference. An investigation into the application of the clustering technique to BONN revealed its potential to help overcome scaling limits and to provide full interconnections for backward directions between doubled input and output ports. BONN's bidirectional nature holds promise for enhancing supervised learning in ANNs and increasing hardware compactness.

Artificial Intelligence (AI) made its first appearance in 1956, and since then it has progressively introduced itself in healthcare system and patients’ information and care. AI functions can be grouped under the following headings: Machine Learning (ML), Deep Learning (DL), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Computer Vision (CV). Chronic limb-threatening ischemia (CLTI) represents the last stage of peripheral artery disease (PAD), and has increased over recent years, together with the rise in prevalence of diabetes and population ageing. Nowadays AI grants the possibility of developing new diagnostic and treatment solutions in the vascular field, given the possibility of accessing clinical, biological, and imaging data. By assessing the vascular anatomy in every patient, as well as the burden of atherosclerosis, and classifying the level and degree of disease, sizing and planning the best endovascular treatment, defining the perioperative complications risk, integrating experiences and resources between different specialties, identifying latent PAD, thus offering evidence-based solutions and guiding surgeons in the choice of the best surgical technique, AI challenges the role of the physician’s experience in PAD treatment.

Partial differential equations (PDEs) are used to describe a wide range of phenomena, such as heat transfer, fluid dynamics, and quantum mechanics. By solving PDEs, we can ob- tain insights into the behavior of the system and make predictions about its future evolution. Conventional numerical methods for obtaining the approximate solutions of PDEs may re- quire extensive computational resources and time, especially for complex PDEs and large- scale problems. The recently developed physics-informed neural network (PINN) has shown promise in a variety of scientific and engineering fields by incorporating physical laws into the loss functions of the neural network (NN). In addition, the training of PINN does not require ground truth data, but it has poor generalization ability to unseen domains. On the other hand, a convolutional neural network (CNN) has fast inference and better generalization ability, but it requires a large amount of training data. Taking the advantages of PINN and CNN by using Legendre multiwavelets (LMWs) as basis functions, we introduce a new method to approach the PDEs in this paper, namely Physics-Informed Legendre Multiwavelets CNN (PiLMWs-CNN), in order to continuously approximate a grid-based state representation that can be handled by a CNN. PiLMWs-CNN enable us to train our models using only physics-informed loss functions without any pre- computed training data, simultaneously providing fast and continuous solutions that gener- alize to previously unknown domains. In particular, the LMWs can simultaneously possess compact support, orthogonality, symmetry, high smoothness, and high approximation order. Compared to orthonormal polynomial (OP) bases, the approximation accuracy can be greatly increased and computation costs can be significantly reduced by using LMWs. We applied PiLMWs-CNN to approximate the damped wave equation, incompressible Navier-Stokes (N- S) equation, and two-dimensional heat conduction equation. The experimental results show that this method provides more accurate, efficient, and fast convergence with better stability when approximating the solution of PDEs.

Human activity recognition (HAR) using wearable sensors is gaining popularity in the healthcare domain, enabling continuous monitoring and timely interventions. The conventional HAR systems are centralised and need data sharing with a server for processing and model training. However, this data sharing leads to various challenges, such as privacy concerns, increased communication and storage costs, inefficient computational algorithms, data scarcity due to unwillingness to share, and difficulty achieving high accuracy and personalised models simultaneously. To overcome these challenges, this paper introduces a privacy and energy-aware federated learning (FL) framework for model training without data sharing. The proposed scheme incorporates spiking neural networks (SNNs), which are particularly suited for resource-constrained devices due to their event-driven information processing. The key idea is to synergise the strengths of SNNs and long short-term memory (LSTM) to create a hybrid model for robust HAR. The proposed hybrid spiking LSTM (S-LSTM) model uses the LSTM as an input layer to capture temporal dependencies efficiently, followed by spiking layers that replace the typical activation using leaky integrate-and-fire neurons. We employ the combination of surrogate gradient learning and backpropagation through time (BPTT) to update the weights of SNN layers in a supervised manner. The performance of the proposed model is evaluated using two publicly available datasets for both indoor and outdoor scenarios. The results show that the proposed S-LSTM performs much better than spike convolutional neural networks (S-CNN) and simple CNN, achieving the accuracy of 97% and 87% for indoor and outdoor datasets, respectively. Personalisation, also known as fine-tuning the global model using local data, is crucial in HAR. It allows the model to adapt to individual behaviour and preferences, making predictions more accurate. The results also show that personalisation improves the performance of participants up to 9% on average. Additionally, analysis with 50% random client participation reduces communication costs by 50% with minimal impact on accuracy.

A scalable optical convolutional neural network (SOCNN) based on free-space optics and Koehler illumination was proposed to address the limitations of the previous 4f correlator system. Unlike Abbe illumination, Koehler illumination provides more uniform illumination and reduces crosstalk. SOCNN allows for scaling up of the input array and the use of incoherent light sources. Hence, the problems associated with 4f correlator systems can be avoided. We analyzed the limitations in scaling the kernel size and parallel throughput and found that SOCNN can offer a multilayer convolutional neural network with massive optical parallelism.

This study uses machine learning to predict the convergence results of the Duffing equation with and without damping. The Duffing equation represents a nonlinear second-order differential equation with interesting behavior in undamped free vibration and forced vibration with damping. Convergence alternates randomly between 1 and -1 in undamped free vibration, depending on initial conditions. For forced vibration with damping, multiple factors influence vibration patterns. We utilize the fourth-order Runge-Kutta method to collect convergence results for both conditions. Machine learning techniques, specifically the long short term memory (LSTM) and LSTM-Neural Network (LSTM-NN) method, are employed to predict these convergence values. The LSTM-NN model is a hybrid approach that combines the LSTM method with the addition of hidden layers of neurons. Both the LSTM and LSTM-NN models are thoroughly explored and analyzed in this research. The research process involves three stages: data preprocessing, training, and verification. The results show that the LSTM-NN model becomes more adept at predicting binary datasets, boasting an impressive accuracy of up to 98%. However, when it comes to predicting multiple solutions, the traditional LSTM method outperforms the LSTM-NN approach.

Convolutional neural networks (CNNs), inspired by biological visual cortex systems, are a powerful category of artificial neural networks that can extract the hierarchical features of raw data to greatly reduce the network parametric complexity and enhance the predicting accuracy. They are of significant interest for machine learning tasks such as computer vision, speech recognition, playing board games and medical diagnosis [1-7]. Optical neural networks offer the promise of dramatically accelerating computing speed to overcome the inherent bandwidth bottleneck of electronics. Here, we demonstrate a universal optical vector convolutional accelerator operating beyond 10 Tera-FLOPS (floating point operations per second), generating convolutions of images of 250,000 pixels with 8-bit resolution for 10 kernels simultaneously — enough for facial image recognition. We then use the same hardware to sequentially form a deep optical CNN with ten output neurons, achieving successful recognition of full 10 digits with 900 pixel handwritten digit images with 88% accuracy. Our results are based on simultaneously interleaving temporal, wavelength and spatial dimensions enabled by an integrated microcomb source. This approach is scalable and trainable to much more complex networks for demanding applications such as unmanned vehicle and real-time video recognition.

Speech recognition, also known as automatic speech recognition (ASR), is a technology that enables software to transcribe spoken language into text. However, existing Amharic ASR methods require multiple separate blocks, such as language, acoustic, and pronunciation models with dictionaries, which can be time-consuming and influence performance. This study proposes an approach that replaces much of the speech pipeline with a single recurrent neural network (RNN) architecture. Our proposed architecture is based on a hybrid approach that combines a convolutional neural network (CNN) with a recurrent neural network (RNN) and a connectionist temporal classification (CTC) loss function. We conducted several experiments with noisy audio data that contain 20,000 valid sentences. The model was evaluated using the word error rate (WER) metric, achieving impressive results of 7% WER on noisy data. This approach has significant implications for the field of speech recognition, as it reduces the human effort required to create dictionaries and improves the efficiency and accuracy of ASR systems, making them more practical for real-world applications.

Creating self-learning algorithms, developing deep neural networks and improving other methods that "learn" for various areas of human activity is the main goal of the theory of machine learning. It helps to replace the human with a machine, aiming to increase the quality of production. The theory of artificial neural networks, which already have replaced the humans in problems of detection of moving objects, recognition of images or sounds, time series prediction, big data analysis and numerical methods remains the most dispersed branch of the theory of machine learning. Certainly, for each area of human activity it is necessary to select appropriate neural network architectures, methods of data processing and some novel tools from applied mathematics. But the universal problem for all these neural networks with specific data is the achieving the highest accuracy in short time. Such problem can be resolved by increasing sizes of architectures and improving data preprocessing, where the accuracy rises with the training time. But there is a possibility to increase the accuracy without time growing, applying optimization methods. In this survey we demonstrate existing optimization algorithms of all types, which can be used in neural networks. There are presented modifications of basic optimization algorithms, such as stochastic gradient descent, adaptive moment estimation, Newton and quasi-Newton optimization methods. But the most recent optimization algorithms are related to information geometry, for Fisher-Rao and Bregman metrics. This approach in optimization extended the theory of classic neural networks to quantum and complex-valued neural networks, due to geometric and probabilistic tools. There are provided applications of all introduced optimization algorithms, what delighted many kinds of neural networks, which can be improved by including any advanced approaches in minimization of the loss function. Afterwards, we demonstrated ways of developing optimization algorithms in further researches, engaging neural networks with progressive architectures. Classical gradient based optimizers can be replaced by fractional order, bilevel and, even, gradient free optimization methods. There is a possibility to add such analogues in graph, spiking, complex-valued, quantum and wavelet neural networks. Besides the usual problems of image recognition, time series prediction, object detection, there are many are other tasks for modern theory of machine learning, such as solving problem of quantum computations, partial differential and integro-differential equations, stochastic processes and Brownian motion, making decisions and computer algebra.

Standard ANNs lack flexibility when handling corrupted input due to their fixed structure. Spiking Neural Networks (SNNs) can utilize biological temporal coding features, such as noise-induced stochastic resonance and dynamical synapses to increase a model’s performance when its parameters are not optimized for a given input. Using the analog XOR task as a simplified Convolutional Neural Network analog, this paper demonstrates two key results: (1) SNNs solve the problem linearly inseparable in ANN with over 10% improvement in the optimal setting, and (2) in SNNs, the synaptic noise and dynamical synapses compensate for non-optimal parameters, achieving near-optimal results.

In the pursuit of energy efficiency and reduced environmental impact, adequate ventilation in enclosed spaces is essential. This study presents a hybrid neural network model designed for real-time monitoring and prediction of environmental variables. The system comprises two phases: An IoT hardware-software platform for data acquisition and decision-making, and a hybrid model combining short-term memory and convolutional recurrent structures. The results are promising and hold potential for integration into parallel processing AI architectures.

In computer vision and image analysis, Convolutional Neural Networks (CNNs) and other deep learning models are at the forefront of research and development. These advanced models have proven to be highly effective in tasks related to computer vision. One technique that has gained prominence in recent years is the construction of ensembles using Deep CNNs. These ensembles typically involve combining multiple pre-trained CNNs to create a more powerful and robust network. The purpose of this study is to evaluate the effectiveness of building CNN ensembles by combining several advanced techniques. Tested here are CNN ensembles constructed by replacing ReLU layers with different activation functions, employing various data augmentation techniques, and utilizing several algorithms, including some novel ones, that perturb network weights. Experimental results performed across many data sets representing different tasks demonstrate that our proposed methods for building deep ensembles produces superior results. All the resources required to replicate our experiments are available at https://github.com/LorisNanni.

We present a simplified computational rule for the back-propagation formulas for artificial neural networks. In this work, we provide a generic two-step rule for the back-propagation algorithm in matrix notation. Moreover, this rule incorporates both the forward and backward phases of the computations involved in the learning process. Specifically, this recursively computing rule permits the propagation of the changes to all synaptic weights in the network, layer by layer, efficiently. In particular, we use this rule to compute both the up and down partial derivatives of the cost function of all the connections feeding into the output layer.

Numerous approaches address over-fitting in neural networks: by imposing a penalty on the parameters of the network (L1, L2, etc); by changing the network stochastically (drop-out, Gaussian noise, etc.); or by transforming the input data (batch normalization, etc.). In contrast, we aim to ensure that a minimum amount of supporting evidence is present when fitting the model parameters to the training data. This, at the single neuron level, is equivalent to ensuring that both sides of the separating hyperplane (for a standard artificial neuron) have a minimum number of data points — noting that these points need not belong to the same class for the inner layers. We firstly benchmark the results of this approach on the standard Fashion-MINST dataset, comparing it to various regularization techniques. Interestingly, we note that by nudging each neuron to divide, at least in part, its input data, the resulting networks make use of each neuron, avoiding a hyperplane completely on one side of its input data (which is equivalent to a constant into the next layers). To illustrate this point, we study the prevalence of saturated nodes throughout training, showing that neurons are activated more frequently and earlier in training when using this regularization approach. A direct consequence of the improved neuron activation is that deep networks are now easier to train. This is crucially important when the network topology is not known a priori and fitting often remains stuck in a suboptimal local minima. We demonstrate this property by training a network of increasing depth (and constant width): most regularization approaches will result in increasingly frequent training failures (over different random seeds) whilst the proposed evidence-based regularization significantly outperforms in its ability to train deep networks.

Background: Disruption of alternative splicing (AS) is frequently observed in cancer and it might represent an important signature for tumor progression and therapy. Exon skipping (ES) represents one of the most frequent AS events and in non-small cell lung cancer (NSCLC) MET exon 14 skipping was shown to be targetable. Methods: We constructed a neural network (NN) specifically designed to detect MET exon 14 skipping events using RNAseq data. Furthermore, for discovery purpose we also developed a sparsely connected autoencoder to identify uncharacterized MET isoforms. Results: The NN had 100% Met exon 14 skipping detection rate, when tested on a manually curated set of 690 TCGA bronchus and lung samples. When globally applied to 2605 TCGA samples, we observed that the majority of false positives was characterized by a blurry coverage of exon 14, but interesting they share a common coverage peak in the second intron and we speculate that this event could be the transcription signature of a LINE1-MET fusion. Conclusions: Taken together our results indicate that neural networks can be an effective tool to provide a quick classification of pathological transcription events and sparsely connected autoencoders could represent the basis for the development of an effective discovery tool.

Optical artificial neural networks (ONNs) have significant potential for ultra-high computing speed and energy efficiency. We report a novel approach to ONNs that uses integrated Kerr optical micro-combs. This approach is programmable and scalable and is capable of reaching ultra-high speeds. We demonstrate the basic building block ONNs — a single neuron perceptron — by mapping synapses onto 49 wavelengths to achieve an operating speed of 11.9 x 10⁹ operations per second, or Giga-OPS, at 8 bits per operation, which equates to 95.2 gigabits/s (Gbps). We test the perceptron on handwritten-digit recognition and cancer-cell detection — achieving over 90% and 85% accuracy, respectively. By scaling the perceptron to a deep learning network using off-the-shelf telecom technology we can achieve high throughput operation for matrix multiplication for real-time massive data processing.

Real-world time series data sets contain a combination of linear and nonlinear patterns, making the time series forecasting problem more challenging. In this paper, a new hybrid methodology is introduced for forecasting univariate time series data sets using a multiplicative error modeling approach. An autoregressive integrated moving average (ARIMA) model is combined with an autoregressive neural network (ARNN) for improving the predictions of individual forecast models. The proposed multiplicative ARIMA-ARNN model glorifies the chances of capturing the different combinations of linear and nonlinear patterns in time series. The model shows outstanding performance on six standard time-series data sets compared to other widely used single and hybrid forecasting models.

Using smartphone especially android platform has already got eighty percent market shares, due to aforementioned report, it becomes attacker’s primary goal. There is a growing number of private data onto smart phones and low safety defense measure, attackers can use multiple way to launch and to attack user’s smartphones.(e.g. Using different coding style to confuse the software of detecting malware). Existing android malware detection methods use multiple features, like safety sensor API, system call, control flow structure and data information flow, then using machine learning to check whether its malware or not. These feature provide app’s unique property and limitation, that is to say, from some perspectives it might suit for some specific attack, but wouldn’t suit for others. Nowadays most malware detection methods use only one aforementioned feature, and these methods mostly analysis to detect code, but facing the influence of malware’s code confusion and zero-day attack, aforementioned feature extraction method may cause wrong judge. So, it’s necessary to design an effective technique analysis to prevent malware. In this paper, we use the importance of word from apk, because of code confusion, some malware attackers only rename variables, if using general static analysis wouldn’t judge correctly, then use these importance value to go through our proposed method to generate picture, finally using convolutional neural network to see whether the apk file is malware or not.

COVID-19 was a terrible worldwide pandemic that caused a global public health crisis, with numerous deaths and a severe economic depression. To suppress the spread of COVID-19 and initiate patient isolation and contact tracing to reduce its effect, early diagnosis was required, with real-time reverse transcriptase polymerase chain reaction (RT-PCR) swab test being the most widely used. However, the RT-PCR test was sometimes found to be lengthy and inaccurate and; for that reason, in many cases of severe complications of COVID-19 pneumonia, chest screening with radiography imaging was preferred. In this paper, we carried out a study for COVID-19 detection from computed tomography images, and compared the obtained results using deep learning (DL) and support vector machine (SVM). The findings showed that, despite the excellent results shown by deep learning, the accuracy in predicting COVID-19 using SVM, in the presence of small databases, was slightly higher than DL. The execution time using SVM was also shorter

Deep learning is emerging in many industrial sectors in hand with big data analytics to streamline production. In the biomanufacturing sector, big data infrastructure is lagging comparatively to other industries. A promising approach is to combine Deep Neural Networks (DNN) with prior knowledge in Hybrid Neural Network (HNN) workflows that are less dependent on the quality and quantity of data. This paper reviews published articles over the past 30 years on the topic of HNN applications to bioprocesses. It revealed that HNNs were applied to various bioprocesses, including microbial cultures, animal cells cultures, mixed microbial cultures, and enzyme biocatalysis. HNNs were mainly applied for process analysis, process monitoring, development of software sensors, open- and closed-loop control, batch-to-batch control, model predictive control, intensified design of experiments, quality-by-design, and recently for the development of digital twins. Most previous HNN studies combined shallow Feedforward Neural Networks (FFNNs) with physical laws, such as macroscopic material balance equations, following the semiparametric design principle. Only recently, deep HNNs based on deep FFNNs, Convolution Neural Networks (CNN), Long Short-Term Memory (LSTM) networks and Physics Informed Neural Networks (PINNs) have been reported. The biopharma sector is currently a major driver but applications to biologics quality attributes, new modalities, and downstream processing are significant research gaps.

Intracortical microelectrode arrays (MEAs) can be used in a range of applications, from basic neuroscience research to providing an intimate interface with the brain as part of a Brain-Computer Interface (BCI) system aimed to restore function for people living with neurological disorders or injuries. Unfortunately, MEAs tend to fail prematurely, leading to a loss in functionality for many applications. An important contributing factor in MEA failure is oxidative stress resulting from chronically inflammatory-activated microglia and macrophages releasing reactive oxygen species (ROS) around the implant site. Antioxidants offer a means for mitigating oxidative stress and improving tissue health and MEA performance. Here, we investigate using the clinically available antioxidant dimethyl fumarate (DMF) to reduce the neuroinflammatory response and improve MEA performance in a rat MEA model. Daily treatment of DMF for 16 weeks resulted in a significant improvement in the recording capabilities of MEA devices during the sub-chronic (Weeks 5-11) phase (42% active electrode yield vs. 35% for control). However, these sub-chronic improvements were lost in the chronic implantation phase, as a more exacerbated neuroinflammatory response occurs in DMF-treated animals by 16 weeks post-implantation. Yet, neuroinflammation was indiscriminate between treatment and control groups during the sub-chronic phase. Although worse for chronic use, a temporary improvement (<12 weeks) in MEA performance is meaningful. Providing short-term improvement to MEA devices using DMF can allow for improved use for limited-duration studies. Further efforts should be taken to explore the mechanism behind a worsened neuroinflammatory response at the 16-week time point for DMF-treated animals to assess its usefulness to specific applications.

Abstract: It has been shown that three-dimensional self-assembled multicellular structures derived from human pluripotent stem cells show electrical activity similar to EEG. More recently, neurons were successfully embedded in a digital game-worlds (Kagan, et al. 2022). The biologically inspired neural network (BNN), expressing human cortical cells, was able to show internal modification and learn the task at hand (predicting the trajectory of a digital ball while moving a digital paddle). In other words, the system allowed to read motor information and write sensory data into cell cultures. In this article, we discuss Neural Correlates of Consciousness (NCC) theories, and their capacity to predict or even allow for consciousness in a BNN. We found that In-formation Integration Theory (IIT) is the only NCC that offers the possibility for a BNN to show consciousness. IIT argues that any system capable of integrating information will have some degree of phenomenal consciousness. The pattern of activity appearing in the BNN, with increased density of sensory information leading to better performance, implies that the BNN could be conscious. This may have profound implications from a physiological, philosophical, and ethical perspective.

This article presents a forecast model that uses a hybrid architecture of recurrent neural networks (RNN) with surface neural networks (ANN), based on historical records of exported active energy (EAE) and weather data. Two types of models were developed: the first type includes six models that use EAE records and weather variables as inputs, while the second type includes eight models that use only weather variables. Different metrics were applied to assess the performance of these models, and the best model of each type was selected. Finally, a comparison of the performance between the selected models of both types is presented, and they are validated with real data provided by a solar plant, achieving acceptable levels of accuracy. The selected model of the first type has an RMSE of 0.19, MSE of 0.03, MAE of 0.09, a correlation coefficient of 0.96, and a determination coefficient of 0.93. The other selected model of the second type showed lower precision in the metrics (RMSE = 0.24, MSE = 0.06, MAE = 0.10, Corr. Coef. = 0.95, and Det. Coef. = 0.90). Both models demonstrated good performance and acceptable accuracy in forecasting the weekly photovoltaic energy production of the solar plant.

Sense element engagement theory explains how neural networks produce cortical prosthetic vision. A major prediction of the theory can be tested by developing a device which is expected to enable perception of continuous forms in altered visual geometries. The research reported here completes several essential steps in developing this device: (1) replication of simulations that are consistent with the theory using the NEST simulator, which can also be used for full-scale network emulation by a neuromorphic computer; (2) testing whether results consistent with the theory survive increasing the scale and duration of simulations; (3) establishing a method that uses numbers of spikes produced by network neurons to report the number of phosphenes produced by cortical stimulation; and (4) simulating essential functions of the prosthetic device. NEST simulations replicated early results and increasing their scale and duration produced results consistent with the theory. A decision function created using multinomial logistic regression correctly classified the expected number of phosphenes for 2080 spike number distributions for each of three sets of data, half of which arise from simulations expected to yield continuous visual forms on an altered visual geometry. A process for modulating electrical stimulation amplitude based on intermittent population recordings that is predicted to produce continuous visual forms was successfully simulated. The classification function developed using logistic regression will be used to tune this process as the scale of simulations is further increased.

Deep learning models are part of the family of artificial neural networks and, as such, it suffers of catastrophic interference when they learn sequentially. In addition, most of these models have a rigid architecture which prevents the incremental learning of new classes. To overcome these drawbacks, in this article we propose the Self-Improving Generative Artificial Neural Network (SIGANN), a type of end-to-end Deep Neural Network system which is able to ease the catastrophic forgetting problem when leaning new classes. In this method, we introduce a novelty detection model to automatically detect samples of new classes, moreover an adversarial auto-encoder is used to produce samples of previous classes. This system consists of three main modules: a classifier module implemented using a Deep Convolutional Neural Network, a generator module based on an adversarial autoencoder; and a novelty detection module, implemented using an OpenMax activation function. Using the EMNIST data set, the model was trained incrementally, starting with a small set of classes. The results of the simulation show that SIGANN is able to retain previous knowledge with a gradual forgetfulness for each learning sequence. Moreover, SIGANN can detect new classes that are hidden in the data and, therefore, proceed with incremental class learning.

The emerging use of visualization techniques in pathology and microbiol- ogy has been accelerated by machine learning (ML) approaches towards image preprocessing, classification, and feature extraction in an increasingly complex series of datasets. Modern Convolutional Neural Network (CNN) architectures have developed into an umbrella of vast image reinforcement and recognition methods, including a combined classification-localization of single/multi-object featured images. As a subtype neural network, CNN cre- ates a rapid order of complexity by initially detecting borderlines, edges, and colours in images for dataset construction, eventually capable in mapping intricate objects and conformities. This paper investigates the disparities between Tensorflow object detection APIs, exclusively, Single Shot Detector (SSD) Mobilenet V1 and the Faster RCNN Inception V2 model, to sample computational drawbacks in accuracy-precision vs. real time visualization capabilities. The situation of rapid ML medical image analysis is theoretically framed in regions with limited access to pathology and disease prevention departments (e.g. 3rd world and impoverished countries). Dark field mi- croscopy datasets of an initial 62 XML-JPG annotated training files were processed under Malaria and Syphilis classes. Model trainings were halted as soon as loss values were regularized and converged.

In the current industrial landscape, increasingly pervaded by technological innovations, the adoption of optimized strategies for asset management is becoming a critical key success factor. Among the various strategies available, the “Prognostics and Health Management” strategy is able to support maintenance management decisions more accurately, through continuous monitoring of equipment health and “Remaining Useful Life” forecasting. In the present study, Convolutional Neural Network-based Deep Neural Network techniques are investigated for the Remaining Useful Life prediction of a punch tool, whose degradation is caused by working surface deformations during the machining process. Surface deformation is determined using a 3D scanning sensor capable of returning point clouds with micrometric accuracy during the operation of the punching machine, avoiding both downtime and human intervention. The 3D point clouds thus obtained are transformed into bidimensional image-type maps, i.e., maps of depths and normal vectors, to fully exploit the potential of convolutional neural networks for extracting features. Such maps are then processed by comparing 15 genetically optimized architectures with the transfer learning of 19 pre-trained models, using a classic machine learning approach, i.e., Support Vector Regression, as a benchmark. The achieved results clearly show that, in this specific case, optimized architectures provide performance far superior (MAPE=0.058) to that of transfer learning which, instead, remains at a lower or slightly higher level (MAPE=0.416) than Support Vector Regression (MAPE=0.857).

In the transition from traditional electrical energy generation with mainly linear sources to increasing inverter-based distributed generation, electrical power systems’ power quality requires new monitoring methods. Integrating a high penetration of distributed generation, which is typically located in medium- or low-voltage grids, shifts the monitoring tasks from the transmission to distribution layers. Compared to high-voltage grids, distribution grids feature a higher level of complexity. Monitoring all relevant buses is operationally infeasible and costly. State estimation methods provide knowledge about unmeasured locations by learning a physical system’s non-linear relationships. This article examines a new flexible, close-to-real-time concept of harmonic state estimation using synchronized measurements processed in a neural network. A physics-aware approach enhances a data-driven model, taking into account the structure of the electrical network. An OpenDSS simulation generates data for model training and validation. Different load profiles for both training and testing were utilized to increase the variance in the data. The results of the presented concept demonstrate high accuracy compared to other methods for harmonic orders 1 to 20.

Building deep learning models proposed by third parties can become a simple task when specialized libraries are used. However, a great deal of mystery still surrounds the design of new models or the modification of existing ones. These tasks require in-depth knowledge of the different components or building blocks as well as their dimensions. This information is limited and broken up in different literature. In this article, we collect and explain in-depth the building blocks to design deep learning models, starting from the artificial neuron to the concepts involved in building deep neural networks. Furthermore, the implementation of each building block is exemplified using the Keras library.

Drug-target affinity (DTA) prediction is crucial for understanding molecular interactions and aiding drug discovery and development. While various computational methods have been proposed for DTA prediction, their predictive accuracy remains limited, failing to delve into the structural nuances of interactions. With increasingly accurate and accessible structure prediction of targets, we developed a novel deep learning model, named S2DTA, to accurately predict DTA by fusing sequence and structural knowledge of drugs, targets, and pockets using heterogeneous models based on graph and semantic networks. Experimental findings underscored that complex feature representations imparted negligible enhancements to the model’s performance. However, the integration of heterogeneous models demonstrably bolstered predictive accuracy. In comparison to three state-of-the-art methodologies, the supremacy of S2DTA became strikingly apparent. It showcased a noteworthy 25.2% reduction in Mean Absolute Error (MAE) and an impressive 20.1% decrease in Root Mean Square Error (RMSE). Furthermore, S2DTA exhibited substantial advancements in other pivotal metrics, including Pearson Correlation Coefficient (PCC), Spearman, Concordance Index (CI), and R2. These metrics experienced remarkable increments of at least 19.6%, 17.5%, 8.1%, and a remarkable 49.4%, respectively. Finally, we conducted interpretability analysis on the effectiveness of S2DTA by bidirectional self-attention mechanism, fully proving that S2DTA is a valuable and accurate tool for predicting DTA. For further exploration, the source data and code repository can be accessed at https://github.com/dldxzx/S2DTA.

In this paper, we continue the research cycle on the properties of convolutional neural network-based image recognition systems and ways to improve noise immunity and robustness [1]. Currently, a popular research area related to artificial neural networks is adversarial attacks. The effect of adversarial attacks on the image is not highly perceptible to the human eye, also it drastically reduces the neural network accuracy. Image perception by a machine is highly dependent on the propagation of high frequency distortions throughout the network. At the same time, a human efficiently ignores high-frequency distortions, perceiving the shape of objects as a whole. The approach proposed in this paper can improve the image recognition accuracy in the presence of high-frequency distortions, in particular, caused by adversarial attacks. The proposed technique makes it possible to measure up the logic of artificial neural network to that of a human, for whom high-frequency distortions are not decisive in object recognition.

The Physics-Informed Neural Networks (PINNs) improve the efficiency of data utilization by combining physical principles with neural network algorithms and ensure that the predictions are consistent and stable with the physical laws. PINNs opens up a new approach to address inverse problems in fluid mechanics. Based on the single-relaxation-time lattice Boltzmann method (SRT-LBM) with the Bhatnagar-Gross-Krook (BGK) collision operator, the PINN-SRT-LBM model is proposed in this paper for solving the inverse problem in fluid mechanics. The PINN-SRT-LBM model consists of three components. The first component involves a deep neural network that predicts the equilibrium control equations in different discrete velocity directions within SRT-LBM. The second component employs another deep neural network to predict non-equilibrium control equations, enabling inference of the fluid's non-equilibrium characteristics. The third component, a physics informed function translates the outputs of the first two networks into physical infor-mation. By minimizing the residuals of the physical partial differential equations (PDEs), the physics informed function infers relevant macroscopic quantities of the flow. The model evolves two sub-models applicable to different dimensions, named PINN-SRT-LBM-I and PINN-SRT-LBM-II models according to the construction of the physical informed function. The innovation of this work is the introduction of SRT-LBM and discrete velocity models as physical drivers into the neural network through the interpretation function. Therefore, PINN-SRT-LBM allows the neural network to handle inverse problems of various dimensions and focus on problem-specific solving. Experimental results confirm the accurate prediction of flow infor-mation at different Reynolds numbers within the computational domain. Relying on the PINN-SRT-LBM models, inverse problems in fluid mechanics can be solved efficiently.

Effectively avoiding gas accident is vital to the security of mineral manufacture, and the coal mine gas accident is often caused by gas concentration overrun. The prediction accuracy of gas emission quantity in coal mine is the key to solve this problem. To maintain concentration of gas in a secure range，grey theory and neural network model increasingly diffusely used in forecasting gas emission quantity in coal mine critically. Nevertheless, the limitation of the grey neural network model is that researchers merely bonded the conventional neural network and grey theory. To enhance accuracy of prediction, a modified grey GM(1,1) and RBF neural network model is proposed combined amended grey GM(1,1) model and RBF neural network model. Then the proposed model was put into simulation experiment which is built based on Matlab software. Ultimately, conclusion of the simulation experiment verified that the modified grey GM(1,1) and RBF neural network model not only boosts the precision of prediction, but also restricts relative error in a minimum range. This showed that the modified grey GM(1,1) and RBF neural network model achieves effectiveness in precision of prediction much better than grey GM(1,1) model and RBF neural network model.

Graph neural networks and graph convolutional networks are effective at representing complex relationships. We attempt to apply them to sports analytics by reviewing existing applications in sports. These methodologies leverage the inherent graph structure of sports data, capturing interactions among players, teams, and game events. By systematically examining various studies, we highlight how GNNs and GCNs can enhance performance prediction, player evaluation, and tactical analysis in sports. Our review emphasizes the unique advantages these models offer over traditional analytical approaches, particularly in their ability to account for the dynamic and interconnected nature of sports activities. We conclude by identifying potential areas for future research and application, suggesting that the integration of advanced graph-based techniques could significantly advance the field of sports analytics.

As sequencing technologies advance, short DNA sequence fragments increasingly serve as DNA barcodes for species identification. Rapid acquisition of DNA sequences from diverse organisms is now possible, highlighting the increasing significance of DNA sequence analysis tools in species identification. This study introduces a new approach for species classification with DNA barcodes based on an ensemble of deep neural networks (DNN). Several techniques are proposed and empirically evaluated for converting raw DNA sequence data into images fed into the DNNs. The best-performing approach is obtained by representing each pair of DNA bases with the value of a related physicochemical property. By utilizing different physicochemical properties, we can create an ensemble of networks. Our proposed ensemble obtains state-of-the-art performance on both simulated and real data sets. The code of the proposed approach is available at https://github.com/LorisNanni/AI-powered-Biodiversity-Assessment-Species-Classification-via-DNA-Barcoding-and-Deep-Learning .

Surface defect detection based on machine vision and convolutional neural networks (CNNs) is an important and necessary process that enables rubber ring manufacturers to improve production quality and efficiency. However, such automatic detection always consumes substantial computer resources to guarantee detection accuracy. To solve this problem, in this paper, we present a CNNs optimization algorithm based on the Ghost module. First, we replace the convolutional layer with the Ghost module in CNNs so that feature maps can be generated using cheaper linear operations. Second, we use an optimization method to obtain the best replacement of the Ghost module to achieve a balance between computer resource consumption and detection accuracy. Finally, we use an image preprocessing method that includes inverting colors. We integrated this algorithm into YOLOv5, which we trained on a dataset with 122 images of rubber ring surface defects. Compared with the original network, the network size decreased by 30.5% and the computational cost decreased by 23.1% whereas average precision only decreased by 1.8%. Additionally, the network's training time decreased by 16.1% as a result of preprocessing. These results show that the proposed approach greatly helped practical rubber ring surface defect detection.

Gaussian mixture models (GMMs) are powerful tools specifically suited for problems where data distributions are multi-modal. Inspired by Fourier expansion, we propose a concept called GMM expansion to approximate arbitrary densities. A simple learning method is introduced under this framework. Theoretical and numerical analyses are provided to show that densities under GMM expansion are able to approximate arbitrary densities with certain accuracy. The proposed learning algorithm demonstrates better accuracy and time efficiency compared to classic density approximation methods such as the Expectation Maximization (EM) algorithm and Bayesian variational inference (BVI) for Gaussian mixture models. The proposed framework also shows better accommodation of neural networks. Three neural network applications are built to demonstrate its usability. The accuracy of density estimation in neural networks is reported to be significantly improved in one of the applications. Another application shows that latent variables can be easily turned into random variables. This user-friendly method opens up more possibilities in terms of latent manipulation, embedding techniques, and many other potential uses.

Neural Fractional Differential Equations (Neural FDE) represent a neural network architecture specifically designed to fit the solution of a fractional differential equation to given data. This architecture combines an analytical component, represented by a fractional derivative, with a neural network component, forming an initial value problem. During the learning process, both the order of the derivative and the parameters of the neural network must be optimised. In this work, we investigate the non-uniqueness of the optimal order of the derivative and its interaction with the neural network component. Based on our findings, we perform a numerical analysis to examine how different initialisations and values of the order of the derivative (in the optimisation process) impact its final optimal value. Results show that the neural network on the right-hand side of the Neural FDE struggles to adjust its parameters to fit the FDE to the data dynamics for any given order of the fractional derivative. Consequently, Neural FDEs do not require a unique α value, instead, they can use a wide range of α values to fit data. This flexibility is beneficial when fitting to given data is required, and the underlying physics is not known.

This study presents the Distilled MixUp Squeeze ResNet (DMSResNet), an optimized variant of the Residual Network architecture designed for high accuracy and computational efficiency. By integrating Squeeze-and-Excitation blocks, MixUp data augmentation, and knowledge distillation, it achieves competitive performance within strict parameter constraints and limited training resources. Evaluated on the CIFAR-10 dataset, the model attains a 96.56% accuracy with only 4.3 million total parameters. The results demonstrate that optimization of established architectures can yield performance comparable to newer models, especially in resource-limited scenarios. This work contributes to the development of efficient deep learning models for applications in constrained computational environments.

Link prediction is a fundamental problem in graphs. Currently, SubGraph Representation Learning (SGRL) methods provide state-of-the-art solutions for link prediction by transforming the task into a graph classification problem. However, existing SGRL solutions suffer from high computational costs and lack scalability. In this paper, we propose a novel SGRL framework called Augmented Feature Diffusion on Sparsely Sampled Subgraph (AFD3S). The AFD3S first uses a conditional variational autoencoder to augment the local features of the input graph, effectively improving the expressive ability of downstream Graph Neural Networks. Then, based on a random walk strategy, sparsely sampled subgraphs are obtained from the target node pairs, reducing computational and storage overhead. Graph diffusion is then performed on the sampled subgraph to achieve specific weighting. Finally, the diffusion matrix of the subgraph and its augmented feature matrix are used for feature diffusion to obtain operator-level node representations as inputs for the SGRL-based link prediction. Feature diffusion effectively simulates the message-passing process, simplifying subgraph representation learning, thus accelerating the training and inference speed of subgraph learning. Our proposed AFD3S achieves optimal prediction performance on several benchmark datasets, with significantly reduced storage and computational costs.

Autism spectrum disorder (ASD) is characterized by diverse patterns of social interaction, communication, and behavior. This study explores the application of clustering and stochastic migration models to visualize and understand the social and neural connectivity patterns associated with autism. The clustering model, with its strong internal connections and limited external connections, mirrors the tight-knit social groups and communication barriers often observed in individuals with autism. In contrast, the stochastic model, characterized by more dispersed connections and greater adaptability, represents the variability and flexibility seen in social interactions across typical individuals. By comparing these models, we can gain deeper insights into the unique challenges and strengths of individuals with autism, highlighting the importance of personalized interventions. Additionally, these models provide a visual framework for understanding the differences in brain connectivity patterns observed in autism, with implications for both behavioral and neural correlates. This study underscores the value of computational models in advancing our understanding of autism and guiding effective support strategies.

Multiple Sclerosis (MS) is a chronic autoimmune disorder characterized by the demyelination of neural fibers in the central nervous system. Understanding the impact of demyelination on neural network efficiency is crucial for elucidating the pathophysiological mechanisms underlying MS. This study aims to model the effects of demyelination on neural network efficiency, specifically focusing on how increased resistance and path lengths due to demyelination correlate with the clinical symptoms of MS. We employed a graph-theoretical approach to simulate neural networks, introducing varying degrees of demyelination. The demyelination was represented by increasing the resistance (weights) of selected edges within the network, affecting both random and clustered distributions of network connections. The primary metric analyzed was the average shortest path length, indicative of network efficiency. The models demonstrated a significant increase in the average shortest path length in demyelinated networks compared to intact ones. Specifically, networks with clustered demyelination exhibited localized inefficiencies, correlating with the focal nature of MS lesions. The increase in path length was associated with reduced signal transmission efficiency, providing insights into a range of MS symptoms including motor and sensory impairments, cognitive dysfunction, visual disturbances, and fatigue. The graph-theoretical model of neural network demyelination offers a novel perspective on how structural changes in the brain's connectivity due to MS can lead to diverse clinical manifestations. While the model simplifies the complexity of neural networks, it underscores the potential of computational approaches in understanding and predicting the course of neurological disorders like MS.

We propose a novel moving median estimator specifically designed for online detection of threshold crossings in multi-channel signals, such as extracellular neural recordings. This estimator offers two key advantages: a reduced sensitivity to outliers and the elimination of memory requirements for storing arrival times. Furthermore, its design facilitates parallel implementation on FPGAs, making it ideal for real-time processing of multi-channel recordings.

In this paper, a special recurrent neural network (RNN) called \emph{Long Short-Term Memory (LSTM)} is used to design a virtual load sensor that estimates the mass of heavy vehicles. The estimation algorithm consists of a two-layer LSTM network, with the two layers based on sequence-to-sequence and sequence-to-one logic, respectively. The network estimates the vehicle mass based on the vehicle speed, longitudinal acceleration, engine speed, engine torque, and the accelerator pedal position. The network is trained and tested with a data set collected in a high-fidelity simulation environment called Truckmaker. Training data is generated in acceleration maneuvers in a range of speeds whereas test data is obtained by simulating the vehicle in a Worldwide harmonized light vehicles test cycle (WLTC). Preliminary results show that, with the proposed approach, the heavy-vehicle mass can be estimated as accurately as commercial load sensors in a range of load mass as wide as four tons.

Artificial neural networks are widely established models of computational intelligence that have been tested for effectiveness in a variety of real-world applications. These models require fitting a set of parameters through the use of some optimization technique. However, an issue that researchers often face is finding an efficient range of values for the parameters of the artificial neural network. This paper proposes an innovative technique of generating a promising range of values for the parameters of the artificial neural network. Finding the value field is done by a series of rules for partitioning the original set of values or expanding it, which rules are generated using Grammatical Evolution. After finding a promising interval of values, any optimization technique such as a genetic algorithm can be used to train the artificial neural network on that interval of values. The new technique was tested on a wide range of problems from the relevant literature and the results were extremely promising.

The Necessity of Human resources beyond perception of human understanding towards the Evaluation of video quality or Data screening methodology is conducted based on human perception level since it is concerned with how visual content is perceived by a observer based on observations with his/ her perception on a particular video sequence. Therefore, we considered that the subject has to grade the encoded video sequences under certain test environment conditions based on ITU-Recommendations. Since Human perception is considered as the true judgment and precise measurement of visual content, data screening has became quite essential and quite comfortable to general public due to introduction of User Experience(UX) concept by User Experience community. The translations of A recurrent neural network is based on certain principles, for instance we considered natural language processing which is certainly adaptable towards understanding sequential data and use patterns to predict the consistency within observers. In our research, we adapted principles based on Recurrent Neural Networks while assuming consistency within observers for predicting video quality within data screening environment towards subjective experiments. Moreover,this research work explores the trade offs between Human perception on visual content and consistency of observations within individual observer.

This article explores the transformative role of neural networks in the realm of biology, particularly within microscopy image processing and illustrations. Neural networks have revolutionized cell segmentation and analysis, enabling precise delineation of cell boundaries and tracking of cellular behaviours. They excel in detecting subcellular structures, unravelling intricate organelle interactions. In 3D image reconstruction, neural networks navigate volumetric datasets, enhancing our understanding of spatial cellular architecture. These networks enhance disease diagnosis by identifying anomalies and irregularities, potentially revolutionizing early detection and classification. Moreover, neural networks restore and enhance microscopy images, unveiling hidden details. Lastly, they bridge art and science, fostering captivating biological art and enriching science communication. As neural networks evolve, they promise a future of limitless possibilities, weaving together technology, science, and art to illuminate the microscopic realm in unprecedented ways.

Oleuropein (OLE), a main constituent of olive, displays a pleiotropic beneficial dynamic in health and disease, based mainly on its antioxidant and hypolipidemic properties, and its capacity to protect myocardium during ischemia. Furthermore, OLE activates peroxisome proliferator-activated receptor (PPARα) in neurons and astrocytes, providing neuroprotection against noxious biological reactions that are induced following cerebral ischemia. Current study investigated the effect of OLE in the regulation of various neural plasticity indices, emphasizing the role of PPARα. For this purpose, 129/Sv wild-type (WT) and Pparα-null mice were treated with OLE for three weeks. The present findings revealed that chronic treatment with OLE up-regulated the brain-derived neurotrophic factor (BDNF) and its receptor TrkB in the prefrontal cortex (PFC) of mice via activation of ERK1/2, AKT and PKA/CREB signaling pathways. No similar effects were observed in the hippocampus. The OLE-induced effects on BDNF and TrkB appear to be mediated by PPARα, because no similar alterations were observed in the PFC of Pparα-null mice. Notably, OLE did not affect the neurotrophic factors NT3 and NT4/5 in both brain tissues. Fenofibrate though, a selective PPARα agonist, up-regulated BDNF and NT3 in the PFC of mice, whereas the drug induced NT4/5 in both brain sites tested. Interestingly, OLE provided neuroprotection in differentiated human SH-SY5Y cells against β-amyloid and H2O2 toxicity independently from PPARα activation. In conclusion, OLE and similar drugs acting either as PPARα agonists or via PPARα independent mechanisms could improve synaptic function/plasticity mainly in the PFC and to a lesser extent in the hippocampus, thus beneficially affecting cognitive functions.

Soft tactile sensors based on piezoresistive materials have various large-area sensing applications. However, their accuracy is often affected by hysteresis, a phenomenon that poses a significant challenge during operation. This paper introduces a novel approach that employs a backpropagation (BP) neural network to address the hysteresis nonlinearity in conductive fibre-based tactile sensors. To assess the effectiveness of the proposed method, four sensor units with different layer configurations (1, 3, 6, and 12) were designed. These sensor units underwent force sequences to collect corresponding output resistances. A backpropagation network was trained using these force sequences, thereby correcting the resistance values. The training process exhibited excellent convergence, effectively adjusting the network's parameters to minimize the error between predicted and actual resistance values. As a result, the trained BP network accurately predicted the output resistances. Several validation experiments were conducted to highlight the primary contribution of this research. The proposed method reduced the maximum hysteresis error from 24.2% of the sensor's full-scale output to 13.5%. This improvement establishes the approach as a promising solution for enhancing the accuracy of soft tactile sensors based on piezoresistive materials. By effectively mitigating hysteresis nonlinearity, the capabilities of soft tactile sensors in various applications can be enhanced. These sensors become more reliable and more efficient tools for force measurement and control, particularly in the fields of soft robotics and wearable technology. Consequently, their widespread applications extend to robotics, medical devices, consumer electronics, and gaming. Notably, the complete elimination of hysteresis in tactile sensors may not be feasible. Nevertheless, the proposed method effectively modifies the hysteresis nonlinearity, leading to improved sensor output accuracy.

One of the significant challenges today in the brain-machine interface using invasive methods is the stability of the chronic record. In recent years, polymer-based electrodes have gained notoriety for achieving mechanical strength values close to that of brain tissue, promoting a lower immune response to the implant. In this work, we fabricated fully polymeric electrodes based on PEDOT:PSS for neural recording in Wistar rats. We characterized the electrical properties and both in-vitro and in-vivo functionality of the electrodes. Also, we employed histological processing and microscopical visualization to evaluate tecidual immune response in 7, 14, and 21 days post-implant days. Electrodes with 400-micrometer channels showed a 12dB signal-to-noise ratio. Local field potentials were characterized under two conditions: anesthetized and free-moving. There was a proliferation of microglia to the tissue-electrode interface in the first days, with a decrease after 14 days. Astrocytes also migrated to the interface, but there was no continuous recruitment of these cells in the tissue, showing inflammatory stability at 21 days. The signal was not affected by this inflammatory action, demonstrating that fully polymeric electrodes can be an alternative to prolong the valuable time of neural recordings.

There is a considerable amount of research in online social networks, most of which focuses on the structural analysis of social graphs. The interpersonal relationships of social networks, especially friend circle, can solve the cold start and sparsity problems, and through the relationship between social networks can effectively recommend users' favorite items (items), such as music , videos, brands/products, preferred tags, location, services, etc. User relationships in social networks are diverse and there are many different perspectives on different social networks. Associations among users can form multi-layered composite networks, and multi-layered social networks present new challenges and opportunities. Different relationships can influence users' preferences to different degrees, which in turn affects their behavior. Therefore, fusing multiple social networks is an effective way to improve recommendation. Although some studies have started to address multiple social network recommendations, simple linear superposition cannot reflect the coupling and nonlinear association between multiple social networks. In this paper, we propose a graph neural network recommendation model under social relationships based on this background. We first propose to compute the 2nd order collaborative signals and their intensities directly from the neighboring matrix for updating the node embedding of the graph convolution layer. Secondly by embedding historical evaluations, various social networks constituting different dimensions, the attention integration of user preferences by different social networks is achieved, and its effectiveness and scalability are demonstrated in theoretical derivation and experimental validation. The theoretical derivation and experimental validation demonstrate its effectiveness and scalability.

Three-phase motors are commonly adopted in several industrial contexts and their failures can result in costly downtime causing undesired service outages; this way, motor diagnostics is an issue that assumes great importance. To prevent their failures and timely face the considered service outages, a non-invasive method to identify electrical and mechanical faults in three-phase asynchronous electric motors is proposed in the paper. In particular, a measurement strategy along with a machine learning algorithm based on Artificial Neural Network is exploited to properly classify failures. In particular, digitized current samples of each motor phase are first processed by means of FFT and PSD in order to estimate the associated spectrum. Suitable features (in terms of frequency and amplitude of the spectral components) are then singled out to either train or feed a neural network acting as a classifier. The method is preliminary validated on a set of 28 electric motors, and its performance is compared with common state-of-art machine learning techniques. The obtained results show that the proposed methodology is able to reach accuracy levels greater than 98\% in identifying anomalous conditions of three-phase asynchronous motors.

This work describes and compares the backpropagation algorithm with the Extended Kalman filter, a second-order training method which can be applied to the problem of learning neural network parameters and is known to converge in only a few iterations. The algorithms are compared with respect to their effectiveness and speed of convergence using simulated data for both, a regression and a classification task.

Nowadays smartphone utilization for disease diagnosis and remote health care applications has become promising due to their ubiquity. Here, a novel convolutional neural network method for detecting keratoconus that is wholly implemented on a smartphone is proposed. The proposed method provides accurate detection of over 72.9% for all stages of keratoconus. Preliminary results indicate 90%, 83%, 64% and 52% detection rate for severe, advanced, moderate and mild stages of disease, respectively.

It is important to reduce the computation complexity while maintaining the accuracy of convolution neural networks. We deem it is possible to further reduce the network complexity while ensuring the accuracy. In this paper, we propose a novel feature extraction network called DSRNet which is lightweight but effective. DSRNet follows the basic ideas of stacking modules and short connection, introduces Depthwise Separable convolution and utilizes the Dilated convolution. The proposed network has fewer parameters and achieves outstanding speed. We conducted comprehensive experiments on CIFAR10, CIFAR100 and STL10 datasets, and the results showed the DSRNet has great performance improvement in terms of accuracy and speed.

The Station logo is a way for a TV station to claim copyright, which can realize the analysis and understanding of the video by the identification of the station logo, so as to ensure that the broadcasted TV signal will not be illegally interfered. In this paper, we design a station logo detection method based on Convolutional Neural Network by the characteristics of the station, such as small scale-to-height ratio change and relatively fixed position. Firstly, in order to realize the preprocessing and feature extraction of the station data, the video samples are collected, filtered, framed, labeled and processed. Then, the training sample data and the test sample data are divided proportionally to train the station detection model. Finally, the sample is tested to evaluate the effect of the training model in practice. The simulation experiments prove its validity.

Most systems that are the subject of control engineering studies have some non-linearity. An example of this is the horizontal cylindrical tank, commonly used in process industries. To deal with cases like this, several control theories have been developed over time, each one presenting better results in certain systems. This work presents an alternative for the control of nonlinear systems, without necessary modeling or previous information about the system, based on a new optimization law for the artificial neural network training in real time.

A theory is presented about the information available for guiding purposeful actions by any organism, whether animal or plant, and about how the information is used in guiding actions.

Network Embedding (NE) is an important method to learn the representations of network via a low-dimensional space. Conventional NE models focus on capturing the structure information and semantic information of vertices while neglecting such information for edges. In this work, we propose a novel NE model named BimoNet to capture both the structure and semantic information of edges. BimoNet is composed of two parts, i.e., the bi-mode embedding part and the deep neural network part. For bi-mode embedding part, the first mode named add-mode is used to express the entity-shared features of edges and the second mode named subtract-mode is employed to represent the entity-specific features of edges. These features actually reflect the semantic information. For deep neural network part, we firstly regard the edges in a network as nodes, and the vertices as links, which will not change the overall structure of the whole network. Then we take the nodes' adjacent matrix as the input of the deep neural network as it can obtain similar representations for nodes with similar structure. Afterwards, by jointly optimizing the objective function of these two parts, BimoNet could preserve both the semantic and structure information of edges. In experiments, we evaluate BimoNet on three real-world datasets and task of relation extraction, and BimoNet is demonstrated to outperform state-of-the-art baseline models consistently and significantly.

We study latching dynamics in the adaptive Potts model network, through numerical simulations with randomly and also weakly correlated patterns, and we focus on comparing its slowly and fast adapting regimes. A measure, Q, is used to quantify the quality of latching in the phase space spanned by the number of Potts states S, the number of connections per Potts unit C and the number of stored memory patterns p. We find narrow regions, or bands in phase space, where distinct pattern retrieval and duration of latching combine to yield the highest values of Q. The bands are confined by the storage capacity curve, for large p, and by the onset of finite latching, for low p. Inside the band, in the slowly adapting regime, we observe complex structured dynamics, with transitions at high crossover between correlated memory patterns; while away from the band latching transitions lose complexity in different ways: below, they are clear-cut but last so few steps as to span a transition matrix between states with few asymmetrical entries and limited entropy; while above, they tend to become random, with large entropy and bi-directional transition frequencies, but indistinguishable from noise. Extrapolating from the simulations, the band appears to scale almost quadratically in the p - S plane, and sublinearly in p - C. In the fast adapting regime the band scales similarly, and it can be made even wider and more robust, but transitions between anti-correlated patterns dominate latching dynamics. This suggest that slow and fast adaptation have to be integrated in a scenario for viable latching in a cortical system. The results for the slowly adapting regime, obtained with randomly correlated patterns, remain valid also for the case with correlated patterns, with just a simple shift in phase space.

The vast majority of autonomous driving systems are limited to applications on roads with clear lane markings and are implemented using commercial grade sensing systems coupled with specialized graphic accelerator hardware. This research reviews an alternative approach for autonomously steering vehicles that eliminates the dependency on road markings and specialized hardware. A combination of machine vision, machine learning and artificial intelligence based on popular pre-trained Convolutional Neural Networks (CNN) and Recurrent Neural Networks (RNN) was used to drive a vehicle along roads lacking lane markings (unmarked roads). The team developed and tested this approach on the ACTor (Autonomous Camus TranspORt) vehicle - an autonomous vehicle development and research platform coupled with a low-cost webcam based sensing system and minimal computational resources. The proposed solution was evaluated on real-world roads and varying environmental conditions. It was found that this solution may be used to successfully navigate unmarked roads autonomously with acceptable road following behavior.

The utilization of Pig Emotion Recognition (PER) driven by Artificial Intelligence (AI) promises to mitigate labor costs and alleviate stress among domestic pigs, thereby minimizing the need for consistent human intervention. Nevertheless, this research acknowledges the inherent limitations within the raw PER datasets, which often include irrelevant porcine features, hence impeding genuine progress in real-world evaluations. A significant proportion of PER datasets derive from sequential pig imagery obtained from video recordings, and a common pitfall in these studies is the unregulated shuffling of data. This lack of control can result in the overlap of data samples between training and testing groups, thereby yielding skewed experimental evaluations. To address these challenges, this paper introduces a novel solution in the form of the Semi-Shuffle-Pig Detector (SSPD) for PER datasets, with the intent to facilitate a less biased experimental output. By applying the SSPD, we can ensure that all testing data samples remain distinct from the training datasets, and any superfluous information from raw images is systematically discarded. This optimized method enhances the true performance of classification, providing unbiased experimental evaluations. Notably, our approach has led to a remarkable improvement in the Isolation After Feeding (IAF) metric by 20.2\% and achieved higher accuracy in segregating IAF and Paired After Feeding (PAF) classifications exceeding 92\%. This methodology, thereby, ensures the preservation of pertinent data within the PER system and eliminates potential biases in experimental evaluations. Consequently, it elevates the accuracy and reliability of real-world PER applications, resulting in a tangible positive impact on both pig welfare management and food safety standards.