Self-driving vehicles demand efficient and reliable depth sensing technologies. Lidar, with its capacity for long-distance, high-precision measurement, is a crucial component in this pursuit. However, conventional mechanical scanning implementations suffer from reliability, cost, and frame rate limitations. Solid-state lidar solutions have emerged as a promising alternative, but the vast amount of photon data processed and stored using conventional direct time-of-flight (dToF) prevents long-distance sensing unless power-intensive partial histogram approaches are used. In this paper, we introduce a groundbreaking ‘guided’ dToF approach, harnessing external guidance from other onboard sensors to narrow down the depth search space for a power and data-efficient solution. This approach centers around a dToF sensor in-which the exposed time widow of independent pixels can be dynamically adjusted. We utilize a 64-by-32 macropixel dToF sensor and a pair of vision cameras to provide the guiding depth estimates. Our demonstrator captures a dynamic outdoor scene at 3 fps with distances up 75 m. Compared to a conventional full histogram approach, on-chip data is reduced by over 25 times, while the total laser cycles in each frame are reduced by at least 6 times compared to any partial histogram approach. The capability of guided dToF to mitigate multipath reflections is also demonstrated. For self-driving vehicles where a wealth of sensor data is already available, guided dToF opens new possibilities for efficient solid-state lidar.

Medical image analysis is an important branch in the field of medicine, which mainly uses image processing and analysis techniques to interpret and diagnose medical image data. Medical image data helps doctors to effectively observe and diagnose patients' body structures, tissues and lesions. Medical image analysis has been an important research area in the medical field, and it is important for disease diagnosis, treatment planning, and condition monitoring. In recent years, the rapid development of deep learning and computer vision technologies has contributed greatly to the automation, multimodal data fusion, real-time application, and accuracy improvement of medical image analysis. In addition, the development of deep learning has given rise to some new research areas in medical image analysis, such as Generative Adversarial Networks (GANs) for synthetic medical images, self-supervised learning for unsupervised feature learning, and neural network interpretability. In this paper, we will introduce some optimisation methods for medical images which are effective in improving the accuracy, efficiency and reliability of medical image analysis.

Some traditional robots are based on offline programming reciprocal motion, and with the continuous upgrading of vision technology, more and more tasks are replaced by machine vision. For the current problem of insufficient accuracy of robot target localization based on binocular vision, and an improved random sampling consistency algorithm is proposed to complete parallel robot target localization and grasping under the guidance For the current problem of insufficient accuracy of robot target localization based on binocular vision, an improved random sampling consistency algorithm is proposed to complete parallel robot target localization and grasping under the guidance of multi Firstly, the RANSAC algorithm is improved based on the SURF algorithm; then the parallax gradient method is applied to iterate the matched point pairs several times to further optimize the data; then the 3D reconstruction is completed by the program technique; finally the obtained data is finally the obtained data is input into the robot arm and the camera internal and external parameters are obtained by the calibration method so that the robot can accurately locate and The experiments show that the improved algorithm has advantages in recognition accuracy and grasping success rate under multi- The experiments show that the improved algorithm has advantages in recognition accuracy and grasping success rate under multi-vision system.

The use of visual signals in horticulture has attracted significant attention and encompassed a wide range of data types such as 2D images, videos, hyperspectral images, and 3D point clouds. These visual signals have proven to be valuable in developing cutting-edge computer vision systems for various applications in horticulture, enabling plant growth monitoring, pest and disease detection, quality and yield estimation, and automated harvesting. However, unlike other sectors, developing deep learning computer vision systems for horticulture encounters unique challenges due to the limited availability of high-quality training and evaluation datasets necessary for deep learning models. This paper investigates the current status of vision systems and available data in order to identify the high-quality data requirements specific to horticultural applications. We analyse the impact of the quality of visual signals on the information content and features that can be extracted from these signals. To address the identified data quality requirements, we explore the usage of a deep learning-based super-resolution model for generative quality enhancement of visual signals. Furthermore, we discuss how these can be applied to meet the growing requirements around data quality for learning-based vision systems. We also present a detailed analysis of the competitive quality generated by the proposed solution compared to cost-intensive hardware-based alternatives. This work aims to guide the development of efficient computer vision models in horticulture by overcoming existing data challenges and paving a pathway forward for contemporary data acquisition.

Research has been growing on object detection using semi-supervised methods in past few years. We examine the intersection of these two areas for floor-plan objects to promote the research objective of detecting more accurate objects with less labelled data. The floor-plan objects include different furniture items with multiple types of the same class, and this high inter-class similarity impacts the performance of prior methods. In this paper, we present Mask R-CNN based semi-supervised approach that provides pixel-to-pixel alignment to generate individual annotation masks for each class to mine the inter-class similarity. The semi-supervised approach has a student-teacher network that pulls information from the teacher network and feeds it to the student network. The teacher network uses unlabeled data to form pseudo-boxes, and the student network uses both unlabeled data with the pseudo boxes and labelled data as ground truth for training. It learns representations of furniture items by combining labelled and unlabeled data. On the Mask R-CNN detector with ResNet-101 backbone network, the proposed approach achieves mAP of 98.8%, 99.7%, and 99.8% with only 1%, 5% and 10% labelled data, respectively. Our experiment affirms the efficiency of the proposed approach as it outperforms the fully supervised counterpart using only 10% of the labels.

Manual visual inspections typically conducted after an earthquake are high-risk, subjective, and time-consuming. Delays from inspections often exacerbate the social and economic impact of the disaster on affected communities. Rapid and autonomous inspection using images acquired from unmanned aerial vehicles offer the potential to reduce such delays. Indeed, a vast amount of re-search has been conducted toward developing automated vision-based methods to assess the health of infrastructure at the component and structure level. Most proposed methods typically rely on images of the damaged structure, but seldom consider how the images were acquired. To achieve autonomous inspections, methods must be evaluated in a comprehensive end-to-end manner, incorporating both data acquisition and data processing. In this paper, we leverage recent advances in computer generated imagery (CGI) to construct a 3D synthetic environment for simulation of post-earthquake inspections that allows for comprehensive evaluation and valida-tion of autonomous inspection strategies. A critical issue is how to simulate and subsequently render the damage in the structure after an earthquake. To this end, a high-fidelity nonlinear finite element model is incorporated in the synthetic environment to provide a representation of earthquake-induced damage; this finite element model, combined with photo-realistic rendering of the damage, is termed herein a physics-based graphics models (PBGM). The 3D synthetic en-vironment with PBGMs provide a comprehensive end-to-end approach for development and validation of autonomous post-earthquake strategies using UAVs, including: (i) simulation of path planning of virtual UAVs and image capture under different environmental conditions; (ii) au-tomatic labeling of captured images, potentially providing an infinite amount of data for training deep neural networks; (iii) availability of the ground truth damage state from the results of the finite-element simulation; and (iv) direct comparison of different approaches to autonomous as-sessments. Moreover, the synthetic data generated has the potential to be used to augment field datasets. To demonstrate the efficacy of PBGMs, models of reinforced concrete moment-frame buildings with masonry infill walls are examined. The 3D synthetic environment employing PBGMs is shown to provide an effective testbed for development and validation of autonomous vision-based post-earthquake inspections that can serve as an important building block for ad-vancing autonomous data to decision frameworks.

The relationship between the street environment and the health, education, mobility, and criminal behaviors of its citizens has long been investigated by economists, sociologists and urban planners. Home buyers were found to pay a premium for better street appearance. Prior studies considering streetscapes mainly focus on objective measures such as the number of nearby trees, the tree canopy area, or the view index of physical features such as greenery, sky or building. However, subjective perceptions may have complex or subtle relationships to physical features, individual physical features or simply summing them up do not capture people’s comprehensive perception. In contrast, this study proposed a new approach for the urban-scale application to quantify both subjectively and objectively measured streetscape scores for six important perception qualities, namely Greenness, Walkability, Safety, Imageability, Enclosure, and Complexity. Built on prior quantitative studies in urban design quality and emerging applications in deep learning and open source street view imagery for urban perceptions, we integrated existing frameworks to (1) effectively collect and evaluate both subjectively and objectively- measured perceptions; (2) investigate the coherence and divergence in ML-predicted subjective scores and formula-derived objective scores; and (3) compare their effects in affecting house prices taking Shanghai as a case study using a large-scale dataset on home transactions. The results implied: first, the percentage increase in sales price attributable to street scores is significant for both subjective and objective measurements. In general subjective scores explained more variance over structural attributes and objective scores in hedonic price model. Particularly, objective Greenness, subjective Safety and Imageability scores positively affected house prices. Second, for Greenness and Imageability scores, the subjective and objective measures exhibited opposite signs in affecting house prices, which implied that there might be mechanisms related to the psychological, social-demographical characteristics of street users that have not been fully incorporated by objective measures that taking view indices or recombination of them. In addition, certain objective measure might outperform subjective counterpart when the connotation of the perception is self-evident and not complicated, for example the Greenness. For those concepts were not familiar to the average person, subjective framework exhibits better performance. This is the first study comprehensively expanding hedonic price method with both subjectively and objectively measured streetscape qualities. It suggested that city authorities could levy a street environment tax to compensate the public budget invested in street environment where developers secured benefits from a price premium. This study enriches our understanding of the economic values of the subjective and objective measures street qualities. It sheds light on promising future study areas where the coherence and divergence of the two measurements should be further stressed.

As the Autonomous Vehicle (AV) industry is rapidly advancing, classification of non-motorized (vulnerable) road users (VRUs) becomes essential to ensure their safety and to smooth operation of road applications. The typical practice of non-motorized road users’ classification usually takes numerous training time and ignores the temporal evolution and behavior of the signal. In this research effort, we attempt to detect VRUs with high accuracy be proposing a novel framework that includes using Deep Transfer Learning, which saves training time and cost, to classify images constructed from Recurrence Quantification Analysis (RQA) that reflect the temporal dynamics and behavior of the signal. Recurrence Plots (RPs) were constructed from low-power smartphone sensors without using GPS data. The resulted RPs were used as inputs for different pre-trained Convolutional Neural Network (CNN) classifiers including constructing 227×227 images to be used for AlexNet and SqueezeNet; and constructing 224×224 images to be used for VGG16 and VGG19. Results show that the classification accuracy of Convolutional Neural Network Transfer Learning (CNN-TL) reaches 98.70%, 98.62%, 98.71%, and 98.71% for AlexNet, SqueezeNet, VGG16, and VGG19, respectively. The results of the proposed framework outperform other results in the literature (to the best of our knowledge) and show that using CNN-TL is promising for VRUs classification. Because of its relative straightforwardness, ability to be generalized and transferred, and potential high accuracy, we anticipate that this framework might be able to solve various problems related to signal classification.

In order to design efficient and sustainable office spaces and to automate lighting, heating and air circulation in these facilities, solving the challenge of occupancy prediction is crucial. In office spaces where large areas need to be observed, multiple sensors must be used for full coverage. In these cases, it is normally important to keep the costs low, but also to make sure that the privacy of the people who use such environments are preserved. Low-cost and low-resolution heat (thermal) sensors can be very useful to build solutions that address these concerns. However, they are extremely sensitive to noise artifacts which might be caused by heat prints of the people who left the space or by other objects which are either using electricity or exposed to sunlight. There are some earlier solutions for occupancy prediction that employ low-resolution heat sensors, however, they have not addressed nor compensate for such heat artifacts. Therefore, in this paper, we present a low-cost and low-energy consuming smart space implementation to predict the number of people in the environment based on whether their activity is static or dynamic in time. We use a low-resolution (8×8) and non-intrusive heat sensor to collect data from an actual meeting room. We propose two novel workflows to predict the occupancy; one based on computer vision and one based on machine learning. Besides comparing the advantages and disadvantages of these different workflows, we use several state-of-the-art explainability methods in order to provide a detailed analysis of the algorithm parameters and how the image properties influence the resulting performance. Furthermore, we analyze noise resources which affect the heat sensor data. We hope that our analysis brings light into understanding how to handle very low-resolution heat images in these environments. The presented workflows could be used in various domains and applications other than smart offices, where occupancy prediction is essential, e.g., for elderly care.

Research on Traffic Light Recognition (TLR) has grown in recent years, primarily driven by the growing interest in autonomous vehicles development. Machine Learning algorithms have been widely used to that purpose. Mainstream approaches, however, require large amount of data to properly work, and as a consequence, a lot of computational resources. In this paper we propose the use of Expert Instruction (IE) as a mechanism to reduce the amount of data required to provide accurate ML models for TLR. Given an image of the exterior scene taken from the inside of the vehicle, we stand the hypothesis that the picture of a traffic light is more likely to appear in the central and upper regions of the image. Frequency Maps of traffic light location were thus constructed to confirm this hypothesis. The frequency maps are the result of a manual effort of human experts in annotating each image with the coordinates of the region where the traffic light appears. Results show that EI increased the accuracy obtained by the classification algorithm in two different image datasets by at least 15%. Evaluation rates achieved by the inclusion of EI were also higher in further experiments, including traffic light detection followed by classification by the trained algorithm. The inclusion of EI in the PCANet achieved a precision of 83% and recall of 73% against 75.3% and 51.1%, respectively, of its counterpart. We finally presents a prototype of a TLR Device with that expert model embedded to assist drivers. The TLR uses a smartphone as a camera and processing unit. To show the feasibility of the apparatus, a dataset was obtained in real time usage and tested in an Adaptive Background Suppression Filter (AdaBSF) and Support Vector Machines (SVMs) algorithm to detect and recognize traffic lights. Results show precision of 100% and recall of 65%.

This paper presents the robotic platform, PiBot, that has been developed and that is aimed at improving the teaching of Robotics with vision to secondary students. Its computational core is the Raspberry Pi 3 controller board, and the greatest novelty of this prototype is the support developed for the powerful camera mounted on board, the PiCamera. An open software infrastructure written in Python language was implemented so that the student may use this camera, or even a WebCam, as the main sensor of this robotic platform. Also, higher level commands have been provided to enhance the learning outcome for beginners. In addition, a PiBot 3D printable model and the counterpart for the Gazebo simulator were also developed and fully supported. They are publicly available so that students and educational centers that do not have the physical robot or can not afford the costs of these, can nevertheless practice and learn or teach Robotics using these open platforms: DIY-PiBot and/or simulated-PiBot.

The rapid pace of urbanization, coupled with the growing need for environmental sustainability and technological innovation, has brought smart city initiatives to the forefront of global urban development. This paper provides a comparative analysis of smart city policies from leading cities such as Oslo, Zurich, and Canberra, and derives strategic insights for Qatar. By examining the strategies, priorities, outcomes, and challenges associated with implementing smart city initiatives in these cities, the study identifies areas of alignment and divergence with Qatar's own smart city policy framework. The research highlights potential opportunities for cross-learning and suggests how Qatar can leverage lessons from international best practices to enhance its smart city initiatives. Key areas of focus include fostering strategic partnerships, promoting innovation, and ensuring effective governance and data management systems. The findings provide actionable insights to improve the effectiveness of Qatar's smart city projects, aligning them with the Qatar National Vision 2030.

This paper presents a real-time eye blink detection system using computer vision techniques and machine learning. The system uti- lizes the MediaPipe face mesh model for facial landmark detection and calculates the Eye Aspect Ratio (EAR) to determine the eye state. Results demonstrate high accuracy and responsiveness, indicating po- tential applications in driver drowsiness detection, human-computer interaction, and medical diagnostics.

(1) Background: Efficient detection and rectification of defects or post-processing manufacturing conditions such as sharp edges and burrs on metal components are crucial for quality control in precision manufacturing industries. (2) Methods: This paper describes a lab-scale integrated system for real-time and automated metal edge image detection using a customized YOLOv5 ma-chine vision algorithm. The YOLOv5 algorithm and model were developed and embedded in the NVIDIA Jetson Nano microprocessor. The YOLOv5 model can detect three different image types for classification i.e., normal, sharp, and burrs edge on aluminum workpiece blocks. An integrated system connects the NVIDIA Jetson Nano with an embedded YOLOv5 model to a Mitsubishi Electric Melfa RV-2F-1D1-S15 manipulator to perform selective automated chamfer-ing and grinding when certain condition defects are detected. (3) Results: The model demonstrates durable performance, achieving a mean average precision of 0.886 across defect classes with minimal misclassifications. The Mitsubishi Electric Melfa RV-2F-1D1-S15 manipulator received input from the machine vision system and performed an automated chamfering and grinding process accordingly; (4) Conclusions: By integrating camera, embedded deep learning in the microprocessor and manipulator, automated chamfering and grinding process in metal component shapes can be efficiently rectified. This tailored solution promises to improve productivity and consistency in metal manufacturing and remanufacturing.

Edvard Munch, a founder of Expressionism was pre-occupied with depicting his subjective experience and the most prolific painter of self-portraits since Rembrandt. Based on evidence that faces and self-related images are more salient than other objects it is plausible that self-portraits, in which the subject and object are the same, reflect the artist’s state of mind. Although he suffered from a variety of physical and psychological illness Munch was not diagnosed with any specificity in his lifetime. Posthumous diagnoses include schizophrenia, anxiety, bipolar and other disorders. Recent research has revealed altered visual perception in such patients. The present study empirically analysed three stylistic elements of Munch’s painted self-portraits and portraits: contrast, colour and fractal dimension and his painted productivity, to determine if variations correlate with critical life events and if so whether they indicate states of mind. It found significant increases in contrast and colour brightness and persistent high complexity during critical periods supporting the conclusion that Munch is diagnostically best described as suffering from early onset schizophrenia and comorbid social anxiety disorder.

In the last decades we have experienced great social, cultural, and educational changes, spending more time performing tasks in near vision and less hours in outdoor activities, having more overexposure to digital screens, especially the youngest and increasingly from younger ages. It is common to see cases when performing tasks in near vision the posture or position of the head is not adequate, especially the youngest, being essential to have a correct posture of the head to avoid visual, muscular, or articular problems. We present a procedure to try to control the correct position of the head in real time when performing close-up activities. The aim is to detect the inclination of the head and to know if a bad position is adopted due to a visual problem or simply to inadequate visio-postural habits, warning us of the postural anomaly to correct it.

Это исследование сосредоточено на разработке системы искусственного зрения для гибкого дельта-робота-манипулятора и ее интеграции с межмашинной связью для оптимизации взаимодействия устройств в реальном времени. Эта интеграция направлена ​​на увеличение скорости роботизированной системы и улучшение ее общей производительности. Предлагаемое сочетание системы искусственного зрения с межмашинной связью позволяет обнаруживать и распознавать цели с высокой точностью в режиме реального времени в ограниченном пространстве, предназначенном для позиционирования, дальнейшей локализации и выполнения производственных процессов, таких как сборка или сортировка деталей. . В данном исследовании RGB-изображения используются в качестве входных данных для алгоритма MASK-R-CNN, а результаты обрабатываются в соответствии с особенностями прототипа дельта-манипулятора робота. Данные, полученные от MASK-R-CNN, адаптированы для использования в системе управления дельта-роботом с учетом ее уникальных характеристик и требований к позиционированию. Технология M2M позволяет манипулятору робота быстро реагировать на изменения, например, на перемещение объектов или изменение их положения, что имеет решающее значение для задач сортировки и упаковки. Система была протестирована в условиях, близких к реальным, чтобы оценить ее производительность и надежность.

Microscopic vision plays an important role in automated micro-assembly. However, some uncertain factors in the assembly process, such as occlusion and stains can lead to the mistakes of feature extraction. Herein, to solve the problem, the deep learning techniques are introduced into the feature recognition tasks, focusing on the attention mechanism and visualizing CNNs for DL-based microscopic vision. The main contributions are summarized as follows: The CBAM attention mechanism is combined with the YOLOv5 algorithm to improve the accuracy and robustness of feature extraction. The micropart feature occlusion experiment results show that at 70% occlusion degree, YOLOV5-CBAM can reach 97.9% mAP@0.5, which is 4.6% higher than the original one. Visualization analysis of DL-based model is conducted using Grad-CAM to make the decision result more transparent and avoid potential visual detection risks during assembly. The heatmap matching degree between GT area and high-light area is increased by 27.81% on average, which further verify the effectiveness of attention mechanism in micropart feature localization. Additionally, micropart surface stain and droplet quality classification models based on ResNet50 are trained to replace the manual sorting. The visual results are consistent with human eye discernment and judgement, confirming the reliability of parts and droplets sorting.

The state-of-the-art smart city has been calling for an economic but efficient energy management over large-scale network, especially for the electric power system. It is a critical issue to monitor, analyze and control electric loads of all users in system. In this paper, we employ the popular computer vision techniques of AI to design a non-intrusive load monitoring method for smart electric energy management. First of all, we utilize both signal transforms (including wavelet transform and discrete Fourier transform) and Gramian Angular Field (GAF) methods to map one-dimensional current signals onto two-dimensional color feature images. Second, we propose to recognize all electric loads from color feature images using a deep neural network with multi-scale feature extraction and attention mechanism. Third, we design our method as a cloud-based, non-intrusive monitoring of all users, thereby saving energy cost during electric power system control. Experimental results on both public and our private datasets have demonstrated our method achieves superior performances than its peers, and thus supports efficient energy management over large-scale Internet of Things.

Ultrasound (US) has become a widely used imaging modality in clinical practice, characterized by its rapidly evolving technology, advantages, and unique challenges such as low imaging quality and high variability. There is a critical need to develop advanced automatic US image analysis methods to enhance diagnostic accuracy and objectivity. Vision transformer, a recent innovation in machine learning, has demonstrated significant potential in various research fields, including general image analysis and computer vision, due to its capacity to process large datasets and learn complex patterns. Its suitability for automatic US image analysis tasks, such as classification, detection, and segmentation, has been recognized. This review provides an introduction to vision transformer and discusses its applications in specific US image analysis tasks, while also addressing the open challenges and potential future trends in its application in medical US image analysis. Vision transformer has shown promise in enhancing the accuracy and efficiency of ultrasound image analysis and is expected to play an increasingly important role in the diagnosis and treatment of medical conditions using ultrasound imaging as technology progresses.

Cycle times at workstations in offsite construction factories fluctuate widely due to various influencing factors. Consequently, relying on average rates, such as length per unit of time, for estimating cycle times proves to be inaccurate, often leading to significant deviations between production schedules and actual operations. To address this issue, this study proposes an estimation system that leverages machine-learning-based prediction, statistical methods, 3D simulation, and computer vision to predict cycle times at the workstation level. Testing of the system on a semi-automated wood-wall framing workstation in a panelized construction factory shows that it reduces the mean absolute error and sum of errors by approximately 36% and 68%, respectively, compared to the fixed rate method. The results also highlight the efficacy of using computer vision data for training machine-learning models for cycle time estimation, the importance of identifying and understanding the factors influencing cycle times, and the impact of random delays on the accuracy of cycle time estimation systems.

ORB (Oriented FAST and Rotated Brief) features are the most commonly used features in visual SLAM and visual odometry that have high computational speed, and rotation scale invariance. However, due to the homogenization of ORB features, the corner properties of specific features are poor in some artificial environments, and it can lead to poor matching performance when the environmental texture is not rich. The underlying reason is that the pixel grayscale changes in the less-richly textured regions on the image are not obvious, leading to a certain degree of mis-matching in the matching process. At the same time, when the camera motion speed is low, there is much overlap between adjacent frames. This results in minimal or almost no changes in the projection of feature points, and the system is highly sensitive to errors, In this case, even small errors can cause significant fluctuations in the calculation results. In response to the above issues, an improved feature point method for stereo vision mileage calculation is proposed to solve the problem of system stability deterioration when cameras move at low speeds in artificial envi-ronments. Firstly, weight calculation based on different texture regions is used to solve the problem of low corner properties of feature points caused by environmental texture differences. Then, keyframes are used for motion model estimation to improve the system's stability under low-speed motion. The experimental results of the dataset showed that the key frame rate was optimal between 10% and 12%, and the positioning accuracy was higher than these open-source systems compared with three common open-source VO systems.

Indoor localization (IL) is a significant topic of study with several practical applications. The area of IL has evolved greatly in recent years due to the introduction of numerous technologies such as WiFi, Bluetooth, cameras, and other sensors. Despite the growing interest in this field, there are numerous challenges and drawbacks that must be addressed to develop more accurate and sustainable systems for IL and its real-life applications. This review study gives an in-depth look into IL, covering the most promising artificial intelligence-based and hybrid strategies that have shown excellent potential in overcoming some of the limitations of classic methods. In addition, the paper investigates the significance of high-quality datasets and evaluation metrics in the design and assessment of IL algorithms. Furthermore, this overview study emphasizes the crucial role that machine learning techniques, such as deep learning and transfer learning, play in the advancement of IL. A focus on the importance of IL and the various technologies, methods, and techniques that are being used to improve it. Finally, The survey highlights the need for continued research and development to create more accurate and scalable techniques that can be applied across a range of industries, such as evacuation-egress routes, hazard-crime detection, smart occupancy-driven energy reduction and asset tracking and management.

The appropriate maintenance of highway roads is critical for the safe operation of road networks and conserves maintenance costs. Multiple methods have been developed to investigate the surface of roads for various types of cracks and potholes, among other damage. Like road surface damage, the condition of expansion joints in concrete pavement is important to avoid unexpected hazardous situations. Thus, in this study, a new system is proposed for autonomous expansion joint monitoring using a vision-based system. The system consists of the following three key parts: (1) a camera-mounted vehicle, (2) indication marks on the expansion joints, and (3) a deep learning-based automatic evaluation algorithm. With paired marks indicating the expansion joints in a concrete pavement, they can be automatically detected. An inspection vehicle is equipped with an action camera that acquires images of the expansion joints in the road. You Only Look Once (YOLO) automatically detects the expansion joints with indication marks, which has a performance accuracy of 95%. The width of the detected expansion joint is calculated using an image processing algorithm. Based on the calculated width, the expansion joint is classified into the following two types: normal and dangerous. The obtained results demonstrate that the proposed system is very efficient in terms of speed and accuracy.

Intertidal seagrass plays a vital role in estimating the overall health and dynamics of coastal environments due to its interaction with tidal changes. However, most seagrass habitats around the globe have been in steady decline due to human impacts, disturbing the already delicate balance in environmental conditions that sustain seagrass. Miniaturization of multi-spectral sensors has facilitated very high resolution mapping of seagrass meadows, which significantly improve the potential for ecologists to monitor changes. In this study, two analytical approaches used for classifying intertidal seagrass habitats are compared: Object-based Image Analysis (OBIA) and Fully Convolutional Neural Networks (FCNNs). Both methods produce pixel-wise classifications in order to create segmented maps, however FCNNs are an emerging set of algorithms within Deep Learning with sparse application towards seagrass mapping. Conversely, OBIA has been a prominent solution within this field, with many studies leveraging in-situ data and multiscale segmentation to create habitat maps. This work demonstrates the utility of FCNNs in a semi-supervised setting to map seagrass and other coastal features from an optical drone survey conducted at Budle Bay, Northumberland, England. Semi-supervision is also an emerging field within Deep Learning that has practical benefits of achieving state of the art results using only subsets of labelled data. This is especially beneficial for remote sensing applications where in-situ data is an expensive commodity. For our results, we show that FCNNs have comparable performance with standard OBIA method used by ecologists, while also noting an increase in performance for mapping ecological features that are sparsely labelled across the study site.

Purpose The gradual increase in geriatric issues and global imbalance of the ratio between patients and healthcare professionals has created a demand for intelligent systems with the least error-prone diagnosis results to be used by less medically trained persons and save clinical time. This paper aims at investigating the development of an image-based colourimetric analysis. The purpose of recognising such tests is to support wider users to begin a colourimetric test to be used at homecare settings, telepathology, etc. Design/methodology/approach The concept of an automatic colourimetric assay detection is delivered by utilising two cases. Training Deep Learning (DL) models on thousands of images of these tests using transfer learning, this paper i) classifies the type of the assay, and ii) classifies the colourimetric results. Findings This paper demonstrated that the assay type can be recognised using DL techniques with 100% accuracy within a fraction of a second. Some of the advantages of the pre-trained model over the calibration-based approach are robustness, readiness and suitability to deploy for similar applications within a shorter period of time. Originality/value To the best of our knowledge, this is the first attempt to provide Colourimetric Assay Type Classification (CATC) using DL. Humans are capable to learn thousands of visual classifications in their life. Object recognition may be a trivial task for humans, due to photometric and geometric variabilities along with the high degree of intra-class variabilities it can be a challenging task for machines. However, transforming visual knowledge into machines, as proposed, can support non-experts to better manage their health and reduce some of the burdens on experts.

Autonomous aerial refueling technology is an effective solution to extend flight duration of unmanned aerial vehicles, and also a great challenge due to its high risk. For autonomous probe-and-drogue refueling tasks, relative navigation to provide relative position between the receiver aircraft and the refueling drogue is the first and essential step, and vision-based method is the most frequently used. A new monocular vision navigation sensor with image processing strategy consisting of the drogue detection method and the tracking method is developed for autonomous aerial refueling in this paper. In the drogue detection method, thresholding and mathematical morphology method are adopted to eliminate image interference, and contours extraction method is applied to obtain all contours, which are then subsequently checked to achieve target contour of drogue. In the tracking method, a rectangle of interest (ROI) of current frame image is determined by positioning results of last frame, and then processed by the previous drogue detection method. Finally, the proposed image processing strategy in monocular vision navigation sensor is validated using real flight images, which are captured from an autonomous aerial refueling testbed using a micro six-rotor aircraft as receiver aircraft.

The European Space Agency (ESA) defines as Earth observation (EO) Level 2 information product a single-date multi-spectral (MS) image corrected for atmospheric, adjacency and topographic effects, stacked with its data-derived scene classification map (SCM), whose thematic map legend includes quality layers cloud and cloud-shadow. ESA EO Level 2 product generation is an inherently ill-posed computer vision (CV) problem never accomplished to date in operating mode by any EO data provider at the ground segment. Herein, it is considered: (I) necessary not sufficient pre-condition for the yet-unaccomplished dependent problems of semantic content-based image retrieval (SCBIR) and semantics-enabled information/knowledge discovery (SEIKD) in multi-source EO big data cubes. (II) Synonym of EO Analysis Ready Data (ARD) format. (III) Equivalent to a horizontal policy for background developments in Space Economy 4.0. In compliance with the GEO-CEOS Quality Assurance Framework for EO Calibration/Validation guidelines, to contribute toward filling an analytic and pragmatic information gap from multi-sensor EO big data to timely, comprehensive and operational EO value-adding information products and services, this work presents an innovative AutoCloud+ CV software toolbox for cloud and cloud-shadow quality layer detection in ESA EO Level 2 product. In vision, spatial information dominates color information. Inspired by this true-fact, the inherently ill-posed AutoCloud+ CV software was conditioned, designed and implemented to be “universal”, meaning fully automated (no human-machine interaction is required), near real-time, robust to changes in input data and scalable to changes in MS imaging sensor’s spatial and spectral resolution specifications.

Background/Objectives: The World Federation of the Deafblind Global Report 2023 reports that many countries do not have a comprehensive identification, assessment, and referral system for persons with deafblindness, a combination of hearing and vision loss, across all age groups and geographic regions. The scan seeks to inform researchers, policymakers, and community-based organizations about the status of and gaps in sensory healthcare initiatives in Canada, with intent to raise awareness to enhance integration and coordination of eye and ear care services. Methods: We conducted an environmental scan of Canada’s healthcare system and current public health policies addressing vision and hearing care in Canada at federal and provincial levels. The scan was conducted using published literature searches from five scientific databases – Embase, Medline, PsycINFO, PubMed and CINAHL, in combination with the grey literature review of federal, provincial, and territorial government and non-profit organizations websites during April-October 2022. Out of 1257 articles screened, 82 studies included that met the inclusion/exclusion criteria. 13 reports were included in the grey literature search. Results: Thematic findings indicate stigma and discrimination towards individuals with disabilities and marginalized groups, including hearing, vision, or dual sensory loss, persist. Barriers to vision and hearing healthcare access include inadequate policies, underinvestment in vision and/or hearing services, limited collaboration and coordinated services between hearing and vision services, discrepancies in insurance coverages, and lack of health system support. Conclusions: This scan demonstrates the persistent barriers to vision and/or hearing services present in Canada, stemming from inadequate policy and limited-service coordination. Future work to address gaps, evaluate public education, and develop integrated sensory health care initiatives to enhance coordinate eye and ear care services as recommended in WHO Report on Hearing and Vision is imperative.

A large amount of sea ice exists in the polar regions, which is prone to ship-ice collisions leading to ship breakage and sinking events. Complex environmental factors pose challenges to navigation safety, and it is particularly important to perceive the polar environment and make decisions in advance. In this paper, You only look once (YOLOv5) is targeted to be optimized according to the characteristics of ice remote sensing images, which includes adding Squeeze-and-Excitation (SE) attention mechanism, improving spatial pyramid pooling, and replacing Flexible Rectified Linear Unit (FReLU) activation function. In order to verify the effect of the optimized algorithm, ablation experiments are set up, and the accuracy of the optimized algorithm is improved by 3.5% com-pared with the original YOLOv5. Comparison experiments with other target identification algo-rithms are conducted, and the results show that the optimized YOLOv5 has higher accuracy and better robustness, and can identify small target sea ice that is difficult to identify in large-size re-mote sensing images. Polar ship path planning based on the identification results. The identifica-tion output of remote sensing ice images is used as the input for the construction of path planning maps, and raster maps corresponding to the actual polar scenes are built. Taking the path length, the number of ship turns, and the sea ice risk value as the objective function, the Any-Angle Path Planning on Grids (Theta*) algorithm is used to simulate the path planning under different sea conditions with different sea ice concentration, and a path with a shorter length, a smoother path, and a higher safety coefficient is found.

Crowd abnormal behavior recognition is one of the research hotspots in computer vision. Its goal is to use computer vision technology and abnormal behavior detection models to accurately perceive, predict, and intervene in potential abnormal behaviors of the crowd, and monitor the status of the crowd system in public places in real-time, to effectively prevent and deal with public security risks and ensure public life safety and social order. To this end, focusing on the crowd abnormal behavior recognition technology in the computer vision system, a systematic review study of its theory and cutting-edge technology is conducted. First, the crowd level and abnormal behaviors in public places are defined, and the challenges faced by crowd abnormal behavior recognition are expounded. Then, from the dimensions based on traditional methods and based on deep learning, the mainstream technologies of abnormal behavior recognition are discussed, and the design ideas, advantages, and limitations of various methods are analyzed. Next, the mainstream software tools are introduced to provide a comprehensive reference for the technical framework; Secondly, typical abnormal behavior datasets at home and abroad are sorted out, and the characteristics of these datasets are compared in detail from multiple perspectives such as scale, characteristics and uses, and the performance indicators of different algorithms on the datasets are compared and analyzed; Finally, the full text is summarized and the future development direction of crowd abnormal behavior recognition technology is prospected.

This research explores the transformative impact of artificial intelligence (AI) on personalizing marketing campaigns, highlighting how AI technologies are reshaping modern marketing practices. AI tools such as machine learning, natural language processing (NLP), predictive analytics, deep learning, and computer vision are integral to enhancing customer engagement and optimizing marketing strategies. Machine learning algorithms enable precise customer segmentation and personalized recommendations, leading to more targeted and effective marketing messages. NLP facilitates the analysis of customer sentiment and enhances communication through advanced language processing and AI-powered chatbots. Predictive analytics and deep learning provide businesses with the capability to forecast customer behaviors and recognize complex data patterns, further refining marketing efforts. Computer vision technology contributes to personalization through innovations like visual search and augmented reality, creating immersive customer experiences. Despite the numerous benefits, the adoption of AI in marketing presents several challenges. Data privacy and security concerns, integration complexities, and the potential for algorithmic bias are critical issues that need to be addressed. Ethical considerations, including transparency and fairness in data usage, are also paramount to maintaining customer trust and ensuring responsible AI application. The future of AI in marketing promises continued advancements, with trends such as hyper-personalization and enhanced integration with technologies like the Internet of Things and voice search expected to drive further innovation. Overall, AI significantly enhances marketing effectiveness by offering tailored solutions and strategic insights while necessitating a thoughtful approach to ethical and practical challenges.

The Eurasian grapevine (Vitis vinifera L.) is the most widely grown horticultural crop in the world and is important for the economy of many countries. In the wine production chain, grape varieties play an important role as they directly influence the authenticity and classification of the product. Identifying the different grape varieties is therefore fundamental for quality control and inspection activities, as well as for regulating production. Currently, ampelography and molecular analysis are the main approaches to identifying grape varieties. However, both methods have limitations. Ampelography is subjective and prone to errors and is experiencing enormous difficulties as ampelographers are increasingly scarce. On the other hand, molecular analyses are very demanding in terms of cost and time. In this scenario, Deep Learning (DL) and Machine Learning (ML) methods have emerged as a classification alternative to deal with the scarcity of ampelographs and avoid molecular analyses. In this study, the most recent and current methods for identifying grapevine varieties using DL classification-based approaches are presented through a systematic literature review. The classification pipeline of the 31 studies found in the literature was described, highlighting its pros and cons. Most of the studies used DL-based models trained with leaf images acquired in a controlled environment at a maximum distance of 1.2 metres to classify grape varieties. In addition, there is a large gap between practical applications and the datasets used: a great lack of varieties, limited data acquired in the field and a lack of tests on plants under adverse conditions. Potential directions for improving this area of research were also presented.

All types of sports are potential application scenarios for automatic and real-time visual object and event detection. In rink hockey, the popular roller quad skate variant of hockey team sports, it is of great interest to automatically track player’s movements and positions, player’s sticks and, also, making other judgments, such as being able to locate the ball. In this work, we introduce a real-time pipeline composed by an object detection model, created specifically for rink hockey games, followed by a knowledge-based event detection module. Even in the presence of occlusions and quick motions, our deep learning object detection model effectively identifies and tracks, in real-time, important visual elements such as: ball; players; sticks; referees; crowd; goalkeeper; and goal. Using a curated dataset composed by a collection of videos of rink hockey, comprising 2525 annotated frames, we trained and evaluated the algorithm performance and compare it to state of the art object detection techniques. Our object detection model, based on YOLOv7, presents a global accuracy of 80%, and presents a good performance in terms of accuracy and speed, according to our results, making it a good choice for rink hockey applications. In our initial tests, the event detection module successfully detected one important event type in rink hockey games, the occurrence of penalties.

This research introduces a pioneering solution to the challenges posed by gastrointestinal tract (GI) cancer in radiation therapy, focusing on the imperative task of precise organ segmentation for minimizing radiation-induced damage. GI imaging has historically used manual demarcation, which is laborious and uncomfortable for patients. We address this by introducing the ResECA-U-Net deep learning model, a novel combination of the UNet and ResNet34 architectures. Furthermore, we further augment its functionality by incorporating the Efficient Channel Attention (ECA-Net) methodology. By utilizing data from the UW-Madison Carbone Cancer Centre, we carefully investigate several image processing techniques designed to capture critical local characteristics. With its foundation in computer vision concepts, the ResECA-U-Net model is excellent at extracting fine details from GI images. Sophisticated metrics such as intersection over union (IoU) and the dice coefficient are used to evaluate performance. Our study's outcomes demonstrate the effectiveness of the suggested method, yielding an impressive 96.27% Dice coefficient and 91.48% IoU. These results highlight the significant contribution that our strategy has made to the advancement of cancer therapy. Beyond its scientific merits, this work has the potential to significantly enhance cancer patients' quality of life and provide better long-term outcomes. Our work is a significant step towards automating and optimizing the segmentation process, which can potentially change how GI cancer is treated completely.

People often make mistakes, so we try to automate every aspect of our lives. Sports is no exception. While just over a decade ago humans were analyzing games, today this is being done by artificial intelligence. Due to rapid development over the past decade, neural networks are now faster, more accurate, and in some areas even better than their human counterparts. In this paper, we present an algorithm that can detect player statistics during an NBA broadcast. It also helps users better understand the game and the use of augmented reality. The algorithm detects players on the court, tracks their movements, and assigns them to their respective teams. Using homography estimation, we transform the players’ positions from a three-dimensional space in the video to a two-dimensional space on the playing field plane. We define a new algorithm that predicts the players’ actions and their statistics. The results show that the proposed method can effectively identify the players, their respective teams, and their positions. It can also analyze their actions with high accuracy.

There is a need for cost-effective and non-invasive methods of monitoring feeding behavior in livestock operations, considering the significant impact of feed costs on economic efficiency and assisting in detecting health issues of group-fed animals. This paper proposes using deep learning-based computer vision techniques to detect pen-fed beef cattle feeding behavior using Mask Region-based Convolutional Neural Network (RCNN). A deep learning model was pre-trained on the Common Objects in Context (COCO) dataset to generate cattle instance segmentation. Manually defined feed bunk polygons are compared with these segmentation masks to derive feeding time for each bunk. A full day’s worth of video data and the corresponding physical sensor data are collected for the experiment. By benchmarking the computer vision detected data with physical ground truth over random time segments from morning to evening (thus various lighting conditions), the optimal thresholds for Mask RCNN are determined to be 0.7 for bounding boxes and 0.1 for masks. Using these parameters. The reports suggest that the computer vision system achieved a precision of 87.2% and a recall of 89.1%, signifying precise detection of feeding events. Our study, to the best of our knowledge, was one of the first investigations of instance segmentation on feeding time sense, which combines deep learning methods with traditional computer vision logistics, reporting on feeding time data collection and processing, camera testing and adjustment, and performance evaluation. Future research directions include computer vision applied in feed grading and animal re-identification for individual production analysis.

The Kingdom of Saudi Arabia (KSA) is known for its high car ownership and usage as well its high GDP per capita. This is combined with low/ or no provision of public transportation (PT) systems, has been resulting in perceptual attitudes of full dependency on private car travel. The level of awareness of the benefits of reducing car use and increasing the travel by more sustainable options, has a great impact on social change and behaviour. The Kingdom is currently progressing towards a new phase of “national transfer” through implementation of strategic and sustainable measures and programs. The city of Riyadh is construction a massive metro-system in Riyadh, that is nearing completion and operation. The public is aware of the national agenda, aware of the newly constructed projects and aware of the needed social change to realize the new vision of the country. This paper aims to assess travel behaviour and attitudes towards public transportation of Saudi travellers’ who are witnessing the new transformation in the Kingdom and who are aware of the new sustainable projects. Depicted from the theory of random utility, a discrete choice model of the intent to use public transportation is calibrated as a function of social and attitudinal factors. An online survey was designed and carried out using social media means; a completed 399 questionnaires have been obtained and studied. The methodology includes examining attitudes and preferences of the participants towards using public transportation options against participants’ socio-economic data. The analysis was carried out using ordinal logistic regression analysis (OLR) which is an efficient technique that is derived from the theory of random utility. The results show a good support for PT; a higher support for public transportation modes, form participants who are young females, lower income groups and the university graduates were reported. The level of support seems higher with the higher level of awareness about the new PT system.

Identifying a vestibular source of pathology in patients complaining of post-traumatic brain injury (TBI) dizziness can be difficult. We describe a possible new method utilizing the reduction of post-TBI symptom (including dizziness) with the use of a noise cancellation device (NCD). This retrospective case series included patients with TBI and dizziness presenting to a binocular vision specialty clinic, who were diagnosed with a vertical heterophoria (VH). If they did not respond adequately to microprism lenses and/or if they experienced hyperacusis, they were evaluated with an NCD. If there was marked reduction of TBI symptoms (including dizziness), the patients were referred to a neuro-otologist for vestibular diagnostic evaluation and treatment. Fourteen patients were identified and found to have abnormalities on vestibular testing consistent with Third Mobile Window Disorder (TMWD). All were treated with a 6-week medical protocol (diuretics, no straining, low sodium/no caffeine diet). Five responded positively, requiring no further treatment. Nine required surgical intervention and responded positively. In conclusion, 14 patients with post-concussive dizziness and VH, a positive response to NCD was associated with abnormal vestibular testing, a diagnosis of TMWD, and symptom reduction/resolution with a medical or surgical approach. The removal of sound resulting in reduction or resolution of vestibular symptoms represents an Inverse Tullio Phenomena.

Nuclear technologies have the potential to play a major role in the transition to a global net-zero society. Their primary advantage is the capability to deliver controllable 24/7 energy on demand. However, as a prerequisite for successful worldwide application, significant innovation will be required to create the nuclear systems of the 21st century, the need of the hour. The pros and cons of nuclear are discussed and analysed at different levels – the societal and public recognition as well as the scientific/engineering and economic level – to assure a demand driven development. Based on the analysis of the different challenges a vision for the nuclear system of the 21st century is synthesised consisting of three pillars – unlimited nuclear energy, zero waste nuclear, and accident free nuclear. These three combined visions are then transformed into dedicated and verifiable missions which are discussed in detail regarding challenges and opportunities. In the following a stepwise approach for the development of such a highly innovative nuclear system is described. Essential steps to assure active risk reduction and the delivery of quick progress are derived as answer to the critique on the currently observed extensive construction time and cost overruns on new nuclear plants. The 4-step process consisting of basic studies, experimental zero power reactor, small scale demonstrator, and industrial demonstrator is described. The 4 steps including sub-steps deliver the pathway to a successful implementation of such a ground breaking new nuclear system. The potential sub-steps are discussed with the view not only onto the scientific development challenges, but also as an approach to reduce the regulatory challenges of a novel nuclear technology.

In recent years, due to the advancement of machine learning, object detection has become a mainstream task in the computer vision domain. The first phase of object detection is to find the regions where objects can exist. With the improvement of deep learning, traditional approaches such as sliding windows and manual feature selection techniques have been replaced with deep learning techniques. However, object detection algorithms face a problem when performing in low light, challenging weather, and crowded scenes like any other task. Such an environment is termed a challenging environment. This paper exploits pixel-level information to improve detection under challenging situations. To this end, we exploit the recently proposed hybrid task cascade network. This network works collaboratively with detection and segmentation heads at different cascade levels. We evaluate the proposed methods on three complex datasets of ExDark, CURE-TSD, and RESIDE and achieve an mAP of 0.71, 0.52, and 0.43, respectively. Our experimental results assert the efficacy of the proposed approach.

Autonomous driving systems are increasingly becoming a necessary trend towards building smart cities of the future. Numerous proposals have been presented in recent years to tackle particular aspects of the working pipeline towards creating a functional end-to-end system, such as object detection, tracking, path planning, sentiment or intent detection. Nevertheless, few efforts have been made to systematically compile all of these systems into a single proposal that effectively considers the real challenges these systems will have on the road, such as real-time computation, hardware capabilities, etc. This paper has reviewed various techniques towards proposing our own end-to-end autonomous vehicle system, considering the latest state on the art on computer vision, DSs, path planning, and parallelization.

Soybean maturity is a trait of critical importance for the development of new soybean cultivars, nevertheless, its characterization based on visual ratings has many challenges. Unmanned aerial vehicles (UAVs) imagery-based high-throughput phenotyping methodologies have been proposed as an alternative to the traditional visual ratings of pod senescence. However, the lack of scalable and accurate methods to extract the desired information from the images remains a significant bottleneck in breeding programs. The objective of this study was to develop an image-based high-throughput phenotyping system for evaluating soybean maturity in breeding programs. Images were acquired twice a week, starting when the earlier lines began maturation until the latest ones were mature. Two complementary convolutional neural networks (CNN) were developed to predict the maturity date. The first using a single date and the second using the five best image dates identified by the first model. The proposed CNN architecture was validated using more than 15,000 ground truth observations from five trials, including data from three growing seasons and two countries. The trained model showed good generalization capability with a root mean squared error lower than two days in four out of five trials. Four methods of estimating prediction uncertainty showed potential at identifying different sources of errors in the maturity date predictions. The architecture used solves limitations of previous research and can be used at scale in commercial breeding programs.

Crowd accident surveys have shown that regardless of the initial triggering factors, pedestrian fall behaviour is the most critical factor causing and aggravating crowd accidents in public traffic areas (PTAs). Application of pedestrian fall behaviour detection methods in PTAs is significant. Once deployed, it would prevent many pedestrians from losing life in crowed traffic areas accidents. However, most existing methods is still focused on medical assistance for the elderly. Therefore, this paper conducted bibliometric and content analyses, combining fall detection-related keywords to retrieve from internationally recognized literature databases and benchmark pedestrian behaviour datasets. Based on the analysis of the state-of-the-art (SOTA) achievements in fall detection methods, the fall detection methods were classified into different categories according to the research approach. Among them, it undertakes a comprehensive analysis of five predominant methods, namely computer vision, Internet of Things, smartphone, kinematic, and wearable device-based methods. Furthermore, the benchmark datasets including fall scenarios were introduced and compared. Finally, it provides a detailed discussion of existing fall detection methods, and possible future directions are identified considering application requirements in PTAs. This overview may help researchers understand the SOTA fall detection methods and devise new methodologies by improving and synthesizing the highlighted issues in PTAs.

Cancer diagnosis and classification are pivotal for effective patient management and treatment planning. In this study, we present a comprehensive approach utilizing ensemble deep learning techniques to analyze breast cancer histopathology images. Our datasets are based on two widely employed datasets from different centers for two different tasks: BACH and BreakHis. Within the BACH Dataset, we deployed an ensemble strategy incorporating VGG16 and ResNet50 architec-tures to achieve precise classification of breast cancer histopathology images. Introducing a novel image patching technique, to preprocess a high-resolution image, which facilitates focused analysis of localized regions of interest. The annotated BACH dataset encompasses 400 WSIs across four distinct classes: Normal, Benign, In Situ Carcinoma, and Invasive Carcinoma. In addition, we employed the BreakHis dataset, utilizing VGG16, ResNet34, and ResNet50 models to classify mi-croscopic images into eight distinct categories (four benign and four malignant). For both models we leveraged a five-fold cross-validation approach for rigorous training and testing. Preliminary ex-perimental results indicate a Patch classification accuracy of 95.31% (on BACH dataset) and WSI image classification accuracy of 98.43% (BreakHis). This research significantly contributes to on-going endeavors in harnessing artificial intelligence to advance breast cancer diagnosis, potentially fostering improved patient outcomes and alleviating healthcare burdens.

Vertical Indoor Farming (VIF) with hydroponics offers a promising perspective for sustainable food production. Intelligent control of VIF system components plays a key role in reducing operating costs and increasing crop yields. Modern machine vision (MV) systems use Deep Learning (DL) in combination with camera systems for various tasks in agriculture, such as disease and nutrient deficiency detection, and flower and fruit identification and classification for pollination and harvesting. This study presents the applicability of MV technology with DL modelling to detect the growth stages of chilli plants using YOLOv8 networks. The influence of different bird’s eye and side view datasets and different YOLOv8 architectures was analysed. To generate the image data for training and testing the YOLO models, chilli plants were grown in a hydroponic environment and imaged throughout their life cycle using four camera systems. The growth stages were divided into growing, flowering and fruiting classes. All trained YOLOv8 models showed reliable identification of growth stages with high accuracy. The results indicate that models trained with data from both views show better generalisation. YOLO’s middle architecture achieved the best performance.

Robotic odor source localization (OSL) is a technology that enables mobile robots or autonomous vehicles to find an odor source in unknown environments. An effective navigation algorithm that guides the robot to approach the odor source is the key to successfully locating the odor source. While traditional OSL approaches primarily utilize an Olfaction-only strategy, guiding robots to find the odor source by tracing emitted odor plumes, our work introduces a fusion navigation algorithm that combines both vision and olfaction-based techniques. This hybrid approach addresses challenges such as turbulent airflow, which disrupts olfaction sensing, and physical obstacles inside the search area, which may impede vision detection. In this work, we propose a hierarchical control mechanism that dynamically shifts the robot’s search behavior among four strategies: crosswind maneuver, Obstacle-avoid Navigation, Vision-based Navigation, and Olfaction-based Navigation. Our methodology includes a custom-trained deep-learning model for visual target detection and a moth-inspired algorithm for Olfaction-based navigation. To assess the effectiveness of our approach, we implemented the proposed algorithm on a mobile robot in a search environment with obstacles. Experimental results demonstrate that our Vision and Olfaction Fusion algorithm significantly outperforms Vision-only and Olfaction-only methods, reducing average search time by 54% and 30%, respectively.

The intricate functionality of the vertebrate retina relies on the interplay between neurotransmitter activity and calcium dynamics, offering important insights into de-velopmental processes, physiological functioning, and disease progression. Neuro-transmitters orchestrate cellular processes to shape the behavior of the retina under diverse circumstances. Despite research to elucidate the roles of individual neuro-transmitters in the visual system, there remains a gap in our understanding of the holistic integration of their interplay with calcium dynamics in the broader context of neuronal development, health, and disease. To address this gap, the present review explores the mechanisms used by the neurotransmitters glutamate, GABA, glycine, dopamine, and acetylcholine and their interplay with calcium dynamics. This con-ceptual outline is intended to inform and guide future research, underpinning novel therapeutic avenues for retinal-associated disorders.

Skin cancer is one of the widespread diseases that typically develop on the skin due to continuous exposure to sunlight. Although cancer can appear on any part of the human body, skin cancer reports account for over half of all cancer occurrences worldwide. There are substantial obstacles to the precise diagnosis and classification of skin lesions because of the morphological variety and indistinguishable characteristics across skin malignancies. Recently, Deep Learning models have been used in the field of image-based lesion diagnosis, and it has demonstrated diagnostic efficiency on par with that of dermatologists. To increase classification efficiency and accuracy for skin lesions, a cutting-edge multi-layer deep Convolutional Neural Network (CNN) termed SkinLesNet has been built in this study. The ResNetV50 and VGG16 models have been carefully compared to review the performance of the proposed model. The dataset used in this study, PAD-UFES-20, contains 1314 samples in total and includes three common forms of skin lesions. The proposed approach, SkinLesNet, significantly outperforms the well-known compared models in the given conditions

In autonomous driving systems, high-speed and real-time image processing, along with object recognition, are crucial technologies. This paper builds upon the research achievements in in-dustrial item sorting systems and proposes an object recognition and sorting system for auton-omous driving. In industrial sorting lines, goods sorting robots often need to work at high speeds to efficiently sort large volumes of items. This poses a challenge to the robot's real-time vision and sorting capabilities, making it both practical and economically viable to implement a real-time and low-cost sorting system in a real-world industrial sorting line. Existing sorting systems have limitations such as high cost, high computing resource consumption, and high power consump-tion. These issues lead to the fact that existing sorting systems are typically used only in large industrial plants. In this paper, we design a high-speed, low-cost, low-resource-consumption FPGA (Field-Programmable Gate Array) based item sorting system that achieves similar perfor-mance to current mainstream sorting systems at a lower cost and consumption than existing sorting systems. The recognition part employs a morphological recognition method, which segments the image using a frame difference algorithm and then extracts the color and shape features of the items. To handle sorting, a six-degree-of-freedom robotic arm is introduced in the sorting segment. The improved cubic B-spline interpolation algorithm is employed to plan the motion trajectory and consequently control the robotic arm to execute the corresponding actions. Through a series of experiments, this system achieves an average recognition delay of 25.26ms, ensures smooth operation of the gripping motion trajectory, minimizes resource consumption, and reduces implementation costs.

Deepfakes are manipulated or altered images, or video, that are created using deep learning models with high levels of photorealism. The two popular methods of producing a deepfake are based on either convolutional neural networks (CNN), or autoencoders. Deepfakes created using CNN comparatively show higher qualities of realism, yet oftentimes leave artifacts and distortions in the generated media that can be detected using machine learning and deep learning algorithms. In recent years, there has been an influx of periocular image and video data because of the increase usage of face masks. By wearing masks, much of what is used for facial recognition is hidden, leaving only the periocular region visible to an observer. This loss of vital information leads to easier misidentification of media, allowing deepfakes to less likely be identified as fake. In this work, feature extraction methods, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), and CNN, are used to train an ensemble deep learning model to detect deepfakes in videos on a frame-by-frame level based on the periocular region. Our proposed model is able to distinguish original and manipulated images with accuracies around 98.9 percent, which is an improvement to previous works by combining SIFT and HOG for deepfake detection in convolutional neural networks.

The central goal of few-shot scene classification is to learn a model that can generalize well to a novel scene category (UNSEEN) from only one or a few labeled examples. Recent works in the remote sensing (RS) community tackle this challenge by developing algorithms in a meta-learning manner. However, most prior approaches have either focused on rapidly optimizing a meta-learner or aimed at finding good similarity metrics while overlooking the embedding power. Here we propose a novel Task-Adaptive Embedding Learning (TAEL) framework that complements the existing methods by giving full play to feature embedding’s dual roles in few-shot scene classification - representing images and constructing classifiers in the embedding space. First, we design a lightweight network that enriches the diversity and expressive capacity of embeddings by dynamically fusing information from multiple kernels. Second, we present a task-adaptive strategy that helps to generate more discriminative representations by transforming the universal embeddings into task-specific embeddings via a self-attention mechanism. We evaluate our model in the standard few-shot learning setting on two challenging datasets: NWPU-RESISC4 and RSD46-WHU. Experimental results demonstrate that, on all tasks, our method achieves state-of-the-art performance by a significant margin.

Maze navigation using one or more robots has become a recurring challenge in scientific literature and real life practice, with fleets having to find faster and better ways to navigate environments such as a travel hub (e.g. airports) or to evacuate a disaster zone. Many methods have been used to solve this issue, including the implementation of a variety of sensors and other signal receiving systems. Most interestingly, camera-based techniques have increasingly become more popular in this kind of applications, given their robustness and scalability. In this paper, we have implemented an end-to-end strategy to address this scenario, allowing a robot to solve a maze in an autonomous way, by using computer vision and path planning. In addition, this robot shares the generated knowledge to another by means of communication protocols, having to adapt its mechanical characteristics to be able to solve the same challenge. The paper presents experimental validation of the four components of this solution, namely camera calibration, maze mapping, path planning and robot communication. Finally, we present the integration and functionality of these methods applied in a pair of NAO robots.

Currently the majority of studies on vision-based measurement has been conducted under ideal environments so that an adequate measurement performance and accuracy is ensured. However, vision-based systems may face some adverse influencing factors such as illumination change and fog interference, which can affect the measurement accuracy. This paper develops a robust vision-based displacement measurement method which can handle the two common and important adverse factors given above and achieve sensitivity at the subpixel level. The proposed method leverages the advantage of high-resolution imaging incorporating spatial and temporal context aspects. To validate the feasibility, stability and robustness of the proposed method, a series of experiments was conducted on a two-span three-lane bridge in the laboratory. The illumination change and fog interference are simulated experimentally in the laboratory. The results of the proposed method are compared to conventional displacement sensor data and current vision-based method results. It is demonstrated that the proposed method gives better measurement results than the current ones under illumination change and fog interference.

To build a representation of what we see, the human brain recruits regions throughout the visual cortex in cascading sequence. Recently, an approach was proposed to evaluate the dynamics of visual perception in high spatiotemporal resolution at the scale of the whole brain. This method combined functional magnetic resonance imaging (fMRI) data with magnetoencephalography (MEG) data using representational similarity analysis and revealed a hierarchical progression from primary visual cortex through the dorsal and ventral streams. To assess the replicability of this method, here we present results of a visual recognition neuro-imaging fusion experiment, and compare them within and across experimental settings. We evaluated the reliability of this method by assessing the consistency of the results under similar test conditions, showing high agreement within participants. We then generalized these results to a separate group of individuals and visual input by comparing them to the fMRI-MEG fusion data of Cichy et al. (2016), revealing a highly similar temporal progression recruiting both the dorsal and ventral streams. Together these results are a testament to the reproducibility of the fMRI-MEG fusion approach and allows for the interpretation of these spatiotemporal dynamic in a broader context.

Treatment efficacy for age-related macular degeneration relies on early diagnosis and precise determination of the disease stage. It involves analyzing biomarkers in retinal images, which can be challenging when handling a large flow of patients and compromise the quality of healthcare services. Clinical decision support systems offer a solution to this issue by employing intelligent algorithms to recognize biomarkers and specify the age-related macular degeneration stage through the analysis of retinal images. However, different stages of age-related macular degeneration may exhibit similar biomarkers, complicating the application of intelligent algorithms. This paper introduces an approach to overcome these challenges using hybrid and hierarchical classification. By leveraging the hybrid structure of the classifier, we can effectively manage issues commonly encountered with medical data sets, such as class imbalance and strong correlations between variables. The modifications to the intelligent algorithm proposed in this work for staging age-related macular degeneration resulted in an increase in average accuracy, sensitivity, and specificity by 20% compared to initial values. The Cohen’s Kappa coefficient used for consistency estimation between the regression model and expert assessments of the intermediate class severity was 0.708, indicating a high level of agreement.

This work investigates into the enhancement of iris recognition systems through a two-module approach focusing on low level image preprocessing techniques and advanced feature extraction. The first module is dedicated to the preprocessing of iris images, leveraging the Canny algorithm for edge detection followed by the circle-based Hough transform for precise iris extraction. This process, integral to our methodology, ensures the quality and consistency necessary for the subsequent Binary Statistical Image Features (BSIF) analysis. The second module employs the BSIF technique, incorporating domain-specific filters trained on iris-specific data, for robust biometric identification. By combining these advanced image preprocessing techniques, the proposed method addresses key challenges in iris recognition, such as occlusions, varying pigmentation, and textural diversity. Experimental results on the Human-inspired Domain-specific Binarized Image Features (HDBIF) Dataset composed of 1892 iris images demonstrate the effectiveness of the developped approach, with potential future improvements including adaptive algorithms and machine learning integrations for enhanced performance in diverse and unpredictable real-world applications. The novel contributions of this paper lie in the provision of a reproducible research framework, making the iris-domain-specific BSIF filters, training patches, testing database, and source codes publicly available for further application and study.

Directed movement towards a target requires spatial working memory, including processing of sensory inputs and motivational drive. In a stimulus-driven, operant conditioning paradigm designed to train zebrafish, we present a pulse of light via LED’s and/or sounds via an underwater transducer. A webcam placed below a glass tank records fish swimming behavior. During operant conditioning, a fish must interrupt an infrared beam at one location to obtain a small food reward at the same or different location. A timing-gated interrupt activates robotic-arm and feeder stepper motors via custom software controlling a microprocessor (Arduino). “Ardulink”, a JAVA facility, implements Arduino-computer communication protocols. In this way, full automation of stimulus-conditioned directional swimming is achieved. Precise multi­day scheduling of training, including timing, location and intensity of stimulus parameters, and feeder control is accomplished via a user-friendly interface. Our training paradigm permits tracking of learning by monitoring, turning, location, response times and directional swimming of individual fish. This facilitates comparison of performance within and across a cohort of animals. We demonstrate the ability to train and test zebrafish using visual and auditory stimuli. Current methods used for associative conditioning often involve human intervention, which is labor intensive, stressful to animals, and introduces noise in the data. Our relatively simple yet flexible paradigm requires a simple apparatus and minimal human intervention. Our scheduling and control software and apparatus (NemoTrainer) can be used to screen neurologic drugs and test the effects of CRISPR­-based and optogenetic modification of neural circuits on sensation, locomotion, learning and memory.

This work presents an approach for detecting mathematical formulas in scanned document images. The proposed approach is end-to-end trainable. Since many OCR engines cannot reliably work with the formulas, it is essential to isolate them to obtain the clean text for information extraction from the document. Our proposed pipeline comprises a hybrid task cascade network with deformable convolutions and a Resnext101 backbone. Both of these modifications help in better detection. We evaluate the proposed approaches on the ICDAR-2017 POD and Marmot datasets and achieve an overall accuracy of 96% for the ICDAR-2017 POD dataset. We achieve an overall reduction of error of 13%. Furthermore, the results on Marmot datasets are improved for the isolated and embedded formulas. We achieved an accuracy of 98.78% for the isolated formula and 90.21% overall accuracy for embedded formulas. Consequently, it results in an error reduction rate of 43% for isolated and 17.9% for embedded formulas.

Object detection is one of the most critical tasks in the field of Computer vision. This task comprises identifying and localizing an object in the image. Architectural floor plans represent the layout of buildings and apartments. The floor plans consist of walls, windows, stairs, and other furniture objects. While recognizing floor plan objects is straightforward for humans, automatically processing floor plans and recognizing objects is a challenging problem. In this work, we investigate the performance of the recently introduced Cascade Mask R-CNN network to solve object detection in floor plan images. Furthermore, we experimentally establish that deformable convolution works better than conventional convolutions in the proposed framework. Identifying objects in floor plan images is also challenging due to the variety of floor plans and different objects. We faced a problem in training our network because of the lack of publicly available datasets. Currently, available public datasets do not have enough images to train deep neural networks efficiently. We introduce SFPI, a novel synthetic floor plan dataset consisting of 10000 images to address this issue. Our proposed method conveniently surpasses the previous state-of-the-art results on the SESYD dataset and sets impressive baseline results on the proposed SFPI dataset. The dataset can be downloaded from SFPI Dataset Link. We believe that the novel dataset enables the researcher to enhance the research in this domain further.

This paper presents a novel architecture for detecting mathematical formulas in document images, which is an important step for reliable information extraction in several domains. Recently, Cascade Mask R-CNN networks have been introduced to solve object detection in computer vision. In this paper, we suggest a couple of modifications to the existing Cascade Mask R-CNN architecture: First, the proposed network uses deformable convolutions instead of conventional convolutions in the backbone network to spot areas of interest better. Second, it uses a dual backbone of ResNeXt-101, having composite connections at the parallel stages. Finally, our proposed network is end-to-end trainable. We evaluate the proposed approach on the ICDAR-2017 POD and Marmot datasets. The proposed approach demonstrates state-of-the-art performance on ICDAR-2017 POD at a higher IoU threshold with an f1-score of 0.917, reducing the relative error by 7.8%. Moreover, we accomplished correct detection accuracy of 81.3% on embedded formulas on the Marmot dataset, which results in a relative error reduction of 30%.

Interaction between multiple phases especially liquid-gas is one of the most common phenomena in industrial systems. Computer Vision enables modern systems to derive an automated and robust image-processing technique to address this issue. However, diverse experimental conditions still make it a challenging task to overcome. In Deep Learning, instance segmentation has created a niche in various computer vision applications as it aims to provide different instance IDs to objects belonging to the same class. In this paper, we propose a novel approach employing the YOLO (You Look Only Once) an end-to-end deep learning architecture for the detection and segmentation of bubbles in fluid flows. YOLOv5 offers significant improvements in both speed and accuracy compared to its predecessors like YOLOv3. Therefore, we train the YOLOv5 network on a dataset consisting of annotated images representing instances of bubbles across varied fluid flow environments. This paper provides both qualitative and quantitative evaluation of three YOLOv5 networks (small, medium, and extra-large) in detecting and segmenting bubbles with high accuracy and efficiency. Based on the metrics all three YOLOv5 models achieved high precision above 0.98 for bounding box detection and 0.97 for segmentation mask, and high recall value greater than 0.97 for box and 0.96 for mask quality was achieved. The achieved results indicate that the models accurately localize bubbles within the image frame, and distinguish bubble pixels from the background. Also, mean average precision (mAP50-95) consistently exceeds 0.84 across all models for the bounding box and 0.73 for the segmentation mask. The ascertained results emphasize the potential of YOLOv5 architecture as a powerful tool for automated bubble analysis in real-world applications. Our research contributes to advancing the field of deep neural networks in fluid systems and provides a fresh approach for future developments in automated bubble analysis.

Deployment of Level 3 and Level 4 autonomous vehicles (AVs) in urban environments is significantly constrained by adverse weather conditions, limiting their operation to clear weather due to safety concerns. Ensuring that AVs remain within their designated Operational Design Domain (ODD) is a formidable challenge, making boundary monitoring strategies essential for safe navigation. This study explores the critical role of an ODD monitoring system (OMS) in addressing these challenges. It reviews various methodologies for designing an OMS and presents a comprehensive visualization framework incorporating trigger points for ODD exits. These trigger points serve as essential references for effective OMS design. The study also delves into a specific use case concerning ODD exits: the reduction in road friction due to adverse weather conditions. It emphasizes the importance of contactless computer vision-based methods for road condition estimation (RCE), particularly using vision sensors such as cameras. The study details a timeline of methods involving classical machine learning and deep learning feature extraction techniques, identifying contemporary challenges such as class imbalance, lack of comprehensive datasets, annotation methods, and the scarcity of generalization techniques. Furthermore, it provides a factual comparison of two state-of-the-art RCE datasets. In essence, the study aims to address and explore ODD exits due to weather-induced road conditions, decoding the practical solutions and directions for future research in the realm of AVs.

Fall prediction using machine learning has become one of the most fruitful and socially relevant applications of computer vision in gerontological research. Since its inception in the early 2000s, this subfield has proliferated into a robust body of research underpinned by various machine learning algorithms (including neural networks, support vector machines, and decision trees) as well as statistical modeling approaches (Markov chains, Gaussian mixture models, and hidden Markov models). Furthermore, some advancements have been translated into commercial and clinical practice, with companies in various stages of development capitalizing on the aging population to develop new commercially available products. Yet despite the marvel of modern machine learning-enabled fall prediction, little research has been conducted to shed light on the security and privacy concerns that such systems pose for older adults. The present study employs an interdisciplinary lens in examining privacy issues associated with machine learning fall prediction and exploring the implications of these models in elderly care and the Internet of Medical Things (IoMT). Ultimately, a justice-informed set of best practices rooted in social geroscience is suggested to help fall prediction researchers and companies continue to advance the field while preserving elderly privacy and autonomy.

Color can direct visual attention to specific locations through bottom-up and top-down mechanisms. Using Continuous Flash Suppression (CFS) as way to investigate the factors that gate access to consciousness, the current study investigated whether color also directly affected the timing of conscious perception. Low or high spatial frequency (SF) gratings with different orientations were shown as targets to the non-dominant eye of human participants. CFS patterns were presented at a rate of 10Hz to the dominant eye to delay conscious perception of the targets, and participants had the task to report the target’s orientation as soon as they could see it. With low-SF targets, two types of color-based effects became evident. First, when the targets and the CFS patterns had different colors, the targets entered consciousness faster than in trials where the targets and CFS patterns had the same color. Second, when participants searched for a specific target color, targets that matched these search settings entered consciousness faster compared to conditions where the target color was irrelevant and could vary from trial to trial. Thus, the current study demonstrates that color is a central feature of human perception and leads to faster conscious perception of visual stimuli through bottom-up and top-down attentional mechanisms.

The fourth Industrial Revolution, well-known as “Industry 4.0”, based on the integration of information and communication technologies, has introduced significant improvements in manufacturing. However, vision systems still experience various impracticalities in dealing with the effect of complex lighting on the systems platform. Therefore, a machine vision system for automatic identification of pen parts under varying lighting conditions at a digital learning factory is proposed. The developed vision system presents a straightforward approach by effectively minimizing the environmental lighting effect on the identification process. First, the obtained information of the designed vision framework is exported to a program, where a reduction of non-uniform illumination is achieved through the implementation of Retinex image enhancement techniques. Then, the color-based Fuzzy C-means (FCM) algorithm, including improved mark watershed segmentation, is employed for pen parts object classification. Finally, the position features of the selected pen part are reported. The process applied to a total number of 210 upper pen parts (caps) and 241 lower pen parts (tubes) images under different lighting scenarios. Results indicate that average parts identification precision for cap and tube parts is different and equals to 98.64% and 95.26%, respectively. The present methodology provides a promising scheme that can be feasibly adapted for other industrial Color-based object recognition applications.

Dietary vitamin A/all-trans retinol/ROL plays a critical role in human vision. ROL circulates bound to the plasma retinol-binding protein (RBP4) as RBP4-ROL. In the eye, the STRA6 membrane receptor binds to circulatory RBP4 and internalizes ROL. STRA6 is however not expressed in systemic tissues, where there is high-affinity RBP4 binding and ROL uptake. We tested the hypothesis, that the second retinol-binding protein 4 receptor 2 (Rbpr2) which is highly expressed in systemic tissues of zebrafish and mouse, contains a functional RBP4 binding domain, critical for ROL transport. As for STRA6, modeling and docking studies confirmed three conserved RBP4 binding residues in zebrafish Rbpr2. In cell culture studies, disruption of the RBP4 binding residues on Rbpr2 almost completely abolished uptake of exogenous vitamin A. CRISPR generated rbpr2-RBP4 domain zebrafish mutants showed microphthalmia, shorter photoreceptor outer segments, and decreased opsins, that were attributed to impaired ocular retinoid content. Injection of WT-Rbpr2 mRNA into rbpr2 mutant or all-trans retinoic acid treatments rescued the mutant eye phenotypes. In conclusion, zebrafish Rbpr2 contains a putative extracellular RBP4-ROL ligand-binding domain, critical for yolk vitamin A transport to the eye for ocular retinoid production and homeostasis, for photoreceptor cell survival.

Until the 1950s, developmental dyslexia was defined as a hereditary visual disability, selectively affecting reading without compromising oral or non-verbal reasoning skills. This changed radically after the development of the phonological theory of dyslexia; this not only ruled out any role for visual processing in its aetiology, but also cast doubt on the use of discrepancy between reading and reasoning skills as a criterion for diagnosing it. Here I argue that this theory is set at too high a cognitive level to be explanatory; we need to understand the pathophysiological visual and auditory mechanisms that cause children’s phonological problems. I discuss how the ‘magnocellular theory’ attempts to do this in terms of slowed and error prone temporal processing which leads to dyslexics’ defective visual and auditory sequencing when attempting to read. I attempt to deal with the criticisms of this theory and show how it leads to a number of successful ways of helping dyslexic children to overcome their reading difficulties.

Background: Ulcerative colitis is a chronic inflammatory bowel disease of the colon mucosa associated with a higher risk of colorectal cancer. Objective: This study is a proof-of-concept analysis aimed to test the feasibility of classifying hematoxylin and eosin (H&E) histological images of ulcerative colitis, normal colon, and colorectal cancer using Artificial Intelligence (AI); Methods: A convolutional neural network (CNN) was designed and trained to classify the 3 types of diagnosis, including 35 cases of ulcerative colitis (n = 9,281 images), 21 colon control (n = 12,246), and 18 colorectal cancer (n = 63,725). The data were partitioned into training (70%) and validation sets (10%) for training the network, and a test set (20%) to test the performance on the new data. Explainable artificial intelligence for computer vision was performed using gradient-weighted class activation mapping (Grad-CAM) to understand the classification decisions; and additional LAIR1 and TOX2 immunohistochemistry was performed in ulcerative colitis to analyze the immune microenvironment; Results: The trained network classified the 3 diagnoses with high performance. The model classified ulcerative colitis with an accuracy of 99.1%, precision of 97.1%, recall of 94.8%, an F1-score of 95.9%, and specificity of 99.6%. For colorectal cancer, the performance was as follows: accuracy, 99.8%; precision, 99.9%; recall, 99.8%; F1-score 99.9%; and specificity, 99.7%. The Grad-CAM heatmap confirmed which parts of the image were most important for classification. The CNN also managed to differentiate between steroid-requiring (SR) and nonsteroid requiring (non-SR) ulcerative colitis based on H&E, LAIR1 and TOX2 staining; Conclusions: Convolutional neural networks are essential tools for deep learning and are especially suited for image recognition in conditions such as ulcerative colitis and colorectal cancer; LAIR1 and TOX2 are promising immuno-oncology markers in ulcerative colitis.

Low learning rates in Convolutional Neural Networks (CNNs) for image segmentation tasks can lead to convergence issues, unstable models, large oscillations, risk of divergence, sensitivity to the initial weights, and biases of the model. All these challenges make CNNs computationally expensive and require considerable training data. In contrast, achieving a high learning rate has been a significant challenge for CNNs. In response to the abovementioned challenge, this work aims to increase the learning rate of the image segmentation process. Images of any object in the universe taken from any camera can broadly be divided into spectral and geometric properties. Both these properties are prominently visible in satellite images. Therefore, if a method can classify satellite images, it can be applied to classify any image using the same technique. In this paper, Evolutionary Convergent Functions (ECF) are proposed. They convert features' spectral and geometric properties in a satellite image into mathematical equations using a decision tree and then converge with a neural network. Different high-resolution data have been chosen to extract additional features from it. This transformation process, anchored in decision tree methodology, converges with neural networks to yield unparalleled results, all while eliminating the need for computationally intensive convolutions. The proposed method extends beyond conventional boundaries by using varied high-resolution datasets. Each dataset is carefully selected with distinctive features. This research explores the untapped potential of ECF to advance the field of automated image classification, aiming to broaden the scope of current methodologies. The synergy between spectral and geometric properties emerges as a powerful combination, endowing our methods with the ability to extract nuanced and context-rich information for image analysis. The results show a high accuracy, i.e., above 90%, in almost all objects of different shapes and spectral signatures. Additionally, in terms of prediction time, ECF is faster than U-Net, a state-of-the-art method that is evidence of ECF efficiency and speed.

Synthetic Aperture Radar has been extensively used in numerous fields and can gather a wealth of information about the area of interest. This large-scene data-intensive technology puts a high value on automatic target recognition (ATR) which can free the utilizers and boost the efficiency. Recent advances in artificial intelligence have made it possible to create a deep learning-based SAR ATR that can automatically identify target features from massive input data. In the last 6 years, intensive research has been conducted in this area, however, most papers in the current SAR ATR field used recurrent neural network (RNN) and convolutional neural network (CNN)-varied models to deepen the regime’s understanding of the SAR images. To equip SAR ATR with updated deep learning technology, this paper tries to apply a lightweight vision transformer (LViT)-based model to classify SAR images. The entire structure was verified by an open- accessed SAR data set and recognition results show that the final classification outcomes are robust and more accurate in comparison with referred traditional network structures without even using any convolutional layers.

Simultaneous Localization and Mapping (SLAM) is a crucial function for most autonomous systems, allowing them to both navigate through and create maps of unfamiliar surroundings. Traditional Visual SLAM, also commonly known as VSLAM, relies on frame-based cameras and structured processing pipelines, which face challenges in dynamic or low-light environments. However, recent advancements in event camera technology and neuromorphic processing offer promising opportunities to overcome these limitations. Event cameras inspired by biological vision systems capture the scenes asynchronously consuming minimal power but with higher temporal resolution. Neuromorphic processors, which are designed to mimic the parallel processing capabilities of the human brain, offer efficient computation for real-time data processing of event-based data streams. This paper provides a comprehensive overview of recent research efforts in integrating event cameras and neuromorphic processors into VSLAM systems. It discusses the principles behind event cameras and neuromorphic processors, highlighting their advantages over traditional sensing and processing methods. Furthermore, an in-depth survey was conducted on state-of-the-art approaches in event-based SLAM, including feature extraction, motion estimation, and map reconstruction techniques. Additionally, the integration of event cameras with neuromorphic processors, focusing on their synergistic benefits in terms of energy efficiency, robustness, and real-time performance was explored. The paper also discusses the challenges and open research questions in this emerging field, such as sensor calibration, data fusion, and algorithmic development. Finally, the potential applications and future directions for event-based SLAM systems are outlined, ranging from robotics and autonomous vehicles to augmented reality.

Few prior works study self-driving cars by deep learning with IoT collaboration. SDC-Net, which is an end-to-end multitask self-driving car camera cocoon IoT-based system, is one of the research that tackles this direction. However, by design SDC-Net is not able to identify the accident locations, it only classifies if it is a crash scene or not. In this work, we introduce an enhanced design for the SDC-Net system by 1) replacing the classification network with a detection one, 2) adapting our benchmark dataset labels built on the CARLA simulator to include the vehicles bounding boxes while keeping the same training, validation, and testing samples, 3) modifying the shared information via IoT to include the accident location. We keep the same path planning and automatic emergency braking network, the digital automation platform, and the input representations to formulate the comparative study. SDC-Net++ system is proposed to 1) output the relevant control actions, especially in case of accidents: accelerate, decelerate, maneuver, and brake, and 2) share the most critical information to the connected vehicles via IoT, especially the accident locations. A comparative study is also conducted between SDC-Net and SDC-Net++ with the same input representations: front camera only, panorama and bird-eye views, and with single-task networks: crash avoidance only, and multitask networks. Multitask network with a BEV input representation outperforms the nearest representation in precision, recall, f1-score, and accuracy by more than 15.134%, 12.046%, 13.593%, and 5%, respectively. SDC-Net++ multitask network with BEV outperforms SDC-Net multitask with BEV in precision, recall, f1-score, accuracy, and average MSE by more than 2.201%, 2.8%, 2.505%, 2%, and 18.677% respectively.

A single paragraph of about 200 words maximum. For research articles, abstracts should give a pertinent overview of the work. We strongly encourage authors to use the following style of structured abstracts, but without headings: (1) Background: Place the question addressed in a broad context and highlight the purpose of the study; (2) Methods: briefly describe the main methods or treatments applied; (3) Results: summarize the article’s main findings; (4) Conclusions: indicate the main conclusions or interpretations. The abstract should be an objective representation of the article and it must not contain results that are not presented and substantiated in the main text and should not exaggerate the main conclusions.

Deep learning heavily relies on statistical correlations to drive artificial intelligence (AI) innovations, particularly in computer vision applications like autonomous driving and robotics. However, despite providing a solid foundation for deep learning, these statistical correlations can be vulnerable to unforeseen and uncontrolled factors. The lack of prior knowledge guidance can result in spurious correlations, introducing confounding factors and affecting the model's robustness. To address this challenge, recent research efforts have focused on integrating causal theory into deep learning methodologies. By modelling the inherent and unbiased causal structure, causal theory can potentially mitigate the impact of spurious correlations effectively. Hence, this paper explores the basics of causal methodologies in image classification.

Document deskewing is a fundamental problem in document image processing. While existing methods have limitations, such as Hough Line Transformation that can deskew images upside down, and Deep Learning models that require huge amounts of human labour and computational resources and still fail to deskew while taking care of orientation, OCR-based methods also struggle to read text when it is tilted. In this paper, we propose a novel, simple, cost-effective deep learning method for fixing the skew and orientation of documents. Our approach reduces the search space for the machine learning model to predict whether an image is upside down or not, avoiding the huge search space of predicting an angle between 0 and 360. We finetuned a MobileNetV2 model, which was pre-trained on imagenet, using only 200 images and achieve good results. This method is useful for automation-based tasks, such as data extraction using OCR technology, and can greatly reduce manual labour.

A machine learning method for classifying Lung UltraSound is here proposed to pro- vide a point of care tool for supporting a safe, fast and accurate diagnosis, that can also be useful during a pandemic like as SARS-CoV-2. Given the advantages (e.g. safety, rapidity, portability, cost-effectiveness) provided by the ultrasound technology over other methods (e.g. X-ray, computer tomography, magnetic resonance imaging), our method was validated on the largest LUS public dataset. Focusing on both accuracy and efficiency, our solution is based on an efficient adaptive ensembling of two EfficientNet-b0 models reaching 100% of accuracy, which, to our knowledge, outperforms the previous state-of-the-art. The complexity of this solution keeps the number of parameters in the same order as an EfficientNet-b0 by adopting specific design choices that are adaptive ensembling with a combination layer, ensembling performed on the deep features, minimal ensemble only two weak models. Moreover, a visual analysis of the saliency maps on sample images of all the classes of the dataset reveals where the focus is on an inaccurate weak model versus an accurate model.

Imaging systems mounted to ground vehicles are used to image fruit tree canopies for estimation of fruit load, but frequently need correction for fruit occluded by branches, foliage or other fruits. This can be achieved using an orchard ‘occlusion factor’, estimated from a manual count of fruit load on a sample of trees (referred to as the reference method). It was hypothesised that canopy images could hold information related to the number of occluded fruit. Five approaches to correct for occluded fruit based on canopy images were compared using data of three mango orchards in two seasons. However, no attribute correlates to the number of hidden fruit were identified. Several image features obtained through segmentation of fruit and canopy areas, such as the proportion of fruit that were partly occluded, were used in training Random forest and multi-layered perceptron (MLP) models for estimation of a correction factor per tree. In another approach, deep learning convolutional neural networks (CNNs) were directly trained against harvest fruit count on trees. The supervised machine learning methods for direct estimation of fruit load per tree delivered an improved prediction outcome over the reference method for data of the season/orchard from which training data was acquired. For a set of 2017 season tree images (n=98 trees), a R2 of 0.98 was achieved for the correlation of the number of fruits predicted by a Random forest model and the ground truth fruit count on the trees, compared to a R2 of 0.68 for the reference method. The best prediction of whole orchard (n = 880 trees) fruit load, in the season of the training data, was achieved by the MLP model, with an error to packhouse count of 1.6% compared to the reference method error of 13.6%. However, the performance of these models on new season data (test set images) was at best equivalent and generally poorer than the reference method. This result indicates that training on one season of data was insufficient for the development of a robust model. This outcome was attributed to variability in tree architecture and foliage density between seasons and between orchards, such that the characters of the canopy visible from the interrow that relate to the proportion of hidden fruit are not consistent. Training of these models across several seasons and orchards is recommended.

AI is leveraging all aspects of life. Medical services are not untouched. Especially in the field of medical image processing and diagnosis. Big IT and Biotechnology companies are investing millions of dollars in medical and AI research. The recent outbreak of SARS COV-2 gave us a unique opportunity to study for a non interventional and sustainable AI solution. Lung disease remains a major healthcare challenge with high morbidity and mortality worldwide. The predominant lung disease was lung cancer. Until recently, the world has witnessed the global pandemic of COVID19, the Novel coronavirus outbreak. We have experienced how viral infection of lung and heart claimed thousands of lives worldwide. With the unprecedented advancement of Artificial Intelligence in recent years, Machine learning can be used to easily detect and classify medical imagery. It is much faster and most of the time more accurate than human radiologists. Once implemented, it is more cost-effective and time-saving. In our study, we evaluated the efficacy of Microsoft Cognitive Service to detect and classify COVID19 induced pneumonia from other Viral/Bacterial pneumonia based on X-Ray and CT images. We wanted to assess the implication and accuracy of the Automated ML-based Rapid Application Development (RAD) environment in the field of Medical Image diagnosis. This study will better equip us to respond with an ML-based diagnostic Decision Support System(DSS) for a Pandemic situation like COVID19. After optimization, the trained network achieved 96.8% Average Precision which was implemented as a Web Application for consumption. However, the same trained network did not perform like Web Application when ported to Smartphone for Real-time inference, which was our main interest of study. The authors believe, there is scope for further study on this issue. One of the main goals of this study was to develop and evaluate the performance of AI-powered Smartphone-based Real-time Applications. Facilitating primary diagnostic services in less equipped and understaffed rural healthcare centers of the world with unreliable internet service.

This paper presents the development of a hardware-in-the-loop ground testbed featuring active gravity compensation via software-in-the-loop integration, specially designed to support research in autonomous robotic removal of space debris. The testbed is designed to replicate six degrees of freedom (6DOF) motion maneuvering to accurately simulate the dynamic behaviors of free-floating robotic manipulators and free-tumbling space debris under microgravity conditions. The testbed incorporates two industrial 6DOF robotic manipulators, a 3-finger robotic gripper, and a suite of sensors, including cameras, force/torque sensors, and tactile tensors. Such a setup provides a robust platform for testing and validating technologies related to autonomous tracking, capture, and post-capture stabilization within the context of active space debris removal missions. Preliminary experimental results have demonstrated advancements in motion control, computer vision, and sensor fusion. This facility is positioned to become an essential resource for the development and validation of robotic manipulators in space, offering substantial improvements in the effectiveness and reliability of autonomous capture operations in space missions.

Celiac disease is a gluten-sensitive immune-mediated enteropathy of the small intestine that occurs in genetically predisposed individuals. Abnormal immune response results in mucosal inflammation, villous atrophy, and crypt hyperplasia. This study was a proof-of-concept exercise that used a convolutional neural network to classify hematoxylin and eosin (H&E) histological images of celiac disease, normal small intestine control, and non-specified duodenal inflammation; 7294, 11,642, and 5966 images, respectively. The trained network classified celiac disease images with high performance (accuracy 99.7%, precision 99.6%, recall 99.3%, F1-score 99.5%, and specificity 99.8%). Interestingly, when the same network (already trained for the 3 class images), analyzed duodenal adenocarcinoma (3723 images), the new images were classified as duodenal inflammation in 63.65%, small intestine control in 34.73%, and celiac disease in 1.61% of the cases. Finally, when the network was retrained using the 4 histological subtypes of images, all performance parameters were above 99% for celiac disease. In conclusion, the convolutional neural network (CNN)-based deep neural system was able to classify medical histological images with high performance. Narrow artificial intelligence (AI) is designed to perform tasks that typically require human intelligence, but it operates within limited constraints and is task specific.

The paper delineates the developed vision system of the UR5 cobot and the operating algorithm of the robotic quality control station. The hardware-software architecture of the developed robotic station was shown, consisting of a UR5 cobot equipped with a web camera and a stationary industrial camera with a lighting system. Image processing and analysis algorithms were described, the method of control and communication between the station components was discussed, and two operating scenarios were presented as a single robotic station and a robotic line. Based on the results obtained, the level of measurement noise, accuracy, and repeatability of the developed vision system were estimated.

Computer vision models of salient object detection attempt to mimic the ability of the human visual system to select relevant objects in images. To this end, the development of deep neural networks on high-end computers has recently made it possible to achieve high performance. However, it remains a challenge to develop deep neural network models of the same performance for devices with much more limited resources. In this work, we propose a new approach for a lightweight salient object detection neural network model, inspired by the cone and spatial opponent processes of the primary visual cortex (V1), that inextricably link color and shape in human color perception. Our proposed model, namely CoSOV1net, is trained from scratch, without using backbones from image classification or other tasks. Experiments, on the most widely used and challenging datasets for salient object detection, show that CoSOV1Net achieves competitive performance (i.e. Fβ=0.931 on the ECSSD dataset) with state-of-the-art salient object detection models, while having low number of parameters (1.14M), low FLOPS (1.4G) and high FPS (211.2) on GPU (nvidia Geforce RTX 3090 TI) compared to the state-of-the-art in the salient object detection or lightweight salient object detection task. Thus, CoSOV1net turns out to be a lightweight salient object detection that can be adapted to mobile environments and resource-constrained devices.

Hand gestures may play an important role in medical applications for health care of elderly people, where providing a natural interaction for different requests can be executed by making specific gestures. In this study we explored three different scenarios using a Microsoft Kinect V2 depth sensor then evaluated the effectiveness of the outcomes. The first scenario utilized the default system embedded in the Kinect V2 sensor, which depth metadata gives 11 parameters related to the tracked body with five gestures for each hand. The second scenario used joint tracking provided by Kinect depth metadata and depth threshold together to enhance hand segmentation and efficiently recognize the number of fingers extended. The third scenario used a simple convolutional neural network with joint tracking by depth metadata to recognize five categories of gestures. In this study, deaf-mute elderly people execute five different hand gestures to indicate a specific request, such as needing water, meal, toilet, help and medicine. Then, the requests were sent to the care provider’s smartphone because elderly people could not execute any activity independently. The system transferred these requests as a message through the global system for mobile communication (GSM) using a microcontroller.

The study addresses the challenge of efficiently peeling pineapples, which have a distinct elliptical form, thick skin, and small eyes that are difficult to detect with conventional automated methods. This results in significant flesh waste. To improve the process, the researchers developed an integrated system combining an enhanced BlendMask method, termed SAAF-BlendMask, and a PCP method. SAAF-BlendMask improves the detection of small pineapple eyes, while PCP ensures accurate posture adjustment for precise path planning. The system uses 3D vision and deep learning technologies, achieving an average precision (AP) of 73.04% and a small object precision (APs) of 62.54% in eye detection, with a path planning success rate reaching 99%. The fully automated electromechanical system was tested on 110 real pineapples, demonstrating a reduction in flesh waste by 11.7% compared to traditional methods. This study highlights the potential of advanced machine vision and robotics in enhancing the efficiency and precision of food processing.

The ever-evolving ways attacker continues to im prove their phishing techniques to bypass existing state-of-the-art phishing detection methods pose a mountain of challenges to researchers in both industry and academia research due to the inability of current approaches to detect complex phishing attack. Thus, current anti-phishing methods remain vulnerable to complex phishing because of the increasingly sophistication tactics adopted by attacker coupled with the rate at which new tactics are being developed to evade detection. In this research, we proposed an adaptable framework that combines Deep learning and Randon Forest to read images, synthesize speech from deep-fake videos, and natural language processing at various predictions layered to significantly increase the performance of machine learning models for phishing attack detection.

Amidst the burgeoning global concerns surrounding skin cancer, the quest for precise and efficient diagnostic methodologies has intensified. Conventional diagnostic approaches relying on subjective visual inspection by dermatologists often grapple with subjectivity and resource constraints. This predicament underscores the critical role of Artificial Intelligence (AI) in revolutionizing diagnostic paradigms by significantly enhancing accuracy and accessibility. This study embarks on an extensive exploration of skin cancer classification, specifically targeting the intricate task of 8-class skin cancer classification. Employing cutting-edge deep learning models on the ISIC2019 dataset, a comprehensive analysis was conducted utilizing a diverse array of pre-trained ImageNet architectures and Vision Transformer models. To counteract the inherent class imbalance in skin cancer datasets, a pioneering "Naturalize" augmentation technique was introduced, leading to the creation of indispensable tools—the Naturalized 2.4K ISIC2019 and groundbreaking Naturalized 7.2K ISIC2019 datasets—catalyzing advancements in classification accuracy. The pivotal role of AI in mitigating the risks of misdiagnosis and under-diagnosis in skin cancer becomes evident through this research. AI’s proficiency in analyzing vast datasets and discerning subtle patterns augments the diagnostic prowess of dermatologists significantly. Integration of AI-powered systems in skin cancer diagnosis holds immense promise in facilitating early detection and substantially improving patient outcomes. The meticulous evaluation employed quantitative measures such as confusion matrices, classification reports, and visual analyses using Score-CAM across diverse dataset variations. Astonishingly, the culmination of these endeavors culminated in an unprecedented achievement—100% average accuracy, precision, recall, and F1-Score—within the groundbreaking Naturalized 7.2K ISIC2019 dataset. This remarkable feat underscores the transformative potential of AI-driven methodologies, exemplifying their prowess in redefining the landscape of skin cancer diagnosis and management. In summary, this research represents a pivotal stride towards redefining dermatological diagnosis, showcasing the remarkable impact of AI-powered solutions in surmounting the challenges inherent in skin cancer diagnosis. The attainment of 100% across crucial metrics within the Naturalized 7.2K ISIC2019 dataset serves as a testament to the transformative capabilities of AI-driven approaches in reshaping the trajectory of skin cancer diagnosis and patient care.

In recent years, the analysis of macro- and micro-expression has drawn the attention of researchers since they provide visual cues of an individual's emotions for a broad range of potential applications such as lie detection and criminal detection. In this paper, we address the challenge of spotting facial macro- and micro-expression from videos and present compelling results by using a deep learning approach to analyze the optical flow features. Different from other deep learning approaches that are mainly based on Convolutional Neural Networks (CNNs), we propose a Transformer-based deep learning approach that predicts a score indicating the probability of a frame being within an expression interval. In contrast to other Transformer-based models that achieve high performance by being pre-trained on large datasets, our deep learning model, called SL-Swin, which incorporates Shifted Patch Tokenization and Locality Self-Attention into the backbone network Swin Transformer, effectively spots macro- and micro-expressions by being trained from scratch on small-size expression datasets. Our evaluation outcomes surpass the MEGC 2022 spotting baseline result, obtaining an overall F1-score of 0.1366. Additionally, our approach also performs well in the MEGC 2021 spotting task with an overall F1-score of 0.1824 and 0.1357 on CAS(ME)^2 and SAMM Long Videos, respectively. The code is publicly available on GitHub (https://github.com/eddiehe99/pytorch-expression-spotting).

Recent progress in deep learning has led to accurate and efficient generic object detection networks. Training of highly reliable models depends on large datasets with highly textured and rich images. However, in real-world scenarios, the performance of the generic object detection system decreases when (i) occlusions hide the objects, (ii) objects are present in low-light images, or (iii) they are merged with background information. In this paper, we refer to all these situations as challenging environments. With the recent rapid development in generic object detection algorithms, notable progress has been observed in the field of object detection in challenging environments. However, there is no consolidated reference to cover state-of-the-art in this domain. To the best of our knowledge, this paper presents the first comprehensive overview, covering recent approaches that have tackled the problem of object detection in challenging environments. Furthermore, we present the quantitative and qualitative performance analysis of these approaches and discuss the currently available challenging datasets. Moreover, this paper investigates the performance of current state-of-the-art generic object detection algorithms by benchmarking results on the three well-known challenging datasets. Finally, we highlight several current shortcomings and outline future directions.

In recent years, the demand for efficient and accurate measurement techniques has surged across various industries, driving the development of non-invasive methods that leverage computer vision technology. This paper explores the principles and applications of non-invasive measurement techniques utilizing computer vision, emphasizing their ability to deliver precise measurements without the need for physical contact or specialized sensors. We delve into the underlying algorithms, including image processing, machine learning, and deep learning methods, that enable the extraction of dimensional data from visual inputs. Case studies across diverse fields such as manufacturing, healthcare, and logistics illustrate the effectiveness of these techniques in enhancing operational efficiency and reducing measurement errors. Furthermore, we discuss the challenges and limitations of implementing computer vision-based measurement systems, including the need for robust algorithms capable of functioning in varied environmental conditions. By highlighting the transformative potential of non-invasive measurement techniques, this paper aims to provide insights into future developments in computer vision and its role in advancing measurement practices across multiple sectors.

Loss of photoreceptors in retinal degenerative diseases also impacts the inner retina: bipolar cell dendrites retract, neurons rewire and protein expression changes. ON-bipolar cells represent an attractive target for optogenetic vision restoration. However, above-described maladaptations may negatively impact the quality of restored vision. To investigate this question, we employed human post-mortem retinas and transgenic rd1_Opto-mGluR6 mice expressing the optogenetic construct Opto-mGluR6 in ON-bipolar cells and carrying the retinal degeneration rd1 mutation. We found significant changes in delayed rectifier potassium channel expression in ON-bipolar cells of degenerative retinas. In particular, we found an increase in Kv1.3 expression already in early stages of degeneration. Immunohistochemistry localized Kv1.3 channels specifically to OBC axons. In whole-cell patch-clamp experiments, ON-bipolar cells in the degenerated murine retina were less responsive, which could be reversed by application of the specific Kv1.3 antagonist Psora-4. Notably, Kv1.3 block significantly increased the amplitude and kinetics of Opto-mGluR6-mediated light responses in ON-bipolar cells of the blind retina and increased the signal-to-noise ratio of light-triggered responses in retinal ganglion cells. We propose that reduction of Kv1.3 activity in the degenerated retina, either by pharmacological block or by KCNA3 gene silencing, could improve the quality of restored vision.

Oil spillage over a sea or ocean’s surface is a threat to marine and coastal ecosystems. Spaceborne synthetic aperture radar (SAR) data has been used efficiently for the detection of oil spills due to its operational capability in all-day all-weather conditions. The problem is often modeled as a semantic segmentation task. The images need to be segmented into multiple regions of interest such as sea surface, oil spill, look-alikes, ships and land. Training of a classifier for this task is particularly challenging since there is an inherent class imbalance. In this work, we train a convolutional neural network (CNN) with multiple feature extractors for pixel-wise classification; and introduce to use a new loss function, namely ‘gradient profile’ (GP) loss, which is in fact the constituent of the more generic Spatial Profile loss proposed for image translation problems. For the purpose of training, testing and performance evaluation, we use a publicly available dataset with selected oil spill events verified by the European Maritime Safety Agency (EMSA). The results obtained show that the proposed CNN trained with a combination of GP, Jaccard and focal loss functions can detect oil spills with an intersection over union (IoU) value of 63.95%. The IoU value for sea surface, look-alikes, ships and land class is 96.00%, 60.87%, 74.61% and 96.80%, respectively. The mean intersection over union (mIoU) value for all the classes is 78.45%, which accounts for a 13% improvement over the state of the art for this dataset. Moreover, we provide extensive ablation on different Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) based hybrid models to demonstrate the effectiveness of adding GP loss as an additional loss function for training. Results show that GP loss significantly improves the mIoU and F1 scores for CNNs as well as ViTs based hybrid models. GP loss turns out to be a promising loss function in the context of deep learning with SAR images.

Dams are a critical infrastructure system for many communities, but they are also one of the most challenging to inspect. Dams are typically very large and complex structures, and the result is that inspections are often time-intensive and require expensive, specialized equipment and training to provide inspectors with comprehensive access to the structure. The scale and nature of dam inspections also introduces additional safety risks to the inspectors. Unmanned aerial vehicles (UAV) have the potential to address many of these challenges, particularly when used as a data acquisition platform for photogrammetric three-dimensional (3D) reconstruction and analysis, though the nature of both UAV and modern photogrammetric methods necessitates careful planning and coordination for integration. This paper presents a case study on one such integration at the Brighton Dam, a large-scale concrete gravity dam in Maryland, USA. A combination of multiple UAV platforms and multi-scale photogrammetry was used to create two comprehensive and high-resolution 3D point clouds of the dam and surrounding environment at intervals. These models were then assessed for their overall quality, as well as their ability to resolve flaws and defects that were artificially applied to the structure between inspection intervals. The results indicate that the integrated process is capable of generating models that accurately render a variety of defect types with sub-millimeter accuracy. Recommendations for mission planning and imaging specifications are provided as well.

Recently, the destructive impact of wildfires has proliferated; for instance, the August Complex wildfire in 2020 burned around 4% of California's landmass. This has caused increased economic damage and risk to human life. Additionally, climate change is anticipated to increase the severity of wildfires, making it imperative for accurate and efficient detection of wildfires. Machine Learning approaches allow for the automatic detection of wildfires, simultaneously prioritizing accuracy and efficiency, with minimal human intervention, thus decreasing the likelihood of increased economic damage and increasing firefighting responses. Convolutional Neural Networks (CNNs), while showing promise, are often limited by their inability to learn and capture deep spatial dependencies in satellite imagery tasks. In this paper, we propose FlameViT, a novel wildfire detection architecture based on Vision Transformers (ViT). Satellite images are more efficient to obtain and can cover wide areas prone to wildfires. We obtain a dataset of 40K+ satellite images from Canada's Open Government Portal, allowing FlameViT to be optimized to detect wildfires in satellite imagery. FlameViT uses Patch Extractor and Patch Embedding layers, followed by multiple Transformer Encoder layers with Multi-Head Self-Attention and feed-forward neural networks. FlameViT is hyperparameter-tuned and achieves a validation accuracy of 95%, outperforming various baselines in the wildfire detection task. FlameViT shows the power of Vision Transformers for wildfire detection tasks, and in conjunction with the use of satellite imagery, can provide an efficient and accurate way of detecting wildfires.

As businesses increasingly adopt advanced technologies to enhance productivity and safety, the integration of human-robot collaboration (HRC) emerges as a transformative approach. This paper explores the role of GPU-accelerated computer vision and generative AI in optimizing HRC within business environments. By leveraging the computational power of GPUs, real-time data processing, and advanced machine learning algorithms, robots can better interpret complex visual cues and adapt to dynamic workspaces. Generative AI further enables the design of intelligent robotic systems capable of anticipating human actions, optimizing task allocation, and ensuring safety through predictive modeling. This research highlights the potential of these technologies to improve operational efficiency, reduce human error, and create safer, more adaptable workplaces. The implications of such advancements are discussed, providing insights into the future of HRC in various industries.

This paper presents a methodology to detect the erosion and progradation effects in the Atrato river delta, located in the Gulf of Urabá in Colombia using SAR (Synthetic Aperture Radar) data. Other authors have studied this area by using optical images, encountering difficulties given the persistent presence of clouds in this region. SAR imagery is rescaled, despeckled, composed, classified, registered and processed to obtain a heatmap that shows the overall changes during eight years (2016–2023) in one single image. Since this technology uses microwave active signals, the presence of clouds and other environmental phenomena was not a problem, and all the images from the selected dates could be processed and analyzed. Also, detection of the dynamic processes was obtained for every period considered in this study with respect to the previous one in time, calculating areas and detecting the regions subject to the erosion and progradation processes. The multitemporal analysis performed found that the newly created mouth is the most active area for these processes, coinciding with other studies. A comparison of these findings with the Oceanic Niño Index (ONI) showed a relative delayed coupling to the erosion process and a coupling of progradation with dry and wet seasons.

Background: Processing whole-slide images (WSI) to train neural networks can be intricate and laborious. We developed an open-source library covering recurrent tasks in processing of WSI and in evaluating the performance of the trained networks for classification tasks. Methods: Two histopathology use-cases were selected. First we aimed to train a CNN to distinguish H&E-stained slides obtained from neuropathologically classified low-grade epilepsy-associated dysembryoplastic neuroepithelial tumor (DNET) and ganglioglioma (GG). The second project we trained a convolutional neural network (CNN) to predict the hormone expression of pituitary adenoms only from hematoxylin and eosin (H&E) stained slides. In the same approach, we addressed the issue to also predict clinically silent corticotroph adenoma. We included four clinico-pathological disease conditions in a multilabel approach. Results: Our best performing CNN achieved an area under the curve (AUC) of 0.97 for the receiver operating characteristic (ROC) for corticotroph adenoma, 0.86 for silent corticotroph adenoma and 0.98 for gonadotroph adenoma. Our DNET-GG classifier achieved an AUC of 1.00 for the ROC curve. All scores were calculated with the help of our library on predictions on a case basis. Conclusions: Our comprehensive library is most helpful to standardize the work-flow and minimize the work-burden in training CNN. It is also compatible with fastai. Indeed, our new CNNs reliably extracted neuropathologically relevant information from the H&E staining only. This approach will supplement the clinico-pathological diagnosis of brain tumors, which is currently based on cost-intense microscopic examination and variable panels of immunohistochemical stainings.

Table detection is a preliminary step in extracting reliable information from tables in scanned document images. We present CasTabDetectoRS, a novel end-to-end trainable table detection framework that operates on Cascade Mask R-CNN, including Recursive Feature Pyramid network and Switchable Atrous Convolution in the existing backbone architecture. By utilizing a comparatively lightweight backbone of ResNet-50, this paper demonstrates that superior results are attainable without relying on pre and post-processing methods, heavier backbone networks (ResNet-101, ResNeXt-152), and memory-intensive deformable convolutions. We evaluate the proposed approach on five different publicly available table detection datasets. Our CasTabDetectoRS outperforms the previous state-of-the-art results on four datasets (ICDAR-19, TableBank, UNLV, and Marmot) and accomplishes comparable results on ICDAR-17 POD. Upon comparing with previous state-of-the-art results, we obtain a significant relative error reduction of 56.36%, 20%, 4.5%, and 3.5% on the datasets of ICDAR-19, TableBank, UNLV, and Marmot, respectively. Furthermore, this paper sets a new benchmark by performing exhaustive cross-datasets evaluations to exhibit the generalization capabilities of the proposed method.

Tables in the document image are one of the most important entities since they contain crucial information. Therefore, accurate table detection can significantly improve information extraction from tables. In this work, we present a novel end-to-end trainable pipeline, HybridTabNet, for table detection in scanned document images. Our two-stage table detector uses the ResNeXt-101 backbone for feature extraction and Hybrid Task Cascade (HTC) to localize the tables in scanned document images. Moreover, we replace conventional convolutions with deformable convolutions in the backbone network. This enables our network to detect tables of arbitrary layouts precisely. We evaluate our approach comprehensively on ICDAR-13, ICDAR-17 POD, ICDAR-19, TableBank, Marmot, and UNLV. Apart from the ICDAR-17 POD dataset, our proposed HybridTabNet outperforms earlier state-of-the-art results without depending on pre and post-processing steps. Furthermore, to investigate how the proposed method generalizes unseen data, we conduct an exhaustive leave-one-out-evaluation. In comparison to prior state-of-the-art results, our method reduces the relative error by 27.57% on ICDAR-2019-TrackA-Modern, 42.64% on TableBank (Latex), 41.33% on TableBank (Word), 55.73% on TableBank (Latex + Word), 10% on Marmot, and 9.67% on UNLV dataset. The achieved results reflect the superior performance of the proposed method.

Visual Question Generation (VQG) from images is a rising research topic in both fields of natural language processing and computer vision. Although there are some recent efforts towards generating questions from images in the open domain, the VQG task in the medical domain has not been well-studied so far due to the lack of labeled data. In this paper, we introduce a goal-driven VQG approach for radiology images called VQGRaD that generates questions targeting specific image aspects such as modality and abnormality. In particular, we study generating natural language questions based on the visual content of the image and on additional information such as the image caption and the question category. VQGRaD encodes the dense vectors of different inputs into two latent spaces, which allows generating, for a specific question category, relevant questions about the images, with or without their captions. We also explore the impact of domain knowledge incorporation (e.g., medical entities and semantic types) and data augmentation techniques on visual question generation in the medical domain. Experiments performed on the VQA-RAD dataset of clinical visual questions showed that VQGRaD achieves 61.86% BLEU score and outperforms strong baselines. We also performed a blinded human evaluation of the grammaticality, fluency, and relevance of the generated questions. The human evaluation demonstrated the better quality of VQGRaD outputs and showed that incorporating medical entities improves the quality of the generated questions. Using the test data and evaluation process of the ImageCLEF 2020 VQA-Med challenge, we found that relying on the proposed data augmentation technique to generate new training samples by applying different kinds of transformations, can mitigate the lack of data, avoid overfitting, and bring a substantial improvement in medical VQG.