Vision-language-action (VLA) models represent a promising direction for developing general-purpose robotic systems, demonstrating the ability to combine visual understanding, language comprehension, and action generation. However, systematic evaluation of these models across diverse robotic tasks remains limited. In this work, we present a comprehensive evaluation framework and benchmark suite for assessing VLA models. We profile three state-of-the-art VLM and VLAs —GPT-4o, OpenVLA, and JAT—across 20 diverse datasets from the Open-X-Embodiment collection, evaluating their performance on various manipulation tasks. Our analysis reveals several key insights: (1) current VLA models show significant variation in performance across different tasks and robot platforms, with GPT-4o demonstrating the most consistent performance through sophisticated prompt engineering, (2) all models struggle with complex manipulation tasks requiring multi-step planning, and (3) model performance is notably sensitive to action space characteristics and environmental factors. We release our evaluation framework and findings to facilitate systematic assessment of future VLA models and identify critical areas for improvement in the development of general-purpose robotic systems.

In Mexico, the incorporation of deaf people into education has been lacking since only 14% of the deaf population in the age group between 3 and 29 years access education with the support of a hearing aid. Additionally, those who have been incorporated frequently face inappropriate educational strategies which poorly develop the use of Mexican Sign Language (MSL) and therefore academical success and opportunities for insertion in the workplace are difficult. This research explores a novel mexican sign language lexicon video dataset containing the dynamical gestures most frequently used by MSL. Each gesture consists of a set of different versions of videos under uncontrolled conditions. MX-ITESO-100 data set is composed of a lexicon of 100 gestures and 5,000 videos from three participants with different grammatical elements. Additionally, the data set is evaluated in a two-step neural network model with an accuracy greater than 99%. and thus serves as a benchmark for future training of machine learning models in computer vision systems. Finally, this research provides an inclusive environment within society and organizations in particular for people with hearing impairment.

The integration of sign language recognition systems into digital platforms has the potential to bridge communication gaps between the deaf community and the broader population. This paper introduces an advanced Israeli Sign Language (ISL) recognition system designed to interpret dynamic motion gestures, addressing a critical need for more sophisticated and fluid communication tools. Unlike conventional systems that focus solely on static signs, our approach incorporates both Deep Learning and Computer Vision techniques to analyze and translate dynamic gestures captured in real-time video. We provide a comprehensive account of our preprocessing pipeline, detailing every stage from video collection to the extraction of landmarks using MediaPipe, including the mathematical equations used for preprocessing these landmarks and the final recognition process. The dataset utilized for training our model is unique in its comprehensiveness and is publicly accessible, enhancing the reproducibility and expansion of future research. The deployment of our model on a publicly accessible website allows users to engage with ISL interactively, facilitating both learning and practice. We discuss the development process, the challenges overcome, and the anticipated societal impact of our system in promoting greater inclusivity and understanding.

Accurate crop diseases classification is crucial for ensuring food security and enhancing agricultural productivity. However, existing crop disease classification algorithms primarily focus on a single image modality and typically require a large number of samples. Our research counters these issues by using pre-trained Vision-Language Models (VLMs), which enhances multimodal synergy for better crop disease classification than traditional unimodal approaches. Firstly, we apply the multimodal model Qwen-VL to generate meticulous textual descriptions for representative disease images selected through clustering from the training set, which will serve as prompt text for generating classifier weights. Compared to using solely the language model for prompt text generation, this approach better captures and conveys fine-grained and image-specific information, thereby enhancing prompt quality. Secondly, we integrate the cross-attention and the SE (Squeeze-and-Excitation) attention into the training-free mode VLCD (Vision-Language model for Crop Diseases classification) and the training-required mode VLCD-T (VLCD-Training) respectively for prompt text processing, enhancing classifier weights by emphasizing key text features. Experimental outcomes conclusively prove our method’s heightened classification effectiveness in few-shot crop disease scenarios, tackling data limitations and intricate disease recognition issues. It offers a pragmatic tool for agricultural pathology and reinforces the smart farming surveillance infrastructure.

Recent improvements in deepfake creation have made deepfake videos more realistic. Moreover, open-source software has made deepfake creation more accessible, which reduces the barrier to entry for deepfake creation. This could pose a threat to the people’s privacy. There is a potential danger if the deepfake creation techniques are used by people with an ulterior motive to produce deepfake videos of world leaders to disrupt the order of countries and the world. Therefore, research into the automatic detection of deepfaked media is essential for public security. In this work, we propose a deepfake detection method using upper body language analysis. Specifically, a many-to-one LSTM network was designed and trained as a classification model for deepfake detection. Different models were trained by varying the hyperparameters to build a final model with benchmark accuracy. We achieved 94.39% accuracy on the deepfake test set. The experimental results showed that upper body language can effectively detect deepfakes.

As large language models (LLMs) continue to advance, evaluating their comprehensive capabilities becomes significant for their application in various fields. This research study comprehensively evaluates the language, vision, speech, and multimodal capabilities of GPT-4o. The study employs standardized exam questions, reasoning tasks, and translation assessments to assess the model's language capability. Additionally, GPT-4o's vision and speech capabilities are tested through image classification and object recognition tasks, as well as accent classification. The multimodal evaluation assesses the model's performance in integrating visual and linguistic data. Our findings reveal that GPT-4o demonstrates high accuracy and efficiency across multiple domains in language and reasoning capabilities, excelling in tasks that require few-shot learning. GPT-4o also provides notable improvements in multimodal tasks compared to its predecessors. However, the model shows variability and faces limitations in handling complex and ambiguous inputs, particularly in audio and vision capabilities. This paper highlights the need for more comprehensive benchmarks and robust evaluation frameworks, encompassing qualitative assessments involving human judgment as well as error analysis. Future work should focus on expanding datasets, investigating prompt-based assessment, and enhancing few-shot learning techniques to test the model's practical applicability and performance in real-world scenarios.

Machine learning models facilitate the search for non-linear relationships when modeling hydrological processes, but they are equally effective for automation at the data preparation stage. The tasks whose automation was analyzed consisted in estimating changes in the roughness of a mountain streambed and the extent of floods from images. The Segment Anything Model (SAM) developed this year by Meta was used for this purpose. Images from many years from the Wielka Puszcza mountain stream located in the Polish Carpathians were used as the only input data. The model was not additionally trained for the described tasks. The model can be run in several modes, but the two most appropriate were used in this study. The first one is available in the form of a web application, while the second one is available in the form of a Jupyter notebook run in the Google Colab environment. Both methods do not require specialized knowledge and can be used by virtually any hydrologist. In the roughness estimation task, the average Intersection over Union (IoU) ranges from 0.55 for grass to 0.82 for shrubs/trees. Ultimately, it was possible to estimate the roughness coefficient of the mountain streambed between 0.027 and 0.059 based solely on image data. In the task of estimation of the flood extent, when selecting appropriate images, one can expect IoU at the level of at least 0.94, which seems to be an excellent result considering that SAM is a general-purpose segmentation model. It can therefore be concluded that SAM model can be a useful tool for a hydrologist.

The growing use of drones in precision agriculture highlights the need for enhanced operational efficiency. Despite the ability of computer vision based on deep learning has made remarkable progress in the past ten years, when it comes to segmentation task on UAVs, there is always a conflict between the demand of high precision and low inference latency. Due to such a dilemma, we propose the IRB-YOLO, an efficiency model based on Inverted Residual Block, devoting to provide constructive strategies in real-time detection tasks of UAV camera. The working details of this paper are as follows: (1) This paper innovates with a IR-Block(Inverted Residual Block), integrated into a refined YOLOv8-seg structure to create IRB-YOLO. This model specializes in pixel-level classification of UAV-acquired RGB images, facilitating the creation of exact maps to guide agricultural strategies. (2)When it comes to the experiments on a Vatica dataset with any other light-weight segmentation model, IRB-YOLO achieve at least a 3.3% increase in mAP. Further validation using a diverse species dataset confirms its robust generalization. (3)Without overloading the complex attention mechanism and deeper and deeper network, a stem that incorporates efficient feature extraction components, inverted residual block, can still possess outstanding modeling capabilities. IRB-YOLO builds a bridge between academic research and edge deployment of drones, making it applicable in real-world scenarios.

Deep Reinforcement Learning has achieved state-of-the-art results in the field of autonomous driving, especially in video game simulations. Trained agents were able to beat the best players achieving supremacy in the field. However, these agents often take in their neural network input the geometry of the track during the learning process. Our approach is to create a learning procedure for autonomous driving agents that is not environment-dependent by design to extend Deep Reinforcement Learning to racing games without racetrack information. Our work builds upon the notorious game Trackmania Nations United Forever, an open access 3D realistic Formula 1 simulation game that offers a high degree of customization. We use the Proximal Policy Optimization algorithm, using hybrid raw pixel data and physics sensors vectors as agent observations. The training track is a custom-made circuit carefully crafted from all available blocks in the game. The training track is considered like a standard dataset in machine learning and thus split in several segments in which the agent is randomly put into. We also created two regularization reward functions for the car’s speed and position to increase robustness of the agent in new scenarios.

Through the refinement of approaches to applying visual augmentation to virtual environments, the possibilities of utilising this process in robotics continues to be explored. However these endeavours have only yielded a collaborative relationship between the two technologies, where the augmented reality (AR) component is separate from the robot functionality. Regarding telepresence robots, research has shown that their perceived usefulness wanes in comparison to that of video conferencing tools as they do not offer a level of immersion and modified visual experience to justify their use. Therefore this paper contributes a novel virtually augmented robot (VAR) model that embeds AR functionality into telepresence, through the use of augmentation and facial recognition sub-models and algorithms. Experimental results show that using pulse width modulation (PWM), duty cycles, digital signals and accelerometer angles provides an approach for stabilising robot motion and the transference of human to robot head movement. The results also show the benefit of having a robot model with intrinsic AR functionality, as it offers a greater level of immersion and visual control of an environment.

Image captioning aims at generating meaningful verbal descriptions of a digital image. Our paper focuses on the classic encoder-decoder deep learning model that consists of several components – sub-networks, each performing a separate task that, combined, form an effective caption generator. We investigate image feature extractors, recurrent neural networks, word embedding models, and word generation layers and discuss how each component influences the captioning model’s overall performance. Our experiments are performed on the MS COCO 2014 dataset. The results help design efficient models with optimal combinations of their components.

The paper presents an analysis of the latest developments in the field of stereo vision in the low-cost segment, both for prototypes and for industrial designs. We described the theory of stereo vision and presented information about cameras and data transfer protocols and their compatibility with various devices. The theory in the field of image processing for stereo vision processes is considered and the calibration process is described in detail. Ultimately, we presented the developed stereo vision system and provided the main points that need to be considered when developing such systems. The final, we presented software for adjusting stereo vision parameters in real-time in the python language in the Windows operating system.

Fire detection is crucial for ensuring human safety and minimizing property damage. By utilizing advanced technologies, we can identify fires early, before they escalate. Autonomous fire detection systems are particularly vital in high-risk areas with minimal human presence. To address this need, we propose an automated fire-detection model based on YOLOv8. We modified the architecture of YOLOv8 and achieved impressive performances. Through comprehensive analysis of different YOLO models and their predecessors, we identified and addressed substantial gaps in this field. Also, by augmenting the framework with additional capabilities, we surpassed previous models, enabling real-time fire detection. Our proposed model achieves impressive performances, with 98% accuracy in fire detection and 97.8% in smoke detection. Moreover, recall rates for fire and smoke are 97.1% and 97.4%, respectively, with a mean average precision (mAP) accuracy of 99.1% for both cases. Finally, we applied an explainable artificial intelligence technique to interpret our proposed model and its results. This study lays the foundation for future research aimed at enhancing fire detection efficiency. The reliability of our modified model presents promising opportunities for future advancements, ensuring increased effectiveness and precision in efficient fire detection.

The Avian retina is far less known than that of mammals such as mouse and macaque, and detailed study is overdue. The chicken (Gallus gallus) has potential as a model, in part because research can build on developmental studies of the eye and nervous system. One can expect differences between bird and mammal retinas simply because whereas most mammals have three types of visual photoreceptor birds normally have six. Spectral pathways and colour vision are of particular interest, because filtering by oil droplets narrows cone spectral sensitivities and birds are probably tetrachromatic. The number of receptor inputs is reflected in the retinal circuitry. The chicken probably has four types of horizontal cell, there are at least 11 types of bipolar cell, often with bi- or tri-stratified axon terminals, and there is a high density of ganglion cells, which make complex connections in the inner plexiform layer. In addition, there is likely to be retinal specialisation, for example chicken photoreceptors and ganglion cells have separate peaks of cell density in the central and dorsal retina, which probably serve different types of behaviour.

Blood cell analysis is a crucial diagnostic process in medical practice. In particular, detecting white blood cells (WBCs) is essential for diagnosing of many diseases. The manual screening of blood films is a time-consuming and subjective process, which can lead to inconsistencies and errors. Therefore, automated detection of blood cells can improve the accuracy and efficiency of the screening process. In this study, an explainable Vision Transformer (ViT) model was proposed for the automatic detection of WBCs from blood films. The proposed model utilizes the self-attention mechanism to extract relevant features from the input images and leverages transfer learning by incorporating pre-trained model weights to improve its performance. The proposed model achieved a classification accuracy of 99.40% for five distinct types of WBCs and exhibited potential in reducing the time required for manual screening of blood films by pathologists. Upon examination of the misclassified test samples, it was observed that incorrect predictions were correlated with the presence or absence of granules in the cell samples. To validate this observation, the dataset was divided into two classes, namely Granulocytes and Agranulocytes, and a secondary training process was conducted. The resulting ViT model trained for binary classification achieved an accuracy of 99.70%, recall of 99.54%, precision of 99.32%, and F-1 score of 99.43% during the test phase. To ensure the reliability of the ViT model's multi-class classification of WBCs, the pixel areas that the model focuses on in its predictions are visualized through the Score-CAM algorithm.

We have an unprecedented ability to map the Earth’s surface as deep learning technologies are applied to an abundance of high-frequency Earth observation data. Simple, free, and effective methods are needed to enable a variety of stakeholders to use these tools to improve scientific knowledge and decision making. Here we present a trained U-Net model that can map and delineate ground mounted solar arrays using publicly available Sentinel-2 imagery, and that requires minimal data pre-processing and no feature engineering. By using label overloading and image augmentation during training, the model is robust to temporal and spatial variation in imagery. The trained model achieved a precision and recall of 91.5% each and an intersection over union of 84.3% on independent validation data from two distinct geographies. This generalizability in space and time makes the model useful for repeatedly mapping solar arrays. We use this model to delineate all ground mounted solar arrays in North Carolina and the Chesapeake Bay watershed to illustrate how these methods can be used to quickly and easily produce accurate maps of solar infrastructure.

To handle the problem of solving the results of aircraft’s visual navigation with scene matching, this paper take the measure of the spherical EPnP positioning posture solving by measuring the central angle threshold value and approaches for constructing a measuring model. The detailed steps are as follows: firstly, this approach needs to construct a positioning coordinate model for the earth surface, makes sure the expression for the 3-dimensional coordinate of the earth surface and solves the positioning of constructing data model with EPnP positioning posture solving algo-rithm. Secondly, by contrasting and analyzing the positioning posture value of approximate plane coordinates, the critical value is acquired, which can be recognized as plane calculation. Lastly, this method should construct a theoretical model of measurement for the visual height and central angle with the decided central angle threshold value. The simulation experiment shows that the average positioning precision of taking the spherical coordinates as input is 16.42 percent way higher than taking the plane coordinates as input. When the central angle is less than 0.05 degrees and the surface district is less than 5585 square meters, the positioning precision of the plane co-ordinates is pretty much equal to the spherical coordinates. At this moment, the sphere can be seen as flat. The conclusion of this essay can theoretically guide the further study of positioning posture solving of the scene matching, which is also of vital significance for theory research and engineering application.

Night-time lights interact with human physiology through different pathways starting at the retinal layers of the eye, from the signals provided by the rods, the S-, L- and M-cones, and the intrinsically photosensitive retinal ganglion cells (ipRGC). These individual photic channels combine in complex ways to modulate important physiological processes, among them the daily entrainment of the neural master oscillator that regulates circadian rhythms. Evaluating the relative excitation of each type of photoreceptor generally requires full knowledge of the spectral power distribution of the incoming light, information that is not easily available in many practical applications. One such instance is wide area sensing of public outdoor lighting; present-day radiometers onboard Earth-orbiting platforms with sufficient nighttime sensitivity are generally panchromatic and lack the required spectral discrimination capacity. In this paper we show that RGB imagery acquired with off-the-shelf digital single-lens reflex cameras (DSLR) can be a useful tool to evaluate, with reasonable accuracy and high angular resolution, the photoreceptoral inputs associated with a wide range of lamp technologies. The method is based on linear regressions of these inputs against optimum combinations of the associated R, G, and B signals, built for a large set of artificial light sources by means of synthetic photometry. Given the widespread use of RGB imaging devices, this approach is expected to facilitate the monitoring of the physiological effects of light pollution, from ground and space alike, using standard imaging technology.

Recent research has drawn attention to the ambiguity surrounding the definition and learnability of Out Of Distribution (OOD) recognition. The term "Out of Model Scope" (OMS) detection provides a clearer perspective, although the original problem remains unsolved. The ability to detect OMS inputs is particularly beneficial in safety-critical applications such as autonomous driving or medicine. By detecting OMS situations, we not only enhance the system's robustness but also prevent it from operating in unknown and unsafe scenarios. In this paper, we introduce a novel approach for OMS detection that combines three sources of information: 1) the original input, 2) the latent feature space extracted by an encoder, and 3) the synthetic data generated from the encoder's latent feature space. We demonstrate the effectiveness of combining original images and synthetically generated images against adversary attacks in the computer vision domain. We achieve results comparable to other state-of-the-art methods and demonstrate that any Encoder can be integrated into our pipeline in a plug-and-train fashion. Our experiments further evaluate which combination of the Encoder's features works best in order to discover OMS samples and highlight the importance of a compact feature space for the training of the Generator.

AYOLO differentiates itself with a new fusion architecture proposal that leverages the strengths of unsupervised learning and integrates the Vision Transformer approach, taking the YOLO series models as a reference. This innovation enables the model to effectively use rich, unlabeled data, providing a new example in pre-training methodology for YOLO architectures. On the benchmark COCO val2017 dataset, AYOLO demonstrates its superiority by achieving an impressive 38.7 % AP while maintaining an outstanding rendering speed of 239 FPS (Frame Per Second) on the Tesla K80 GPU. This performance outperforms the previous state-of-the-art YOLO v6-3.0 by a significant margin of +2.2 % AP while maintaining comparable FPS. AYOLO is presented as a demonstration of the potential of integrating complex information fusion techniques with supervised pretraining in improving the precision and speed of object detection models.

The article presents the operation of the automatic pine sawn timber inspection system, which was developed at the Woodinspector company and offered commercially. The vision inspection system is used to detect various wood defects, including knots, blue stain, and mechanical damage caused by worms. Blue stain is a defect that is difficult to detect based on the color of the wood, because it can be easily confused with wood defects or dirt that do not impair its strength properties. In particular, the issues of detecting blue stain in wood, the use of artificial neural networks and improving the operation of the system in production conditions were discussed.

AYOLO differentiates itself with a new fusion architecture proposal that leverages the strengths of unsupervised learning and integrates the Vision Transformer approach, taking the YOLO series models as a reference. This innovation enables the model to effectively use rich, unlabeled data, providing a new example in pre-training methodology for YOLO architectures. On the benchmark COCO val2017 dataset, AYOLO demonstrates its superiority by achieving an impressive 38.7 % AP while maintaining an outstanding rendering speed of 239 FPS (Frame Per Second) on the Tesla K80 GPU. This performance outperforms the previous state-of-the-art YOLO v6-3.0 by a significant margin of +2.2 % AP while maintaining comparable FPS. AYOLO is presented as a demonstration of the potential of integrating complex information fusion techniques with supervised pretraining in improving the precision and speed of object detection models.

Leveraging the open-world understanding capacity of large-scale visual-language pre-trained models has become a hot-spot in point cloud classification. Recent approaches rely on transferable visual-language pre-trained models, classifying point clouds by projecting them into 2D images and evaluating consistency with textual prompts. These methods benefit from the robust open-world understanding capabilities of visual-language pre-trained models and require no additional training. However, they face several challenges summarized as prompt ambiguity, image domain gap, view weights confusion, and feature deviation. In response to these challenges, we propose PointBLIP, a zero-training point cloud classification network based on the recently introduced BLIP-2 visual-language model. PointBLIP is adept at processing similarities between multi-images and multi-prompts. We separately introduce a novel method for point cloud zero-shot and few-shot classification, which involves comparing multiple features to achieve effective classification. Simultaneously, we enhance the input data quality for both the image and text sides of PointBLIP. In point cloud zero-shot classification tasks, we outperform state-of-the-art methods on three benchmark datasets. For few-shot classification tasks, to the best of our knowledge, we present the first zero-training few-shot point cloud method, surpassing previous works under the same conditions and showcasing comparable performance to full-training methods.

This paper presents a two-dimensional mathematical model of compound eye vision. Such a model is useful for solving navigation issues for autonomous mobile robots on the ground plane. The model is inspired by the insect compound eye that consists of ommatidia, which are tiny independent photoreception units, each of which combines a cornea, lens, and rhabdom. The model describes the planar binocular compound eye vision, focusing on measuring distance and azimuth to a circular feature with an arbitrary size. The model provides a necessary and sufficient condition for the visibility of a circular feature by each ommatidium. On this basis, an algorithm is built for generating a training data set to create two deep neural networks (DNN): the first detects the distance, and the second detects the azimuth to a circular feature. The hyperparameter tuning and the configurations of both networks are described. Experimental results showed that the proposed method could effectively and accurately detect the distance and azimuth to objects.

A human eye has about 120 million rod cells and 6 million cone cells. This huge number of light sensing cells inside a human eye will continuously produce a huge quantity of visual signals which flow into a human brain for daily processing. However, the real-time processing of these visual signals does not cause any fatigue to a human brain. This fact tells us the truth which is to say that human-like vision processes do not rely on complicated formulas to compute depth, displacement, and colors, etc. On the other hand, a human eye is like a PTZ camera. Here, PTZ stands for pan, tilt and zoom. We all know that in computer vision, each set of PTZ parameters (i.e., coefficients of pan, tilt and zoom) requires a dedicated calibration to determine a camera’s projection matrix. Since there is an infinite number of PTZ parameters which could be produced by a human eye, it is unlikely that a human brain stores an infinite number of calibration matrices for each human eye. Therefore, it is an interesting question for us to answer, which is to say whether simpler formulas of computing depth and displacement exist or not. Moreover, these formulas must be calibration friendly (i.e., easy process on the fly or on the go). In this paper, we disclose an important discovery of a new solution to 3D projection in a human-like binocular vision system. The purpose of doing 3D projection in binocular vision is to undertake forward and inverse transformations (or mappings) between coordinates in 2D digital images and coordinates in a 3D analogue scene. The formulas underlying the new solution are accurate, easily computable, easily tunable (i.e., to be calibrated on the fly or on the go) and could be easily implemented by a neural system (i.e., a network of neurons). Experimental results have validated the discovered formulas.

The COVID-19 outbreak demonstrated the viability of various remote working solutions, telepresence robots (TPRs) being one of them. High-quality video transmission is one of the cornerstones of using such solutions, as most of the information about the environment is acquired through vision. This study aims to compare the camera capabilities of four models of popular telepresence robots using compact reduced LogMAR and Snellen optometry charts as well as text displayed on the projector screen. The symbols from the images are extracted using OCR (Optical Character Recognition) software and the results of the recognition are compared with the symbols on the charts. Double 3 TPR provides the best quality images of optometric charts, but the OCR results of measurements of the image on the projector do not show the clear advantage of one single model over the others. The results demonstrated by Temi 2 and Double 3 TPRs are generally better than the others, suggesting that these TPRs are more feasible to be used in teaching and learning scenarios.

Accurately estimating the current state of local traffic scenes is one of the key problems in the development of software components for automated vehicles. In addition to details on free space and drivability, static and dynamic traffic participants, information on the semantics may also be included in the desired representation. Multi-layer grid maps allow to include all this information in a common representation. However, most existing grid mapping approaches only process range sensor measurements such as LIDAR and Radar and solely model occupancy without semantic states. In order to add sensor redundancy and diversity it is desired to add vision based sensor setups in a common grid map representation. In this work, we present a semantic evidential grid mapping pipeline including estimates for eight semantic classes that is designed for straightforward fusion with range sensor data. Unlike in other publication our representation explicitly models uncertainties in the evdiential model. We present results of our grid mapping pipeline based on a monocular vision setup and a stereo vision setup. Our mapping resulsts are accurate and dense mapping due to the incorporation of a disparity- or depth-based ground surface estimation in the inverse perspective mapping. We conclude this paper by providing a detailed quantitative evaluation based on real traffic scenarios in the Kitti odometry benchmark and demonstrating the advantages compared to other semantic grid mapping approaches.

Computer vision is considered as an ally to solve business problems that require human intervention, intelligence and criteria. This topic of research has evolved in XXI century at faster peace, delivering various alternatives from open source until commercial platforms. With so many options and market growing, it result difficult to make a decision on which one to use, or even worse, realize it was not suited for different scenarios. In this paper we analyze five options selected arbitrarily and tested on a dataset of 755 images to detect persons in an image, using object detectors. We analyze elapsed time to process an image, error with observations by humans, number of persons detected, correlation of time and person density, object detected size and F1 Score, considering precision and recall. As we found there are score ties and similar behaviors among options available, we introduce a novel index that takes in consideration the number of persons and their pixel size, to propose the Vision Acuity Index of Computer Vision. The results demonstrate this is a good option to serve as indicator to make decisions. Also, this index proposed have a potential to be expanded for different business use cases, and to measure new proposed algorithms in the future along with the traditional metrics used previously.

The proliferation of intelligent monitoring devices has led to the widespread adoption of background extraction technology across a multitude of domains, including intelligent transportation, video surveillance, human-computer interaction, and medical diagnosis. In this work, the model employs a multi-layer convolutional neural network structure, which enables the extraction and fusion of scene features from different scales in a layer-by-layer manner. This approach facilitates the comprehensive capture of complex scene information. The convolutional network comprises multiple layers, with each layer responsible for extracting features at a specific scale. The shallower layers capture more detailed feature information, whereas the deeper layers focus on more global scene features. The strategy of multi-scale feature fusion allows the model to fully capture multi-level and multi-dimensional information in the context of large-scale scenes. Furthermore, the incorporation of an attention mechanism enables the model to adaptively allocate attention to salient regions, thereby enhancing its capacity to discern intricate scenes by assigning greater significance to pivotal features. The experimental results demonstrate that the proposed method is effective in practice, as evidenced by its performance on public datasets.

Vision-based tactile sensors (VBTS) have become the de facto method of giving robots the ability to obtain tactile feedback from their environment. Unlike other solutions to tactile sensing, VBTS offers high spatial resolution feedback without compromising on instrumentation costs or incurring additional maintenance expenses. However, conventional cameras used in VBTS have a fixed update rate and output redundant data, leading to computational overhead downstream. In this work, we present a neuromorphic vision-based tactile sensor (N-VBTS) that employs observations from an event-based camera for contact angle prediction. Particularly, we design and develop a novel graph neural network, dubbed TactiGraph, that asynchronously operates on graphs constructed from raw N-VBTS streams exploiting their spatiotemporal correlations to perform predictions. Although conventional VBTS uses an internal illumination source, TactiGraph is reported to perform efficiently in both scenarios, with and without an internal illumination source. Rigorous experimental results revealed that TactiGraph achieved a mean absolute error of 0.62∘ in predicting the contact angle and was faster and more efficient than both conventional VBTS and other N-VBTS, with lower instrumentation costs. Specifically, N-VBTS requires only 5.5% of the compute-time needed by VBTS when both are tested on the same scenario.

The field of 3D model generation has become essential across various industries, including gaming, virtual and augmented reality (VR/AR), architecture, and medical imaging. Traditionally reliant on manual efforts, 3D content creation is now being transformed by deep generative models, enabling more efficient, scalable, and dynamic generation of complex shapes and environments. This survey provides a comprehensive review of key backbone architectures used for 3D generation, including autoencoders, variational autoencoders (VAEs), generative adversarial networks (GANs), autoregressive models, diffusion models, normalizing flows, attention-based models, CLIP-guided models, and procedural generation techniques. We explore each model’s role in 3D generation, highlighting their strengths—such as the precision of VAEs, the realism of GANs, the stability of diffusion models, and the scalability of procedural methods—alongside their limitations, such as training instability, high computational costs, and the difficulty in handling multi-modal data. Additionally, we discuss the increasing relevance of attention-enhanced models and the integration of text-based CLIP supervision for improved semantic alignment in 3D outputs. The survey concludes with an analysis of open challenges, including balancing efficiency with expressiveness, managing training complexity, and addressing dataset limitations. It also identifies future research directions, such as few-shot learning, hybrid architectures, and neural-symbolic approaches, which promise to advance the field by improving the generalization and versatility of 3D generation models. This paper aims to guide researchers and practitioners in navigating the evolving landscape of 3D generative methods and inspire new innovations in the creation of realistic, high-quality 3D content.

A 31-year-old male noticed blurred vision in his right eye for five days with no obvious predisposing causes, accompanied by mild dizziness. No obvious nodular lesions were found in the body. The patient’s binocular visual acuity was 20/20. Fundus photography showed the optic nerve swelling and radial superficial retinal hemorrhage of both eyes. Blood panel, urine routine, liver, and kidney function were all normal. Total cholesterol, triglycerides, high-density lipoprotein, and low-density lipoprotein were all in the normal limits. Head MRI showed a mass in the right temporal lobe, clear boundary, and multiple separations, which thinned and disappeared closer to the skull. The right temporal lobe and lateral ventricle were all compressed, with the midline structure shifted to the left. The patient was then transferred to Neurosurgery. During the operation, we observed the tumor had invaded the skull. The actual size of the tumor was 5.6 cm × 7.5 cm × 10.1 cm. Histology revealed foam cell accumulation in the mucous connective tissue of the right temporal lobe. The immunohistochemistry showed: CD34 (+), CD99 (+), EMA (−), GFAP (−), IDH-1 (−), Ki-67 (+) index about 10%, Oliga-2 (−), PR (−), S-100 (−), Vim (+), β-Catenin (+), CD1a (−), CD68 (+). Three months after the removal of the tumor, the visual acuity of both eyes was 20/20; The visual fields were normal, the optic disc edema and retinal hemorrhages had disappeared. MRI indicated the midline structure was back to normal.

Many accidents, such as those involving collisions or trips, appear to involve failures of vision; but the association between accident risk and vision as conventionally assessed, is weak or absent. We addressed this conundrum by embracing the distinction inspired by neuroscientific research, between vision for perception and vision for action. A dual-process perspective predicts that accident vulnerability will be associated more strongly with vision for action than vision for perception. Older and younger adults, with relatively high and relatively low self-reported accident vulnerability (Accident Proneness Questionnaire), completed three behavioural assessments targeting: vision for perception (Freiburg Visual Acuity Test); vision for action (Vision for Action Test - VAT); and the ability to perform physical actions involving balance, walking and standing (Short Physical Performance Battery). Accident vulnerability was not associated with visual acuity or with performance of physical actions; but was associated with VAT performance. VAT assesses the ability to link visual input with a specific action –launching a saccadic eye movement as rapidly as possible, in response to shapes presented in peripheral vision. The predictive relationship between VAT performance and accident vulnerability was independent of age, visual acuity and physical performance scores. Applied implications of these findings are considered.

This narrative review synthesizes recent basic and clinical research on visual disturbances in low-light environments, highlighting evaluation techniques for these conditions. It focuses on the degradation of visual acuity under dim lighting, exacerbated by pupil dilation, known as night vision disturbances (NVD). Key contributors to NVD include optical scattering, intraocular diffraction, ocular aberrations, and uncorrected refractive errors, all significantly impacting quality of life and functional abilities. The review also examines the effects of aging, eye disorders, surgical interventions, and corneal irregularities on NVD. It details the definitions, distinctions, and measurement methodologies for various optical phenomena, using both objective and subjective approaches, such as visual function questionnaires, simulators, and the Light Distortion Analyzer (LDA). The LDA is validated for clinical characterization and quantification of light distortion, proving useful in both clinical and research settings. The review advocates for continued innovation in therapeutic interventions to improve patient outcomes and alleviate the impact of visual disturbances.

Vision impairment may delay responses to hazards when driving. In a proof-of-concept driving simulator study, we evaluated a hazard warning device designed for vision impaired drivers. Three groups participated: 11 persons with central vision loss (CVL; median age 60 years), 12 with homonymous field loss (HFL; 52 years) and 11 with normal vision (NV; 60 years). CVL is characterised by impaired visual acuity and contrast sensitivity while HFL is a loss of vision in the same half of the visual field (blind side) in both eyes. Participants completed drives with and without the device which gave a directional vibro-tactile warning for an approaching pedestrian hazard when collision risk exceeded a pre-defined threshold. Warnings significantly (p< 0.001) reduced brake response times for CVL (∆= 0.54s) and HFL (∆= 0.30s), but not NV participants. For HFL participants, the reduction was greater (p = 0.02) for hazards on the blind (∆= 0.57s) than seeing side (∆= 0.10s). The warnings significantly reduced collision rates for CVL participants from 11 to 0% (all hazards) and for HFL participants from 30 to 1.7% (blind side hazards). Thus, hazard warnings may be beneficial in improving safety of vision impaired drivers; further research is warranted.

There is great interest in automatically detecting road weather and understanding its impacts on the overall safety of the transport network. This can, for example, support road condition-based maintenance or even serve as detection systems that assist safe driving during adverse climate conditions. In computer vision, previous work has demonstrated the effectiveness of deep learning in predicting weather conditions from outdoor images. However, training deep learning models to accurately predict weather conditions using real-world road-facing images is difficult due to: (1) the simultaneous occurrence of multiple weather conditions; (2) imbalanced occurrence of weather conditions throughout the year; and (3) road idiosyncrasies, such as road layouts, illumination, road objects etc. In this paper, we explore the use of focal loss function to force the learning process to focus on weather instances that are hard to learn with the objective to help address data imbalance. In addition, we explore the attention mechanism for pixel based dynamic weight adjustment to handle road idiosyncrasies using state-of-the-art vision transformer models. Experiments with a novel multi-label road weather dataset show that focal loss significantly increases the accuracy of computer vision approaches for imbalanced weather conditions. Furthermore, vision transformers outperforms current state-of-the-art convolutional neural networks in predicting weather conditions with a validation accuracy of 92% and F1-score of 81.22%, which is impressive considering the imbalanced nature of the dataset.

In recent times, using image-based deep learning techniques has been very useful for classifying fruits. However, mango is a particular challenge because its shelf life is directly related to the degree of maturity at the time of harvest, which is determined mainly through the visual assessment of the operators when sorting them, doing it incorrectly, coupled with the blows and compressions exposed during storage and transport, trigger the deterioration in their quality, hence the importance of this operation. In this sense, the purpose of this study focuses on designing a computational model based on Deep Learning that uses a convolutional neural network (CNN) parameterized to select mangoes according to their color tone based on their maturity stage. A dataset was established from 201 labeled images, then a CNN architecture was built, obtaining a model on which 3 criteria were evaluated: the number of convolution and grouping layers, the resolution of the input images and the number of training epochs required. As a result, an input image resolution of 32 x 32 pixels was achieved, the process included a convolution layer followed by pooling that fed another layer that processed the results and ended in an output layer with 2 neurons under the One Hot Encoding representation. The model required 20 epochs to perfect its training, achieving 96.04 % accuracy in mango image selection. This study supports the potential of CNNs in quality analysis through mango color selection, even in uncertainty scenarios.

The Robotics Vision Lab of Northwest Nazarene University has developed Orchard Robot (OrBot), which was designed for harvesting fruits. OrBot is composed of a machine vision system to locate fruits on the tree, a robotic manipulator to approach the target fruit, and a gripper to remove the target fruit. Field trials conducted at commercial orchards for apples and peaches during the harvesting season of 2021 yielded a harvesting success rate of about 85% and had an average harvesting cycle time of 12 seconds. Building upon this success, the goal of this study is to evaluate the performance of OrBot during nighttime harvesting. The idea is to have OrBot harvest at night, and then human pickers continue the harvesting operation during the day. This human and robot collaboration will leverage the labor shortage issue with a relatively slower robot working at night. The specific objectives are to determine the artificial lighting parameters suitable for nighttime harvesting and to evaluate the harvesting viability of OrBot during the night. LED lighting was selected as the source for artificial illumination with a color temperature of 5600 K and 10% intensity. This combination resulted with the lowest noise in the images. OrBot was tested in a commercial orchard using twenty Pink Lady apple trees. Results showed an increased success rate during the night, with OrBot gaining 94%. compared to 88% during the daytime operations.

Many sand scorpions are faithful to burrows they dig, however, it is unknown how these animals get back home after hunting excursions. Of the many mechanisms of homing that exist, path integration (PI) is one of the more common tools used by arachnids. In PI, an animal integrates its distance and direction while leaving its home, enabling it to compute an approximate home-bound vector for the return trip. The objective of our study was to test whether scorpions use PI to return home under absolute darkness in the lab. We first allowed animals to establish burrows in homing arenas. Then, after they left their burrow, we recorded the scorpion’s location in the homing arena before we transferred it to the center of a testing arena. We used overhead IR cameras to record its movements in the testing arena. If scorpions exhibited PI, we predicted they would follow a vector in the test arena that approximated the same angle and distance from the capture point to their burrow in their home arena. However, under the conditions of this experiment, we found no evidence that scorpions moved along such home-bound vectors. We speculate that scorpions may need a reliable reference cue to accommodate path integration.

Machine learning and computer vision have proven to be valuable tools for farmers to streamline their resource utilization to lead to more sustainable and efficient agricultural production. These techniques have been applied to strawberry cultivation in the past with limited success. To build on this past work, in this study two separate sets of strawberry images, along with their associated diseases, were collected and subjected to, resizing, and augmentation. Subsequently, a combined dataset consisting of 9 classes was utilized to fine-tune three distinct pre-trained models: Vision Transformer (ViT), MobileNetV2, and ResNet18. To address the imbalanced class distribution in the dataset, each class was assigned weights to ensure nearly equal impact during the training process. To enhance the outcomes, new images were generated by removing backgrounds, reducing noise, and flipping them. The performances of ViT, MobileNetV2, and ResNet18 were compared after being selected. Customization specific to the task was applied to all three algorithms, and their performances were assessed. Throughout this experiment, none of the layers were frozen, ensuring all layers remained active during training. Attention heads were incorporated into the first 5 and last 5 layers of MobileNetV2 and ResNet18, while the architecture of ViT was modified. The results indicated accuracy factors of 98.4%, 98.1%, and 97.9% for ViT, MobileNetV2, and ResNet18, respectively. Despite the data being imbalanced, the precision, which indicates the proportion of correctly identified positive instances among all predicted positive instances, approached nearly 99% with the ViT. MobileNetV2 and ResNet18 demonstrated similar results. Overall, the analysis revealed that the Vision Transformer model exhibited superior performance in strawberry ripeness and disease classification. The inclusion of attention heads in the early layers of ResNet18 and MobileNet18, along with the inherent attention mechanism in ViT, improved the accuracy of image identification. These findings offer the potential for farmers to enhance strawberry cultivation through passive camera monitoring alone, promoting the health and well-being of the population.

Sense element engagement theory explains how neural networks produce cortical prosthetic vision. A major prediction of the theory can be tested by developing a device which is expected to enable perception of continuous forms in altered visual geometries. The research reported here completes several essential steps in developing this device: (1) replication of simulations that are consistent with the theory using the NEST simulator, which can also be used for full-scale network emulation by a neuromorphic computer; (2) testing whether results consistent with the theory survive increasing the scale and duration of simulations; (3) establishing a method that uses numbers of spikes produced by network neurons to report the number of phosphenes produced by cortical stimulation; and (4) simulating essential functions of the prosthetic device. NEST simulations replicated early results and increasing their scale and duration produced results consistent with the theory. A decision function created using multinomial logistic regression correctly classified the expected number of phosphenes for 2080 spike number distributions for each of three sets of data, half of which arise from simulations expected to yield continuous visual forms on an altered visual geometry. A process for modulating electrical stimulation amplitude based on intermittent population recordings that is predicted to produce continuous visual forms was successfully simulated. The classification function developed using logistic regression will be used to tune this process as the scale of simulations is further increased.

Automation in agriculture is transforming the management of field tasks. This article focuses on identifying the needs and preliminary requirements for designing an autonomous robot for fruit harvesting. These robotic systems integrate advanced machine vision technologies and neural networks to accurately detect ripe fruits and obstacles, thereby optimizing harvesting efficiency. Additionally, the use of proximity and touch sensors is emphasized, ensuring careful and safe handling of fruits to prevent damage during harvesting. The technical specifications of these robots, including traction, speed, and their ability to operate on slopes, are evaluated to ensure adaptability to different types of crops and ground conditions. Perception systems, utilizing advanced cameras and sensors, provide essential data for crop inspection and route planning. Through a thorough analysis of operational demands, environmental conditions, and technical specifications, this article aims to establish a solid foundation for guiding the development and construction of an efficient and viable harvesting robot. The prospects and potential of fruit-harvesting robots to transform agriculture are highlighted, underscoring the importance of ongoing research and development to overcome current challenges and enhance the feasibility and effectiveness of these robotic systems.

The Mixture of Experts (MoE) architecture has evolved as a powerful and versatile approach for improving the performance and efficiency of deep learning models. This survey aims to provide the fundamental principles of MoE in details, a paradigm that harnesses the collective power of multiple specialized "expert" models working in concert to tackle complex tasks.Our exploration is a thorough analysis of the core components that constitute the MoE framework. We begin by dissecting the routing mechanism, a crucial element responsible for dynamically assigning input data to the most appropriate expert models. This routing process is pivotal in ensuring that each expert's specialized knowledge is optimally utilized. The survey places significant emphasis on expert specialization, a key feature that sets MoE apart from traditional architectures. We examine various strategies for developing and training specialized experts, exploring how this specialization enables MoE models to effectively handle diverse and multifaceted problems.Load balancing, another critical aspect of MoE systems, receives thorough attention. We discuss techniques for efficiently distributing computational resources among experts, ensuring optimal model performance while managing hardware constraints. This section provides insights into the delicate balance between model capacity and computational efficiency. Furthermore, we provide an in-depth discussion of the expert models themselves, examining their architectural designs, training methodologies, and how they interact within the larger MoE framework. This includes an analysis of different types of experts, from simple neural networks to more complex, task-specific architectures. The survey also navigates through diverse research avenues within the MoE landscape, highlighting recent advancements and innovative applications across various domains of machine learning. We pay particular attention to the burgeoning use of MoE in two rapidly evolving fields: computer vision and large language model (LLM) scaling. By providing this comprehensive overview, our survey aims to offer researchers and practitioners a deep understanding of MoE's capabilities, current applications, and potential future directions in the ever-evolving landscape of deep learning.The Mixture of Experts (MoE) architecture has evolved as a powerful and versatile approach for improving the performance and efficiency of deep learning models. This survey aims to provide the fundamental principles of MoE in details, a paradigm that harnesses the collective power of multiple specialized "expert" models working in concert to tackle complex tasks.Our exploration is a thorough analysis of the core components that constitute the MoE framework. We begin by dissecting the routing mechanism, a crucial element responsible for dynamically assigning input data to the most appropriate expert models. This routing process is pivotal in ensuring that each expert's specialized knowledge is optimally utilized. The survey places significant emphasis on expert specialization, a key feature that sets MoE apart from traditional architectures. We examine various strategies for developing and training specialized experts, exploring how this specialization enables MoE models to effectively handle diverse and multifaceted problems.Load balancing, another critical aspect of MoE systems, receives thorough attention. We discuss techniques for efficiently distributing computational resources among experts, ensuring optimal model performance while managing hardware constraints. This section provides insights into the delicate balance between model capacity and computational efficiency. Furthermore, we provide an in-depth discussion of the expert models themselves, examining their architectural designs, training methodologies, and how they interact within the larger MoE framework. This includes an analysis of different types of experts, from simple neural networks to more complex, task-specific architectures. The survey also navigates through diverse research avenues within the MoE landscape, highlighting recent advancements and innovative applications across various domains of machine learning. We pay particular attention to the burgeoning use of MoE in two rapidly evolving fields: computer vision and large language model (LLM) scaling. By providing this comprehensive overview, our survey aims to offer researchers and practitioners a deep understanding of MoE's capabilities, current applications, and potential future directions in the ever-evolving landscape of deep learning.

Livestock marketplaces consist of infrastructure and rules through which information is exchanged and transactions are completed. Livestock trading incurs significant transaction costs, beyond just commission fees paid to marketplace owners (e.g., se-lecting, loading and transporting animals). In addition, animals being transported, held and transacted in a market setting increases the likelihood of injury or disease. The Covid-19 pandemic contributed to large-scale changes across typically physical marketplaces as stay-at-home orders and travel restrictions were invoked. Livestock marketplaces pivoted online rapidly, facilitated by the deployment of video feeds, al-lowing buyers to view/purchase animals via mobile phone. This shift offered the promise of mitigating many of the transaction costs associated with a fully physical marketplace. This study evaluated how computer vision (CV) approaches can con-tribute to the evolution of livestock marketplaces. It used livestock video data for 240 cattle (bulls) weighing 320-740 Kg from 16 different breeds to elicit additional trade-relevant data (livestock weight) and can be considered as an intermediary step to better understand how video data can be used to further digitalize livestock trade, price formation and transaction completion via technology solutions. The dataset comes from three sales held on different dates in a single mart location, which allows a consistent camera setup throughout. CV technologies were employed to solve the problem of cattle weight estimation and a distinctive achievement lies in curating a dataset cap-turing cattle in motion within the context of a cattle auction thereby injecting realism and complexity into the CV weight estimation task. Employing advanced object detection algorithms, the study created an intricate data pipeline to generate an appropriate modelling dataset and devised new multi-channel architectures for deeper analysis of diverse image inputs. The significance of sizable datasets and the challenge of man-aging inherent input variability for accurate CV regression is highlighted. Top-down view predictions demonstrate comparatively better performance, although failing to achieve benchmark precision. Challenges arise from camera angle variations, roaming behaviors, and breed diversity. Likewise, side-view predictions face challenges due to increased variability and lack of standardization. Despite this, these predictions offer insights into the value of incorporating Region of Interest (ROI) coordinates and data augmentation for enhanced performance. The study concludes with a multi-input ar-chitecture combining top-down and side-view predictions, demonstrating moderate success highlighting potential synergy between both perspectives. The central con-clusion resonates strongly – using standardized and multi-view images encompassing controlled movement and consistent poses could bolster accurate CV weight estima-tion.

Regular inspections during construction work verify that the work completed is consistent with the plans and specifications and ensure that it is within the planned time and budget. This requires frequent physical site observations to independently measure and verify the completion percentage of the construction progress performed over periods of time. The current computer vision-based (CV) techniques for the measurement of as-built elements, predominantly use 3D laser scanning or 3D Photogrammetry modelling to determine the geometrical properties of as-built elements on construction sites. Both techniques require data acquisition from several positions and angles to generate sufficient information about the element’s coordinates making the deployment of these techniques on dynamic construction project sites a challenging task. In this paper, we propose a pipeline for automating the measurement of as-built components using artificial intelligence (AI) and computer vision (CV) techniques. The pipeline requires a single image obtained with a stereo-camera system to measure the size of selected objects or as-built components. We demonstrate our approach by measuring the size of concrete walls and columns. The novelty of this work is attributed to the fully automated CV-based method for measuring any given element using a single image only. The proposed solution is suitable for use in measuring the sizes of as-built components of built assets. It has the potential to be further developed and integrated with BIM models for use on construction projects for progress monitoring.

The mouse visual system recently became the most popular model to study the cellular and circuit mechanisms of sensory processing. However, the importance of eye movements in mice only started to be appreciated recently. Eye movements provide a basis for active sensing and deliver insights into various brain functions and dysfunctions. A plethora of knowledge on the central control of eye movements and their role in perception and behaviour arose from work on primates. However, an overview of the known eye movement types in mice and a comparison to primates is missing.Here, we review the eye movement types described to date in mice and compare them to those observed in primates. We discuss the central neuronal mechanisms for their generation and control. Furthermore, we review the mounting literature on eye movements in mice during head-fixed and freely moving behaviours. Finally, we highlight gaps in our understanding and suggest future directions for research.

To solve two key problems in the identification of people who are infected with COVID-19: the first problem is that the identification accuracy is not high enough. The second problem is that present identification method such as nucleic acid testing is expensive in many countries. Methods: So, I decided to design a fast identification method for COVID-19 patients which is based on deep learning. After the model (CoughNet) learns more than 6,000 cough spectrograms of both COVID-19 patients and normal people, the accuracy rate of identification of COVID-19 patients and normal people is higher than 99% in the test set. Structure: This paper is mainly divided into three parts: the first part introduces the main background and research status of the research; The second part introduces the research methods; The third part introduces the specific process of the experiment.

Daltonisation refers to the recolouring of images such that details normally lost by colour vision deficient observers become visible. This comes at the cost of introducing artificial colours. In a previous work, we presented a gradient-domain colour image daltonisation method that outperformed previously known methods both in behavioural and psychometric experiments. In the present paper, we improve the method by (i) finding a good first estimate of the daltonised image, thus reducing the computational time significantly, and (ii) introducing local linear anisotropic diffusion, thus effectively removing the halo artefacts. The method uses a colour vision deficiency simulation algorithm as an ingredient, and can thus be applied for any colour vision deficiency, and can even be individualised if the exact individual colour vision is known.

Salamanders have been habitual residents of research laboratories for more than a century, and their history in science is tightly interwoven with vision research. Nevertheless, many vision scientists – even those working with salamanders – may be unaware of how much our knowledge about vision, and particularly the retina, has been shaped by studying salamanders. In this review, we take a tour through the salamander history in vision science, highlighting the main contributions of salamanders to our understanding of the vertebrate retina. We further point out specificities of the salamander visual system and discuss the perspectives of this animal system for future vision research.

Vertebrate color vision is evolutionarily ancient. Jawless fish evolved four main spectral types of cone photoreceptor, almost certainly complemented by retinal circuits to process chromatic opponent signals. Subsequent evolution of photoreceptors and visual pigments are now documented for many vertebrate lineages and species, giving insight into evolutionary variation and ecological adaptation of color vision. We look at organization of the photoreceptor mosaic and the functions different types of cone in teleost fish, primates, and birds and reptiles. By comparison less is known about the underlying neural processing. Here we outline the diversity of vertebrate color vision and summarize our understanding of how spectral information picked up by animal photoreceptor arrays is adapted to natural signals. We then turn to the question of how spectral information is processed in the retina. Here, the quite well known and comparatively ‘simple’ system of mammals such as mice and primates reveals some evolutionarily conserved features such as the mammalian Blue^ON system which compares short and long wavelength receptors signals. We then survey our current understanding of the more complex circuits of fish, amphibians, birds and reptiles. Together, these clades make up more than 90% of vertebrate species, yet we know disturbingly little about their neural circuits for colour vision beyond the photoreceptors. Here, long-standing work on goldfish, freshwater turtles and other species is being complemented by new insights gained from the experimentally amendable retina of zebrafish. From this body of work, one thing is clear: The retinal basis of colour vision in non-mammalian vertebrates is substantially richer compared to mammals: Diverse and complex spectral tunings are established at the level of the cone output via horizontal cell feedforward circuits. From here, zebrafish use cone-selective wiring in bipolar cells to set-up color opponent synaptic layers in the inner retina, which in turn lead a large diversity of color-opponent channels for transmission to the brain. However, while we are starting to build an understanding of the richness of spectral properties in some of these species’ retinal neurons, little is known about inner retinal connectivity and cell-type identify. To gain an understanding of their actual circuits, and thus to build a more generalised understanding of the vertebrate retinal basis of color vision, it will be paramount to expand ongoing efforts in deciphering the retinal circuits of non-mammalian models.

Event-based vision is an emerging field of computer vision that offers unique properties such as asynchronous visual output, high temporal resolutions, and dependence on brightness changes to generate data. These properties can enable robust high-temporal-resolution object detection and tracking when combined with frame-based vision. In this paper, we present a hybrid, high-temporal-resolution, object detection and tracking approach, that combines learned and classical methods using synchronized images and event data. Off-the-shelf frame-based object detectors are used for initial object detection and classification. Then, event masks, generated per each detection, are used to enable inter-frame tracking at varying temporal resolutions using the event data. Detections are associated across time using a simple low-cost association metric. Moreover, we collect and label a traffic dataset using the hybrid sensor DAVIS 240c. This dataset is utilized for quantitative evaluation using state-of-the-art detection and tracking metrics. We provide ground truth bounding boxes and object IDs for each vehicle annotation. Further, we generate high-temporal-resolution ground truth data to analyze the tracking performance at different temporal rates. Our approach shows promising results with minimal performance deterioration at higher temporal resolutions (48 – 384 Hz) when compared with the baseline frame-based performance at 24 Hz.

This article explores the efficiency of various clustering methods for image segmentation under different luminosity conditions. Image segmentation plays a crucial role in computer vision appli-cations, and clustering algorithms are commonly used for this purpose. The search for an adaptive clustering mechanism aims to ensure maximum symmetry of real objects with objects/segments in their digital representations. However, clustering methods performance can fluctuate with varying lighting conditions during image capture. Therefore, we assess the performance of several clustering algorithms — including k-means, K-Medoids, fuzzy c-means, Possibilistic C-Means, Gus-tafson-Kessel, Entropy-based Fuzzy, Riddler-Calvard, Kohonen Self-Organizing Maps, and Meanshift — across images captured under different illumination conditions. Additionally, we develop an adaptive image segmentation system utilizing empirical data. Conducted experiments highlight varied performance among clustering methods under different luminosity conditions. This research enhance better understanding of luminosity's impact on image segmentation and aiding in method selection for diverse lighting scenarios.

In this paper, we present a multi-task model that predicts disparities and confidence levels in deep stereo matching simultaneously. We do this by combining its successful model for each separate task and obtaining a multitask model that can be trained with a proposed loss function. We show the advantages of this model compared to training and predicting disparity and confidence sequentially. This method enables an improvement of 15% to 30% in the area under the curve (AUC) metric when trained parallel rather than sequential. In addition, the effect of weighting the components in the loss function on the stereo and confidence performance is investigated. By improving the confidence estimate, the practicality of stereo estimators for creating distance images is increased.

Computational animation of spatiotemporal models for 3-D shape and motion is considered as core of dynamic real-time vision. This is based on sets of features evaluated from multi-focal image streams. Subjects are defined as special objects capable of sensing environmental parameters and of initiating own actions in combination with stored knowledge. Object/subject recognition and scene understanding are achieved on different levels. Multiple objects are tracked individually for perceiving their actual state (‘here and now’). Fast saccadic jumps in gaze direction allow flexible concentration on objects or parts of actual interest. By analyzing motion of relevant objects/subjects over a larger time scale on the level of state variables and documenting them in the ‘scene tree representation’ (computer graphics), the situation with respect to decision making is assessed. Various behavioral capabilities of subjects are represented on an abstract level for characterizing their potential behaviors. These are generated by stereotypical feed-forward and feedback control applications on a separate level close to the actuator hardware with corresponding methods. This dual representation on one level for decision making and one on the implementation level allows for flexibility and easy adaptation or extension. Results are shown for road vehicle guidance based on three cameras on a gaze-controlled platform.

We introduce a novel microscopic image dataset augmented with segmentation and detection labels specifically designed for microplastic analysis in sewage environments. Recognizing the increasing concern over microplastics — particles of synthetic polymers smaller than 5mm — and their detrimental effects on marine ecosystems and human health, our research focuses on enhancing detection and analytical methodologies through advanced computer vision and deep learning techniques. The dataset comprises high-resolution microscopic images of microplastics collected from sewage, meticulously labeled for both segmentation and detection tasks, aiming to facilitate accurate and efficient identification and quantification of microplastic pollution. In addition to dataset development, we present example deep learning models optimized for segmentation and detection of microplastics within complex sewage samples. The models demonstrate significant potential in automating the analysis of microplastic contamination, offering a scalable solution to environmental monitoring challenges. Furthermore, we ensure the accessibility and reproducibility of our research by making the dataset and model codes publicly available, accompanied by detailed documentation on GitHub and LabelBox. The dataset and example deep learning models are publicly available at the following GitHub link. https://anonymous.4open.science/r/Microplastics-in-Sewage-1BEF

Skin cancer is a global health concern with increasing prevalence, necessitating effective early detection and classification systems. Transfer learning has emerged as a powerful tool in this domain, enhancing diagnostic accuracy. Attention mechanisms, which selectively focus on pertinent image features, play a pivotal role in transfer learning. While previous studies have highlighted their effectiveness, their applicability across different models and datasets remains understudied. This empirical study explores the impact of various attention mechanisms on the performance of transfer learning models for skin cancer detection and classification. Using five transfer learning models (DenseNet121, InceptionV3, MobileNet, VGG16, and Xception) and six attention mechanisms (Channel Attention, Global Context Attention, Guided Attention, Nonlocal Attention, Positional Attention, and Spatial Attention), we conducted 105 experiments across three datasets. Traditional metrics (accuracy, precision, recall, and f1-score) were employed for empirical validation. The results reveal a nuanced relationship between attention mechanisms, transfer learning models, and datasets. Overall, attention mechanisms exhibit the potential to enhance skin cancer classification. Spatial and channel attention mechanisms consistently outperform others, offering simplicity and effectiveness. Model-specific selection of attention mechanisms is crucial, with a trade-off between model complexity and performance evident. This study provides insights into developing efficient skin cancer classification models utilizing attention mechanisms.

Image analysis is widely applied in plant science for phenotyping and monitoring botanic and agricultural species. Although a lot of software is available, tools integrating image analysis and statistical assessment of seedling growth in large groups of plants are limited or absent, and do not cover the needs of the researchers. In this study, we developed Morley, a free, open-source graphical user interface written in Python. Morley automates the following workflow: (1) group-wise analysis of a few thousand seedlings from multiple images; (2) recognition of seeds, shoots and roots in seedling images; (3) calculation of shoot and root lengths and surface areas, (4) evaluation of statistically significant differences between plant groups, (5) calculation of germination rates, (6) visualization and interpretation. Morley is designed for laboratory studies of biotic effects on seedling growth, when molecular mechanisms underlying morphometric changes are analyzed. Performance was tested using cultivars of T. aestivum, P. sativum on seedlings of up to 1 week old. Accuracy of the measured morphometric parameters was comparable with the ones obtained using ImageJ and manual measurements. Dose-dependent laboratory tests for germination affected by new bioactive compounds and fertilizers, assuming extraction of seedlings from a substrate and/or dissection are among the suggested applications.

Existing methods for 3D human pose estimation mainly divide the task into two stages. The first stage identifies the 2D coordinates of the human joints in the input image, namely the 2D human joint coordinates. The second stage uses the results from the first stage as input to recover the depth information of human joints from the 2D human joint coordinates to achieve 3D human pose estimation. However, the recognition accuracy of the two-stage method relies heavily on the results of the first stage and includes too many redundant processing steps, which reduces the inference efficiency of the network. To address these issues, we propose the EDD, a fully End-to-end 3D human pose estimation method based on transformer architecture with Dual Decoders. By learning multiple human poses, the model can directly infer all 3D human poses in the image using a pose decoder, and then further optimize the recognition result using a joint decoder based on the kinematic relations between joints. With the attention mechanism, this method can adaptively focus on the most relevant features to the target joint, effectively overcoming the feature misalignment problem in the human pose estimation task and greatly improving the model performance. Any complex post-processing step, such as non-maximum suppression, is eliminated, further improving the efficiency of the model. The results show that the method achieves an accuracy of 87.4% on the MuPoTS-3D dataset, significantly improving the accuracy of end-to-end 3D human pose estimation methods.

This review paper provides a comprehensive overview of machine vision pose measurement algorithms. The paper focuses on the state-of-the-art algorithms and their applications. The paper is structured as follows: The introduction in Section 1 provides a brief overview of the field of machine vision pose measurement. Section 2 describes the commonly used algorithms for machine vision pose measurement. Section 3 discusses the factors that affect the accuracy and reliability of machine vision pose measurement algorithms. Section 4 presents the applications of machine vision pose measurement in various fields. The paper provides specific examples of how machine vision pose measurement is used in each of these fields. Finally, Section 5 summarizes the paper and provides future research directions. The paper highlights the need for more robust and accurate algorithms that can handle varying lighting conditions and occlusion. It also suggests that the integration of machine learning techniques may improve the performance of machine vision pose measurement algorithms. Overall, this review paper provides a comprehensive overview of machine vision pose measurement algorithms, their applications, and the factors that affect their accuracy and reliability. It provides a valuable resource for researchers and practitioners working in the field of computer vision.

Existing one-shot neural architecture search (NAS) methods have to conduct a search over a giant super-net, which leads to the huge computational cost. To reduce such cost, in this paper, we propose a method, called FTSO, to divide the whole architecture search into two sub-steps. Specifically, in the first step, we only search for the topology, and in the second step, we search for the operators. FTSO not only reduces NAS’s search time from days to 0.68 seconds, but also significantly improves the found architecture's accuracy. Our extensive experiments on ImageNet show that within 18 seconds, FTSO can achieve a 76.4% testing accuracy, 1.5% higher than the SOTA, PC-DARTS. In addition, FTSO can reach a 97.77% testing accuracy, 0.27% higher than the SOTA, with nearly 100% (99.8%) search time saved, when searching on CIFAR10.

The increasing use of deep learning techniques to manipulate images and videos, commonly referred to as "deepfakes," is making more and more challenging to differentiate between real and fake content. While various deepfake detection systems have been developed, they often struggle to detect deepfakes in real-world situations. In particular, these methods are often unable to effectively distinguish images or videos when these are modified using novel techniques which have not been used in the training set. In this study, we carry out an analysis of different deep learning architectures in an attempt to understand which is more capable of better generalizing the concept of deepfake. According to our results, it appears that Convolutional Neural Networks (CNNs) seem to be more capable of storing specific anomalies and thus excel in cases of datasets with a limited number of elements and manipulation methodologies. The Vision Transformer, conversely, is more effective when trained with more varied datasets, achieving more outstanding generalization capabilities than the other methods analysed. Finally, the Swin Transformer appears to be a good alternative for using an attention-based method in a more limited data regime. All the analyzed architectures seem to have a different way to look at deepfakes but since in a real-world environment, the generalization capability is essential, based on the carried out experiments the Vision Transformer seems to provide superior performances.

Vision loss can be avoided if diabetic retinopathy (DR) is diagnosed and treated promptly. Following are the main 5 DR stages: none, moderate, mild, proliferate, and severe. In this study, a deep learning (DL) model is presented that diagnoses all 5 stages of DR with more accuracy than previous methods. The suggested method presents two scenarios: case 1 with image enhancement using contrast limited adaptive histogram equalization (CLAHE) filtering algorithm in conjunction with an Enhanced Super-resolution generative adversarial network (ESRGAN), and case 2 without image enhancement; augmentation techniques are then performed to generate a balanced dataset utilizing the same parameters for both cases. Using Inception-V3 applied to the Asia Pacific Tele-Ophthalmology Society (APTOS) datasets, the developed model achieved an accuracy of 98.7% for case 1 and 80.87% for case 2, which is greater than existing methods for detecting the five stages of DR. It was demonstrated that using CLAHE and ESRGAN improves a model's performance and learning ability.

This work presents the development and implementation of a distributed navigation system based on computer vision. The autonomous system consists of a wheeled mobile robot with an integrated colour camera. The robot navigates through a laboratory scenario where the track and several traffic signals must be detected and recognized by using the images acquired with its on-board camera. The images are sent to a computer server that processes them and calculates the corresponding speeds of the robot using a cascade of trained classifiers. These speeds are sent back to the robot, which acts to carry out the corresponding manoeuvre. The classifier cascade should be trained before experimentation with two sets of positive and negative images. The number of images in these sets should be considered to limit the training stage time and avoid overtraining the system.

In recent years, a range of problems under the broad umbrella of computer vision based analysis of ancient coins have been attracting an increasing amount of attention. Notwithstanding this research effort, the results achieved by the state of the art in published literature remain poor and far from sufficiently well performing for any practical purpose. In the present paper we present a series of contributions which we believe will benefit the interested community. We explain that the approach of visual matching of coins, universally adopted in existing published papers on the topic, is not of practical interest because the number of ancient coin types exceeds by far the number of those types which have been imaged, be it in digital form (e.g., online) or otherwise (traditional film, in print, etc.). Rather, we argue that the focus should be on understanding the semantic content of coins. Hence, we describe a novel approach—to first extract semantic concepts from real-world multimodal input and associate them with their corresponding coin images, and then to train a convolutional neural network to learn the appearance of these concepts. On a real-world data set, we demonstrate highly promising results, correctly identifying a range of visual elements on unseen coins with up to 84% accuracy.

Importance: Because of the high prevalence of myopia in Taiwan, understanding the risk factors for its development and progression is important to public health. Background: This study investigated the risk factors for myopia and their influence on the progression of myopia in schoolchildren in Taiwan. Design: Patients’ clinical records were obtained retrospectively from ophthalmologists. Questionnaires were given to collect demographic information, family background, hours spent on daily activities, myopia progression, and treatment methods. Participants: A total of 522 schoolchildren with myopia from a regional medical hospital in northern Taiwan participated the study. Written informed consent was obtained from the participants of legal age or the parents or legal guardians. Methods: Multivariable regression analyses were performed. Myopia measured in dioptres was analysed, controlling for patients’ family and demographic information as well as their daily behaviours. Main Outcome Results: Children with high myopic parents were more myopic. Earlier onset age of myopia was associated with a higher level of myopia and greater annual myopic progression. Children reporting more near work activities had higher levels of myopia and greater progression of myopia. Lower levels of myopia were associated with more exercise, longer periods of sleep, and better vision care knowledge in children and parents. Intake of food supplements had no effect on myopia. Conclusions and Relevance: In addition to genetics, education, environment, and near work activity can influence the development of myopia. Health policies for schoolchildren should promote protective activities and vision care knowledge in order to protect the eyesight of schoolchildren.

Spike shape and morphometric characteristics are among the key characteristics of cultivated cereals associated with their productivity. Identification of the genes controlling these traits requires morphometric data at harvesting and analysis of numerous plants, which could be automatically done using technologies of digital image analysis. A method for wheat spike morphometry utilizing 2D image analysis is proposed. Digital images are acquired in two variants: a spike on a table (one projection) or fixed with a clip (four projections). The method identifies spike and awns in the image and estimates their quantitative characteristics (area in image, length, width, circularity, etc.). Section model, quadrilaterals, and radial model are proposed for describing spike shape. Parameters of these models are used to predict spike shape type (spelt, normal, or compact) by machine learning. The mean error in spike density prediction for the images in one projection is 4.61 (~18%) versus 3.33 (~13%) for the parameters obtained using four projections.

In this paper, we leverage state of the art models on Imagenet data-sets. We use the pre-trained model and learned weighs to extract the feature from the Dog breeds identification data-set. Afterwards, we applied fine-tuning and dataaugmentation to increase the performance of our test accuracy in classification of dog breeds datasets. The performance of the proposed approaches are compared with the state of the art models of Image-Net datasets such as ResNet-50, DenseNet-121, DenseNet-169 and GoogleNet. we achieved 89.66% , 85.37% 84.01% and 82.08% test accuracy respectively which shows the superior performance of proposed method to the previous works on Stanford dog breeds datasets.

This paper presents a system that merges computer vision and surface electromyography techniques to carry out grasping tasks. To perform this, the vision-driven system is used to compute pre-grasping poses of the robotic system based on the analysis of tridimensional object features. Then, the human operator can correct the pre-grasping pose of the robot using surface electromyographic signals from the forearm during wrist flexion and extension. Weak wrist flexions and extensions allow a fine adjustment of the robotic system to grasp the object and finally, when the operator considers that the grasping position is optimal, a strong flexion is performed to initiate the grasping of the object. The system has been tested with several subjects to check its performance showing a grasping accuracy of around 95% of the attempted grasps which increases by around a 9% the grasping accuracy of previous experiments in which electromyographic control was not implemented.

The economic development and infrastructure of a nation are closely interrelated. In addition, public trust in national infrastructure facilities is closely linked to the preservation of the advantages provided by these facilities to the public. Since the 1970s, Korea has achieved exponential economic growth over a short period of time and the number of infrastructure facilities has increased correspondingly. This compressed economic development has been underpinned by the national infrastructure, whose safety and usability have been excluded from the scope of the development. However, after around 30 years, structural deterioration coupled with general insensitivity to safety in today’s society has considerably reduced public trust in using the infrastructure. Realistically, policies that mainly focus on developing new technologies related to infrastructure construction have led to practical limitations that discourage the development of technologies for maintenance or inspection. Furthermore, current maintenance works face certain limitations caused by various reasons: insufficient budget, increasing number of infrastructure facilities requiring maintenance, shortage of manpower, and rapidly increasing number of aging infrastructure facilities. To overcome these limitations, a new approach is required that is different from general inspection methods under the existing rules and regulations. In this context, this study aimed to explore the efficiency of bridge inspection and maintenance by unmanned aerial vehicles (UAVs) that could observe inaccessible areas, could be conveniently and easily controlled, and could offer high economic benefits. To this end, various tests were performed on elevated bridges, and suitable UAV images were obtained. The obtained UAV images were inspected by using machine vision technology, thereby excluding subjective evaluations by humans. Methods for enhancing the objectivity of the inspection were also discussed. The test results showed that both the efficiency and objectivity of the proposed method were better than those of the existing bridge maintenance and inspection methods.

Tracking and early identification of suspected cases are essential to control and prevent potential COVID-19 outbreaks. One of the most popular techniques used to track this disease is the use of Infrared cameras to identify individuals with elevated body temperatures. However, they are limited by their inability to be implemented in open public settings such as public parks or even outdoor recreational centers. This limits their ability to effectively track possible COVID-19 patients as open public recreational places such as parks, concert venues and other public venues are hotspots for the spreading of the virus. Other technological solutions such as thermal scanners require an individual to perform the actual testing as they are not individual standalone technologies. This method of testing can potentially cause the transmission of the virus between the tester and the individual getting tested. As can be seen, an alternative solution is essential to solving this issue. In this study, we aim to present the system, design and potential scope of a non-invasive system that can diagnose and identify potential COVID-19 patients using thermal and optical images of the individual using drone technology. The proposed system (COVIDRONE) combines multi-modal machine intelligence, computer vision and real-time monitoring to enable scalable monitoring. The system will also involve the use of machine learning algorithms for better and more accurate diagnosis. We envisage that development of such technologies may help in developing technological solutions to combat infectious disease threats in the future pandemics.

Drivable area (DA) detection is crucial for autonomous driving. Camera-based methods heavily rely on lighting conditions and often fail to capture accurate 3D information, while LiDAR-based methods offer accurate 3D data and are less susceptible to lighting conditions. However, existing LiDAR-based methods focus on point-wise detection, prone to occlusion and limited by point cloud sparsity, leading to decreased performance in motion planning and localization. We propose Argoverse-grid, a grid-wise DA detection dataset derived from Argoverse 1, comprising over 13K frames with fine-grained BEV DA labels across various scenarios. We also introduce Grid-DATrNet, a first grid-wise DA detection model utilizing global attention through transformers. Our experiments demonstrate the superiority of Grid-DATrNet over various methods, including both LiDAR and camera-based approaches, in detecting grid-wise DA in the proposed Argoverse-grid dataset. We show that Grid-DATrNet can detect grids even in occluded and unmeasured areas by leveraging contextual and semantic information through global attention, unlike CNN-based DA detection methods. The preprocessing code for Argoverse-grid, experiment code, Grid-DATrNet implementation, and result visualization code will be made available at https://github.com/kaist-avelab/grid-wise-DA.

This study investigates the possibility of using machine learning models created in DeepLabCut and Create ML to automate aspects of behavioral coding and aid in behavioral analysis. Two models with different capabilities and complexities were constructed and compared to a manually observed control period. The accuracy of the models was assessed before being applied to 7 nights of footage of the nocturnal behavior of two African elephants (Loxodonta africana). The resulting data was used to draw conclusions regarding behavioral differences between the two elephants and between individually observed nights, thus proving that such models can aid researchers in be-havioral analysis. The models were capable of tracking simple behaviors with high accuracy, but had certain limitations regarding detection of complex behaviors, such as the stereotyped behavior sway, and displayed confusion when deciding between visually similar behaviors. Further expansion of such models may be desired to create a more capable aid with the possibility of automating behavioral coding.

This study introduces a sophisticated neural network structure for segmenting breast tumors. It achieves this by combining a pretrained Vision Transformer (ViT) model with a U-Net framework. The U-Net architecture, commonly employed for biomedical image segmentation, is further enhanced with Depthwise Separable Convolutional Blocks to decrease computational complexity and parameter count, resulting in better efficiency and less overfitting. The Vision Transformer, renowned for its robust feature extraction capabilities utilizing self-attention processes, efficiently captures the overall context within images, surpassing the performance of conventional convolutional networks. By using a pretrained ViT as the encoder in our U-Net model, we take advantage of its extensive feature representations acquired from extensive datasets, resulting in a major enhancement in the model’s ability to generalize and train efficiently. The suggested model has exceptional performance in segmenting breast cancers from medical images, highlighting the advantages of integrating transformer-based encoders with efficient U-Net topologies. This hybrid methodology emphasizes the capabilities of transformers in the field of medical image processing and establishes a new standard for accuracy and efficiency in activities related to tumor segmentation.

Purpose of research was to determine changes in optical quality for monthly disposable soft contact lenses (SCL) wearers with new and monthly worn SCL. Retinal straylight was assessed in SCL wearers using a C-Quant straylight meter. The study included 33 participants (age 22 ± 1.4 years) who wore monthly disposable silicone hydrogel contact lenses. Measurements were taken with month-old SCL that had been worn for no more than 4 hours. Participants then replaced the old lenses with new ones. After a 15-minute adaptation period, straylight measurements were taken with the new SCL. A questionnaire was administered to all participants, inquiring about vision complaints and the type of contact lens material used. The decimal logarithm of the stray-light parameter, log(s), was notably higher with month-old SCL (log(s) = 0.97±0.17) than with brand-new SCL (log(s) = 0.86±0.15), p < 0.001. The average increase in retinal straylight after a month of SCL use was log(s) = 0.11 ± 0.08. The questionnaire revealed that 35% of participants observed a reduction in visual quality after wearing the contact lenses for a month. Results showed that monthly disposable soft contact lenses could reduce visual quality when scheduled replacement time has been reached.

The Saudi Vision 2030 is critical to boosting national development, notably in the construction industry. The construction industry is getting a lot of attention in a variety of ways, the most visible of which is the promotion of best practices, including project management office (PMO) applications. Hail province, like others, is experiencing project delivery issues. Thus, the purpose of this research is to explore the suitable PMO model for Hail public construction projects. A literature review included the drive factors for establishing PMO in construction firms, as well as PMO models and typologies that describe functions. Therefore, those drive factors and PMO models were subjected to a focus group to explore their suitability in the context of Hail. The research findings reveal that the Enterprise PMO is the main for Hail public construction pro-jects, followed respectively by the Program Management Office model and the Project Management Center of Excellence (PMCOE). This method of selection a PMO model is driving construction companies and related businesses to change in order to improve the sector while also ad-dressing construction project delivery challenges in Hail province.

Convolutional Neural Networks (CNNs) present drawbacks for modeling geometric transformations, such as scaling and rotation, caused by the convolution operation’s locality. Deformable convolution (DCON), a mechanism that substitutes standard convolution, increasing the receptive field to capture relevant features, is a promising approach to solve this drawback and improve the robustness of CNNs. However, the optimal way to replace the standard convolution with its deformable counterpart in a CNN model is unclear. In this study, we clarify this aseveration by conducting several experiments using deformable convolutions applied in the layers that conform a small four-layer CNN model. We also use deformable convolutions on the four-layers of several ResNet CNNs with depths 18, 34, 50, and 101. The models were tested in binary balanced classes with 2D data for image classification: Cats &amp; Dogs, EyesPACS, Spyders &amp; Chickens, and Shapes. After this testing, we evaluated DCON in 3D data for action recognition: UCF101 and Human2 (a dataset we compiled to control movement, clothing and background). The contribution of this research lies in a guideline to use DCON. It can be summarized as follows: if DCON is used on the first layers of the proposal of model (with simple features), the computational resources expressed as the quantity of Flops will tend to increase and produce bigger misclassification than the standard CNN. However, if the DCON is used at the end layers, the quantity of Flops used in the training and testing will decrease, and the classification accuracy will improve by up to 20% about the base model. Moreover, it gains robustness when using deformable convolutions because it can adapt to the region of interest. Also, the best kernel size of the DCON is three. It showed better results than size five. In the last case, the quantity of Flops increase quadratically, but their performance does not increase significantly. With these results, we propose a guideline to use the DCON and contribute to understanding the impact of DCON on the robustness of CNNs.

The rise in both the frequency of natural disasters and the ubiquity of artificial intelligence has led to novel applications of new technologies in improving disaster response processes, such as the labor-intensive assessment of disaster damages. Assessment of residential and commercial structure damages, a precursory step to government agencies being able to provide most of their financial assistance, has benefited from aerial and satellite imagery-based computer vision models; however, limitations of using such imagery include the ground structures being obscured by clouds or smoke, as well as the lack of resolution to distinguish individual structures from others. Using a different data source, we propose a damage severity classification model using ground-level imagery, focusing on residential structures damaged by wildfires. This classifier, a Vision Transformer (ViT) model trained on over 18,000 professionally labeled images of damaged homes from the 2020-2022 California wildfires, has achieved an accuracy score of over 95%. Further, we have open sourced the training dataset–the first of its kind and scale–as well as built a publicly available web application prototype, which we demoed to and received feedback from disaster response officials, both of which further contribute to the broader literature beyond the proposed model.

This paper explores the application of machine learning techniques for automated object grasping. Current electronic prosthetics often require extensive training for users to gain fine motor control over the prosthetic fingers, hindering their usability and acceptance. To address this challenge, this paper proposes an automated method that leverages computer vision-based techniques and machine learning algorithms. In this study, three reinforcement learning algorithms, namely Soft Actor-Critic (SAC), Deep Q-Network (DQN), and Proximal Policy Optimization (PPO), are employed to train agents for automated grasping tasks. The results indicate that the SAC algorithm achieves the highest success rate of 99% among the three algorithms at just under 200,000 timesteps. This research also shows that object’s physical characteristics can affect the agent’s ability to learn an optimal policy. Moreover, the findings highlight the potential of the SAC algorithm in developing intelligent prosthetic hands with automatic object gripping capabilities.

The twelve cranial nerves play a crucial role in nervous system, orchestrating a myriad of functions vital for our everyday life. These nerves are each specialized for particular tasks. Cranial Nerve I, known as the olfactory nerve, is responsible for our sense of smell, allowing us to perceive and distinguish various scents. Cranial Nerve II, or the optic nerve, is dedicated to vision, transmitting visual information from the eyes to the brain. Eye movements are governed by Cranial Nerves III, IV, and VI, ensuring our ability to track objects and focus. Cranial Nerve V controls facial sensation and jaw movement, while Cranial Nerve VII, the facial nerve, facilitates facial expressions and taste perception. Cranial Nerve VIII, or the vestibulocochlear nerve, plays a critical role in hearing and balance. Cranial Nerve IX, the glossopharyngeal nerve, affects throat sensation and taste perception. Cranial Nerve X, the vagus nerve, is a far-reaching nerve, influencing numerous internal organs such as the heart, lungs, and digestive system. Cranial Nerve XI, the accessory nerve, is responsible for neck muscle control, contributing to head movement. Finally, Cranial Nerve XII, the hypoglossal nerve, manages tongue movement, essential for speaking, swallowing, and breathing. Understanding these cranial nerves is fundamental in comprehending the intricate workings of our nervous system and the functions that sustain our daily lives.

This study proposes a method for image matching between infrared(IR)-RGB images using a deep learning network to estimate the deformation field. We propose a deformation field generator (DFG) that estimates the deformation field of the transformation matrix to match each pixel or IR image to the RGB image. DFG is a network that receives IR and RGB images as input; the output size is two channels and has the sample resolution as the input image. By warping the IR image through a grid-sampler that warps the image according to the value of the deformation field, we can obtain a warped IR image that matches the RGB image. Additionally, to check whether the warped IR image matched the RGB image, the masking images detecting the segmentation of objects were photographed in two images. Without directly comparing IR and RGB images, we proposed mask loss that warps the IR mask image through the deformation field and grid sampler and then compares the warped IR mask image with the RGB mask image. Mask loss solves the spatial similarity comparison problem with multi-modality images, such as IR and RGB images, by comparing the mask image with the same modality image as the mask image.

Mice are one of the frequently used animal models in science research whose behavioral characteristics can provide much valuable information in biology, neuroscience, and pharmacology. Nowadays, artificial intelligence is widely used in mice behavior analysis. Integrated AI systems such as ChatGPT and VisualGPT are already available, and we discuss the feasibility of MiceGPT to help researchers identify and classify mouse behavior more easily. We review the applications of mice behavior analysis, analyze the tasks of deep learning on these applications based on an AI pyramid, and finally summarize the AI approaches to solve these tasks. Based on these summaries, we propose three MiceGPT architectures to demonstrate the theoretical feasibility of MiceGPT.

Computer vision methods have shown to be effective in classifying garbage into recycling categories for waste processing, existing methods are costly, imprecise, and unclear. To tackle this issue, we introduce MWaste, a mobile application that uses computer vision and deep learning techniques to classify waste materials as trash, plastic, paper, metal, glass or cardboard. Its effectiveness was tested on various neural network architectures and real world images, achieving an average precision of 92% on the test set. This app can help combat climate change by enabling efficient waste processing and reducing the generation of greenhouse gases caused by incorrect waste disposal.

Sometimes when diabetic retinopathy (DR) is found and treated quickly, vision loss can indeed be spared. This study deploys a deep learning (DL) model that can discover all 5 stages of DR more accurately than other methods. The proposed methodology shows two cases scenarios: case 1 with image enhancement using CLAHE and ESRGAN, and case 2 without image enhancement. Augmentation techniques are then employed to produce a balanced dataset with the identical criteria for both scenarios. The generated model using DenseNet-121 on the APTOS dataset outperformed other approaches for locating the 5 stages of DR, with an accuracy of 98.7 percent for case 1 and 81.2 percent for case 2. Using CLAHE and ESRGAN was shown to improve a model's performance and ability to learn.

In the mid-layers of Deep Learning systems, clustered features tend to fit multiple classifications, which are filtered out during the final stages of object recognition. However, many misclassifications remain, regarded as errors of the system. This paper claims that tagging an entity incorrectly for reasons of similarity is evidence of spontaneous machine creativeness. According to the ratings of 40 design educators and researchers, AI-generated false-class inclusions produced creative design ideas, predicting the level of innovation value. These designers were not just anybody but came from a design school in Asia with a top position on the world ranking-lists. They entered an experiment in which 20 classification mistakes were framed as early-design ideas that were either human-made or intentionally suggested by creative AI. Many examples passed the Feigenbaum variant of the Turing test with a conceptual preference to creations supposedly done by human hand.

In this paper, our goal is to perform a virtual restoration of an ancient coin from its image. The present work is the first one to propose this problem, and it is motivated by two key promising applications. The first of these emerges from the recently recognised dependence of automatic image based coin type matching on the condition of the imaged coins; the algorithm introduced herein could be used as a pre-processing step, aimed at overcoming the aforementioned weakness. The second application concerns the utility both to professional and hobby numismatists of being able to visualise and study an ancient coin in a state closer to its original (minted) appearance. To address the conceptual problem at hand, we introduce a framework which comprises a deep learning based method using Generative Adversarial Networks, capable of learning the range of appearance variation of different semantic elements artistically depicted on coins, and a complementary algorithm used to collect, correctly label, and prepare for processing a large numbers of images (here 100,000) of ancient coins needed to facilitate the training of the aforementioned learning method. Empirical evaluation performed on a withheld subset of the data demonstrates extremely promising performance of the proposed methodology and shows that our algorithm correctly learns the spectra of appearance variation across different semantic elements, and despite the enormous variability present reconstructs the missing (damaged) detail while matching the surrounding semantic content and artistic style.

Eye movements support memory encoding by binding distinct elements of the visual world into coherent representations. However, the role of eye movements in memory retrieval is less clear. We propose that eye movements play a functional role in retrieval by reinstating the encoding context. By overtly shifting attention in a manner that broadly recapitulates the spatial locations and temporal order of encoded content, eye movements facilitate access to, and reactivation of, associated details. Such mnemonic gaze reinstatement may be obligatorily recruited when task demands exceed cognitive resources, as is often observed in older adults. We review research linking gaze reinstatement to retrieval, describe the neural integration between the oculomotor and memory systems, and discuss implications for models of oculomotor control, memory, and aging.

Most of the existing calibration methods for binocular stereo vision sensor (BSVS) depend on high-accuracy target with feature points that are difficult to manufacture and costly. In complex light conditions, optical filters are used for BSVS, but they affect imaging quality. Hence, the use of a high-accuracy target with certain-sized feature points for calibration is not feasible under such complex conditions. To solve these problems, a calibration method based on unknown-sized elliptical stripe images is proposed. With known intrinsic parameters, the proposed method adopts the elliptical stripes located on the parallel planes as a medium to calibrate BSVS online. In comparison with the common calibration methods, the proposed method avoids utilizing high-accuracy target with certain-sized feature points. Therefore, the proposed method is not only easy to implement but is a realistic method for the calibration of BSVS with optical filter. Changing the size of elliptical curves projected on the target solves the difficulty of applying the proposed method in different fields of view and distances. Simulative and physical experiments are conducted to validate the efficiency of the proposed method. When the field of view is approximately 400 mm × 300 mm, the proposed method can reach a calibration accuracy of 0.03 mm, which is comparable with that of Zhang’s method.

Primate vision is an active process that constructs a stable internal representation of the 3D world based on 2D sensory inputs that are inherently unstable due to incessant eye movements. We present here a mathematical framework for processing visual information for a biologically-mediated active vision stereo system with asymmetric conformal cameras. This model utilizes the geometric analysis on the Riemann sphere developed in the group-theoretic framework of the conformal camera, thus far only applicable in modeling monocular vision. The asymmetric conformal camera model constructed here includes the fovea’s asymmetric displacement on the retina and the eye’s natural crystalline lens tilt and decentration, as observed in ophthalmological diagnostics. We extend the group-theoretic framework underlying the conformal camera to the stereo system with asymmetric conformal cameras. Our numerical simulation shows that the 1 theoretical horopter curves in this stereo system are conics that well approximate the empirical longitudinal horopters of the primate vision system.

The paper presents a study on optimizing leather defect detection using the YOLOv11 model. Traditional leather processing has faced challenges in quality control, where human inspection accuracy ranges between 70% to 85%, affecting leather utilization rates and resulting in significant material waste. This research introduces an automated solution to improve defect classification and detection accuracy. The study focuses on defects commonly found in leather, such as insect larvae damage and removal cuts, using a specialized light chamber to control environmental variables. By analyzing both grain and flesh sides of the leather, the researchers demonstrated a significant increase in defect detection accuracy, with the flesh side achieving a higher classification rate. YOLOv11's dual-side analysis allowed for clearer identification of subtle defects, leading to a more efficient classification process. The results suggest that integrating advanced AI models like YOLOv11 with controlled digitization environments could drastically reduce human error and improve leather utilization, providing a scalable solution for the leather industry.

Background/Objectives: To compare functional vision parameters - related to the perception of movement - of children who were exposed to SARS-CoV-2 with those who had no contact with the virus in the womb, in postnatal life, or through the family context; Methods: A cross-sectional study was conducted with children aged from 0 to 24 months. An initial assessment was performed through a caregiver interview and participants were divided into four groups considering the nature of their contact with COVID-19. Functional parameters of vision were recorded and two independent blinded researchers performed their analysis. Statistical analysis was conducted using the software SPSS®. Data normality was verified by the Shapiro-Wilk test. The prevalence and the odds ratio of visual changes according to coronavirus exposure were analyzed, considering a p-value &lt; 0.05; Results: The Gestational COVID Group presented an increased risk for unstable fixation time, convergent strabismus, and looking discontinuity; the Familiar COVID Group was more inclined to look discontinuity on horizontal tracking. However, there were no statistically significant differences in the prevalence and odds ratios between the exposure and control groups; Conclusions: Children exposed to SARS-CoV-2 during pregnancy or in the family environment had a higher incidence of gaze discontinuity, as well as a higher rate of unstable fixation and convergent strabismus in those exposed during pregnancy. However, there were no statistically significant differences between the groups.

Increasing concerns on animal welfare in commercial pig industry include aggression between pigs as it affects their health and growth. Early detection of aggressive behaviors is essential for optimizing their living environment. A major challenge for detection is that these behaviors are observed occasionally in normal conditions. Under this circumstance, a limited amount of aggressive behavior data will lead to class imbalance issue, making it difficult to develop an effective classification model for the detection of aggressive behaviors. To overcome this problem, we approach the aggressive behavior detection problem as a case of anomaly detection rather than classification and propose a model based on anomaly detection, which can better handle unbalanced class distribution and effectively detect the infrequent, aggressive episodes of pigs. The model consists of a convolutional neural network (CNN) and a variational long short-term memory (LSTM) autoencoder. Additionally, we adopted a training method similar to weakly supervised anomaly detection and included a few aggressive behavior data in the training set for prior learning. To effectively utilize the aggressive behavior data, we created Reconstruction Loss Inversion, a novel objective function, to train the autoencoder-based model to increase the reconstruction error of aggressive behaviors by inverting the loss function. Our anomaly detection approach significantly outperforms traditional classification-based methods, effectively identifying aggressive behaviors in a natural farming environment. This method offers a robust solution for detecting aggressive animal behaviors and contributes to improving their welfare.

Form deviation generated during the milling profile process poses challenges to the precision and functionality of industrial fixtures and product manufacturing across various sectors. The inspection of contour profile quality often relies on commonly employed contact methods for measuring form deviation, but these methods frequently face limitations that can impact the reliability and overall accuracy of the inspection process. This paper introduces a novel approach, the novel intelligent inspection method (NIIM), developed to accurately inspect and categorize contour profiles in machined parts manufactured through the milling process by computer numerical control (CNC) machines. The NIIM integrates a calibration piece, a vision system (RAM−StarliteTM), and machine learning techniques to analyze the line profile and classify the quality of contour profile deformation generated during CNC milling. The calibration piece is specifically designed to identify form deviations in the contour profile during the milling process. The RAM−StarliteTM vision system captures contour profile images corresponding to curves, lines, and slopes. An algorithm generates a profile signature, extracting Fourier descriptor features from the contour profile to analyze form deviations when compared to an image reference. A feed-forward neural network is employed to classify contour profiles based on quality properties. Experimental evaluations involving 60 machined calibration pieces, resulting in 356 images for training and testing, demonstrate the accuracy and computational efficiency of the proposed NIIM for profile line tolerance inspection. Results demonstrate that the NIIM offers 96.99% accuracy, low computational requirements, 100% inspection capability, quality classification.

Background: Lutein and zeaxanthin are fat-soluble antioxidant nutrients that have evidence of beneficial effects on vision and eye health. The purpose of this study was to examine the effects of supplementation with lutein and zeaxanthin isomers (Lute-gen®) on eye health, eye strain, sleep quality, and attention in high electronic users. Study design: Two-arm, 6-month, parallel-group, randomized, double-blind, placebo-controlled trial. Methods: Seventy volunteers aged 18 to 65 who used electronic screens for more than 6 hours daily were supplemented with 10 mg of lutein and 2mg of zeaxanthin-isomers or a placebo. Outcome measures included several ophthalmic examinations comprising the Schirmer tear test, photo-stress recovery time, contrast sensitivity, tear film break-up time, and self-report measures of visual fatigue, computer vision, sleep quality and attention. Results: Compared to the placebo, lutein and zeaxanthin supplementation was associated with greater improvements in the Schirmer tear test, photo-stress recovery time, and tear film break-up time. However, there were no group differences in the change in self-report measures or contrast sensitivity. Lutein and zeaxanthin supplementation was well-tolerated, with no reports of serious adverse reactions or clinically significant changes in safety blood measures, including liver function, renal function, blood lipids, and full blood examination. Conclusions: The results from this study provide support for the beneficial effects of 6 months of lutein and zeaxanthin supplementation on visual health and dry eyes in regular users of electronic screens. However, these findings were not corroborated by group differences in the administered self-report questionnaires.

Grayscale Lithography (GSL) is an alternative approach to the standard Binary Lithography in MEMSfabrication, enabling the fabrication of complicated arbitrary 3D structures on the wafer, without the need for multiple masks and exposure steps. Despite its advantages, GSL’s effectiveness is highly dependent on controlled lab conditions, equipment consistency, and finely-tuned photoresist exposure and etching processes. This works presents a thorough investigation of the challenges of GSL for Silicon (Si) wafers and presents a detailed approach on how to minimize the fabrication inaccuracies, aiming to replicate the intended design as closely as possible. Utilizing a maskless laser writer, all aspects of the GSL as analyzed, from photoresist (PR) exposure parameters to Si etching conditions. A practical application of GSL is demonstrated in the fabricationof 4 μm deep Silicon Fresnel Lenses for Long Wave Infrared imaging (8-12 μm). The surface topography of a Fresnel lens is a good case to apply GSL as it has varying shapes and size features that need to be preserved.. The final fabricated lens profiles show good match with the initial design and demonstrate successful etching of coarse and fine features and demonstrating images taken with an LWIR camera.

In the digital age, Artificial Intelligence (AI) is emerging as a transformative force in various sectors, in-cluding medicine. This article explores the potential of AI, akin to the magical genie of Aladdin's lamp, particularly within thoracic surgery and lung cancer management. It examines AI applications like ma-chine learning and deep learning in achieving more precise diagnoses, preoperative risk assessment, and improved surgical outcomes. The challenges and advancements in AI integration, especially in computer vision and multi-modal models, are discussed alongside their impact on robotic surgery and operating room management. Despite its transformative potential, implementing AI in medicine faces challenges regarding data scarcity, interpretability issues, and ethical concerns. Collaboration between AI and medi-cal communities is essential to address these challenges and unlock the full potential of AI in revolution-izing clinical practice. This article underscores the importance of further research and interdisciplinary collaboration to ensure the safe and effective deployment of AI in real-world clinical settings.

A surprising insight from the advent of genomic sequencing was that many genes are deeply conserved during evolution. With a particular focus on genes that interact with light in animals, I explore the metaphor of genetic toolkits, which can be operationalized as lists of genes involved in a trait of interest. A fascinating observation is that genes of a toolkit are often used again and again during convergent evolution, sometimes across vast phylogenetic distances. Such a pattern in the evolution of toolkits requires three different stages: origin, maintenance, and redeployment of the genes. The functional origins of toolkit genes might often be rooted in interactions with external environments. The origins of light interacting genes in particular may be tied to ancient responses to photo-oxidative stress, inspiring questions about the extent to which the evolution of other toolkits were also impacted by stress. Maintenance of genetic toolkits over long evolutionary timescales requires gene multifunctionality to prevent gene loss when a trait of interest is absent. Finally, the deployment of toolkit genes in convergently evolved traits like eyes sometimes involves the repeated use of similar, ancient genes but other times involves different genes specific to each convergent origin. How often a particular gene family is used time and again for the same function may depend on how many possible biological solutions are available. When few solutions exist and are maintained, evolution is constrained to use the same genes over and over. However, when many different solutions are possible, the innovative possibilities of evolution are often on display. Therefore, a focus on genetic toolkits highlights the combination of legacy plus innovation that drives the evolution of biological diversity.

This paper introduces an innovative approach to lipreading, leveraging a web application designed to generate subtitles for videos where the speaker's mouth is visible and a comprehensive literature review that precedes the discussion, encompassing a thorough examination of various lipreading methods employed over the past decade. Our method employs a powerful deep learning model, featuring a 3D-convolution network and bidirectional LSTM, enabling accurate sentence-level predictions based solely on visual lip movements. With an impressive accuracy of 97%, our model is trained using pre-segmented lips regions, transformed into animated GIFs for effective pre-training. This work stands as a significant contribution to the evolving landscape of lipreading research, offering a practical and accurate solution for real-world applications.

Image classification in Remote Sensing and Geographic Information Systems (GIS) containing various land-cover classes is essential for efficient and sustainable land-use estimation, and other tasks like object detection, localization and segmentation. Deep Learning (DL) techniques have shown a tremendous potential in the GIS domain. While Convolutional Neural Networks (CNN) have dominated most of the image analysis domain, a new architecture called transformers have proved to be a unifying solution for several AI-based processing pipelines. Vision Transformers (ViT), a variant of transformers can have comparable and in some cases better accuracy than a CNN. However, they suffer from a significant drawback associated with an excessive use of training parameters. In this research we explore several modifications in the vision transformer architectures, especially MobileViT that can be optimized while boosting accuracy. To verify our proposed approach these new architectures are trained on four land-cover datasets AID, EuroSAT, UC-Merced, and WHU-RS19. Experiments reveal that combination of lightweight convolutional layers including ShuffleNet along with depthwise separable convolutions and average pooling can reduce the trainable parameters by 17.85% and yet achieve higher accuracy than the base MobileViT. It is also observed that utilizing a combination of convolution layers along with multi-headed self attention layers in MobileViT variants provide better performance in capturing local and global features unlike the standalone ViT architecture that utilizes almost 95% more parameters than the proposed MobileViT variant.

The relative position of the orchard robot to the rows of fruit trees is an important parameter for achieving autonomous navigations. The current methods for estimating the position parameters between rows of orchard robots obtain low parameter accuracy, and to address this problem, this paper proposes a machine vision-based method for detecting the relative position of orchard robots and fruit tree rows. Firstly, the fruit tree trunk is identified based on the improved YOLOv4 model; secondly, the camera coordinates of the tree trunk are calculated from the principle of binocular camera triangulation, and the ground projection coordinates of the tree trunk are obtained through coordinate conversion; finally, the midpoints of the projection coordinates of different sides are combined and the navigation path is obtained by linear fitting with the least squares method, and the position parameters of the orchard robot are obtained through calculation. The experimental results show that the average accuracy and average recall of the improved YOLOv4 model for fruit tree trunk detection are 97.05% and 95.42%, respectively, which are 5.92 and 7.91 percentage points higher than those of the original YOLOv4 model. The average errors of heading angle and lateral deviation estimates obtained based on the method in this paper are 0.57° and 0.02 m. The method can accurately calculate heading angle and lateral deviation values at different positions between rows, and can provide a reference for autonomous visual navigation of orchard robots.

The traditional understanding of brain function has predominantly focused on chemical and electrical processes. However, new research in fruit fly (Drosophila) binocular vision reveals ultrafast photomechanical photoreceptor movements significantly enhance information processing, thereby impacting a fly's perception of its environment and behaviour. The coding advantages resulting from these mechanical processes suggest that similar physical motion-based coding strategies may affect neural communication ubiquitously. The theory of neural morphodynamics proposes that rapid biomechanical movements and microstructural changes at the level of neurons and synapses enhance the speed and efficiency of sensory information processing, intrinsic thoughts, and actions by regulating neural information in a phasic manner. We propose that morphodynamic information processing evolved to drive predictive coding, synchronising cognitive processes across neural networks to match the behavioural demands at hand effectively.