1. Introduction

2. Materials and Methods

2.2. Data Augmentations

The limitations of the poultry farm environment can result in a singular environment within the collected laying hen image dataset and a limited number of collections. To better adapt this dataset to various breeding environments and deep learning models, we have employed data augmentation techniques to enrich the existing data. Data augmentation considers the impact of factors such as the breeding environment, collection equipment, and chicken head features on the generalizability of the dataset, making the augmented dataset more accurate and reliable in model expression.

2.2.1. Rotation

Three rotation methods are used for data augmentation: rotating the image's red channel by 45°, shearing the image in a 16° direction to create a tilt effect, and rotating the image counterclockwise by 20°. Rotation augmentation is primarily used for enhancing RGB-TIR images, and the enhanced RGB images remain corresponding to the TIR images.

2.2.2. R-Value (Redness) Enhancement

The key to accurately identifying laying hens is precisely determining the chicken head. Since the comb colour of a laying hen is red, enhancing the value of the red channel in the RGB image will facilitate the differentiation between the target and the background. We randomly increase the R channel of each pixel of the image by a value between 10 and 100, as follows:

where R_enhanced is the red channel value of the pixel at position (x, y) in the enhanced image; △R is a random variable ranging from [10, 100]; R_raw is the red channel value of the pixel at position (x, y) in the raw image.

2.2.3. Hue Enhancement

Hue enhancement can make the model more robust during training and capable of dealing with the impact of hue variation. First, convert the image from the RGB colour space to the HSV colour space. Then, select the hue channel H and add a random value. The hue enhancement is calculated as follows:

where H_enhanced is the enhanced H channel value; △H is a random variable ranging from [25, 90]; H_raw is the H channel value of the raw image. Since hue H is periodic (between 0 and 360 degrees), periodic adjustment is made to H_enhanced during the enhancement process to ensure it remains within the valid hue range.

2.2.4. Brightness Enhancement

Brightness enhancement is used to simulate lighting variations in different poultry farms, variations in lighting at different heights of chicken cages within the same farm, and imaging differences in image collection by different cameras, thereby increasing data complexity. A multiplication factor is applied to each channel of every pixel in the image, calculated as follows:

where P_enhanced is the pixel value of the enhanced image; k is a random variable ranging from [0.5, 1.5]; P_raw is the pixel value of the raw image. When k is less than 1, the enhanced image is darker than the raw image; when k is greater than 1, the enhanced image is brighter than the raw image.

2.2.5. Edge Enhancement

Edge enhancement is used to make the edges of the image more pronounced, which can improve the image's visual effect. Incorporating edge enhancement as a step in the laying hen data augmentation process during machine learning model training can enhance the model's ability to recognize edge features of the chicken head (especially the comb part).

2.2.6. Gaussian Noise

Gaussian noise is added for data augmentation to simulate noise interference during the image collection process in the henhouse. A random noise value following a Gaussian distribution is added to each raw image pixel. The enhanced pixel value is calculated as follows:

where P_Gnew is the pixel value of the image after Gaussian enhancement; the clip function limits the value to the range of 0 to 255; N is the noise value, randomly drawn from a Gaussian distribution with a standard deviation of 51.

2.2.7. Laplacian Noise

Laplacian noise enhancement is added to simulate interference caused by low lighting and high compression rates during image collection in the henhouse environment. A random noise value following a Laplacian distribution is generated for each pixel of the image, calculated as follows:

where P_Lnew is the pixel value of the image after Laplacian enhancement; the clip function limits the value to the range of 0 to 255; M is the noise value, randomly drawn from a Laplacian distribution with a scale of 51.

2.2.8. Poisson Noise

Statistical noise following a Poisson distribution is added to enhance the robustness of the model, calculated as follows:

where P_Pnew is the pixel value of the image after Poisson enhancement; the clip function limits the value to the range of 0 to 255; λ is the noise value, randomly drawn from a Poisson distribution with a scale of 40.

2.2.9. Salt-and-Pepper Noise

Salt-and-pepper noise is added for data augmentation to simulate interference in images due to low-quality cameras, wireless transmission errors, storage media damage, and other similar issues. The proportion of pixels covered by noise points in the image is defined as 10%, and within this 10%, white (salt) or black (pepper) pixel points are randomly introduced.

The raw images were processed using the methods above, as illustrated in Figure 3. Before data augmentation, there were 30,993 raw images, with 2,359 xml, json, and txt data labels each. It is important to note that the laying hen data obtained through the above data augmentation methods include not only the laying hen images but also the corresponding xml, json, and txt data labels for the augmented laying hen images, or the TIR images corresponding to the RGB images. This allows for the support of various model training tasks. After data augmentation, there are 61,133 images and 63,693 data labels (21,231 XML, JSON, and TXT data labels each).

3. Results and Analysis

3.1. Data Description

This dataset encompasses three categories of imagery data about laying hens: RGB image data, paired RGB and Thermal Infrared (TIR) image data, and RGB image data incorporating chicken head annotation information. Specifically, the dataset comprises 5,226 RGB images of individual hens, constituting 8% of the total dataset. The paired RGB and TIR images number 34,676, representing 57% of the dataset. The RGB images annotated with chicken head information total 21,231, accounting for 35% of the dataset.

3.1.1. Data Environment

A robust dataset must consider both the quantity and quality of the images and the diversity of the environment [14], taking into account a variety of photographic perspectives, lighting conditions, and background contexts. Figure 4, Figure 5 and Figure 6 depict the range of complex environments captured within the dataset.

(1) We collected data on individual laying hens in a natural environment, as illustrated in Figure 4. During photography, complete and partial images of the hens were captured under various natural lighting and background conditions. The data obtained from the natural setting are conducive to analyzing the morphology of free-range laying hens [15] and assessing their health state.

(2) The activity states of caged laying hens were photographed, considering the variations in cage lighting and camera angles, as depicted in Figure 5. The dataset of caged laying hens can be utilized to analyze flock activity levels, detect deceased hens within cages, and calculate hen population numbers.

(3) An innovative image acquisition technology that fuses RGB images with TIR images was introduced [16], as shown in Figure 6. Corresponding RGB and TIR images were taken at different distances, angles, and with varying numbers of hens, capturing multimodal image data. The image fusion technique overcomes interference caused by camera shake in RGB images and provides data support for studying infectious diseases among hen flocks.

Due to the slightly wider field of view in TIR image compared to RGB images (with the TIR image resolution being lower than that of the RGB images), we employed image correction [17] methods to transform the coordinates and dimensions of the RGB images onto the TIR images to align these two kinds of image. Using the xywh notation for image correction, the calculation method for the two-dimensional coordinates of pixels and the dimensions of length and width in the TIR image is as follows:

where x represents the x-coordinate in the RGB image with the top-left corner as the origin; y represents the y-coordinate in the RGB image with the top-left corner as the origin; w denotes the raw width of the RGB image; h denotes the raw height of the RGB image; T_x signifies the x-coordinate in the transformed TIR image; T_y signifies the y-coordinate in the transformed TIR image; T_w denotes the width of the transformed TIR image; T_h denotes the height of the transformed TIR image; k_x is the scaling factor for the x-coordinate, taken as 0.09661 in this paper; k_y is the scaling factor for the y-coordinate, taken as 0.09558 in this paper; k_w is the scaling factor for the image width, taken as 0.09672 in this paper; k_h is the scaling factor for the image height, taken as 0.09618 in this paper; b_x is the adjustment factor for the x-coordinate offset, taken as 8.66434 in this paper; b_y is the adjustment factor for the y-coordinate offset, taken as 10.1642 in this paper.

To ensure that the transformed coordinates and dimensions do not exceed the boundaries of the TIR image, boundary checks and dimension adjustments are performed on the corrected TIR images. The adjusted width T_adjust(w) of the TIR image is calculated as follows:

The adjusted height T_adjust(h) of the TIR image is calculated as follows:

Furthermore, data augmentation techniques have been applied to the dataset to expand it, including operations such as rotation, hue transformation, brightness transformation, and the addition of noise.

3.1.2. Annotations and Labels

In the collected images of caged laying hen flocks, annotations for chicken heads were performed, with the data annotation task accomplished by 10 data annotators, ensuring the precision and standardization of the data. The annotations provide location information for the heads of the hens within the images, marked with rectangular frames, using the LabelImg annotation software.

As shown in Figure 7, for the same image, we provide annotation information in three different formats: xml, txt, and json. The xml format annotation files include the images' size, category, and location information, adhering to the annotation standards of datasets such as PASCAL VOC and YOLO. The txt format is a plain text file containing only category and location information, commonly used for YOLO format annotations, and is characterized by its simplicity and ease of parsing. The json format annotation files include image size, category, location, and other information, with a level of lightness between xml and txt formats. The three data annotation formats we provide can meet the needs of different application scenarios.

3.1.3. Data Folders

The structure of the data folders is shown in Figure 8 and Figure 9. In Figure 9, the folders from 09RBGtxt+xml+json-2359 to 17SPNoisetxt+xml+json-2359 have divided the dataset in a ratio of 7:2:1, with the training set (folder train1653) accounting for 70%, the validation set (folder val471) accounting for 20%, and the test set (folder test235) accounting for 10%. Each folder for the training, validation, and test sets contains four subfolders: img, json, txt, and xml, for saving the corresponding formats of ten images or labels. Figure 10 and Figure 11 shows the file count statistics within the BClayinghens dataset.

3.1.4. Value of the Data

The dataset is applicable for health monitoring and behavioural analysis of caged laying hens and research on chicken head detection and counting of caged hens. It is particularly suitable for classification studies of hens based on deep learning convolutional neural networks.

Multimodal image data from visible light and thermal infrared are provided, supporting research on health monitoring and behaviour recognition of hens through image fusion techniques. The fused images of hens are especially useful for body temperature detection, aiding disease prevention and control efforts in commercial poultry farming.

Different morphological images of individual hens are supplied, which can be utilized for health assessments of laying hens.

The relevant information regarding the BClayinghens dataset is presented in Table 3.

3.2. Chicken Head Detection

3.2.1. Comparison of Detection Performance of Different Algorithms

The chicken inspection robot in the henhouse should possess high real-time performance during actual use, necessitating the selection of target detection models with superior real-time capabilities for deployment. We have chosen mainstream and novel real-time target detection models to test the dataset, thereby verifying the dataset's outstanding performance across various target detection algorithms. The RT-DETR model [18] and the YOLO series target detection models (YOLOv5 [19], YOLOv8 [20], YOLOv9 [21], YOLOv10 [22]) are utilized as baseline testing methods for chicken head recognition. Model training is implemented on a Linux server, using an Intel(R) Xeon(R) Platinum 8375c CPU @ 2.90GHz processor, an NVIDIA RTX A6000 graphics card, CUDA version 11.3, Python version 3.8.18, and the Pytorch deep learning framework, Pytorch version 1.12.0. None of the experiments uses pre-trained weights, with a batch size set to 48 and the number of epochs set to 200.

We selected 2,359 images from the BClayinghens dataset to verify the chicken head detection performance of the dataset across different algorithms. Figure 12 illustrates the distribution of the dataset used during the chicken head detection training process, where the coordinates are normalized values. The x and y coordinates, with a value of 0, represent the upper left corner of the image in the dataset, and a value of 1 represents the upper right corner. It can be observed that the x-coordinates of the chicken head target boxes in the selected data are relatively evenly distributed, the y-coordinates are concentrated in the central area of the image, and the width and height of the target boxes follow a normal distribution. Additionally, a denser concentration of points is shown in areas with smaller widths and heights within the image, indicating a strong commonality in certain specific sizes or positions of the chicken head targets in the dataset. This aggregation can help the model learn the target position and size distribution more accurately, thereby improving detection accuracy.

Figure 13, Figure 14 and Figure 15 illustrate the variations in loss values and precision during the training process of the RT-DETR and YOLO series chicken head detection models. Given the complexity of object detection tasks, models require optimization across multiple dimensions. To enable models to comprehensively learn various aspects of object detection tasks, different algorithms enhance overall model performance by employing a design philosophy that combines multiple loss functions. Specifically, in Figure 13, Figure 14, and Figure 15, the loss calculation for RT-DETR includes giou_loss (Generalized Intersection over Union Loss [23]), cls_loss (classification loss), and l1_loss (Mean Absolute Error) [18]; YOLOv5 comprises box_loss (bounding box loss) and obj_loss (object classification loss); YOLOv8 includes box_loss, cls_loss, and dfl_loss (distribution focal loss); YOLOv9 consists of box_loss, cls_loss, and dfl_loss; YOLOv10 utilizes a dual label assignment strategy, encompassing box_om, cls_om, dfl_om, box_oo, cls_oo, and dfl_oo [22]. Overall, as the number of training epochs increases, the loss values of all algorithms gradually decrease and stabilize while precision gradually improves and stabilizes.

The detection effects of the models are comprehensively evaluated using Recall and mAP_0.5 (mean Average Precision at Intersection over Union of 0.5), as shown in Figure 16. It can be observed that the dataset exhibits a high degree of average precision across different models and possesses a strong capability to identify chicken head targets. In summary, the BClayinghens dataset is highly quality and adaptable to various object detection algorithms.

3.2.2. Visual Analysis of Chicken Head Detection

Due to the interpretability limitations of deep learning algorithms, we conducted a visual analysis using the inference results of chicken head detection. Figure 17, Figure 18, Figure 19, Figure 20 and Figure 21 demonstrate the recognition effects of different object detection algorithms on laying hen images.

In general, all algorithms can accurately identify the chicken head targets for images of laying hens in various poses with high confidence. The predicted boxes can fully encompass the chicken head area without any missed detections. The RT-DETR series algorithms have chicken head detection confidence, mostly around 0.9, with relatively stable detection results. The YOLOv5 series algorithms have the highest sensitivity in identifying chicken heads, with detection confidence frequently reaching up to 0.98. The YOLOv5 series algorithms have the best learning capability for chicken head features but are prone to false positives in practical applications. The YOLOv8 series algorithms have the highest confidence at 0.97, and the YOLOv9 series algorithms have the highest confidence at 0.98. However, considering the loss values and precision curves during the training process, we found that the YOLOv8 series algorithms consistently have lower loss values than YOLOv9, indicating that the YOLOv8 series algorithms have an advantage in chicken head detection. In comparison, the YOLOv10 series algorithms are less capable of recognizing chicken head targets than others, with the lowest confidence for single chicken head recognition at 0.3. In conclusion, different mainstream object detection models can meet the needs of chicken head identification and provide great convenience for subsequent chicken head temperature detection and deployment on the robot side.

3.2.3. Visual Analysis of Chicken Head Temperature Identification

After applying the object detection algorithm to identify the chicken head using the chicken inspection robot in the henhouse, the position of the chicken head is sent to the information collection and control system in the xywh (coordinates (x, y), width w, height h) format, as shown in equation (7), equation (8), and equation (9). The information collection and control system then invokes the chicken head temperature recognition algorithm and displays the chicken head temperature recognition results on the robot's visualization interface.

During the experimental process, we calibrated the temperature recognition system of the chicken inspection robot in the henhouse to further improve the temperature detection accuracy. The method for correcting the chicken head temperature detection is as follows:

where T_t is the raw chicken head temperature detected by the infrared thermal camera, and T_a is the corrected actual chicken head temperature.

When the detection and recognition results of the chicken head are accurate, the chicken inspection robot in the henhouse can call the temperature detection algorithm to calculate the chicken head temperature. Since the RT-DETR above and YOLO series object detection algorithms can accurately identify the chicken head targets, the effect of chicken head temperature recognition is only related to the temperature recognition system. The YOLOv5n algorithm, which has the highest sensitivity for chicken head detection in the YOLOv5 series, was deployed on the chicken inspection robot in the henhouse to test the effect of chicken head temperature detection. The YOLOv5n algorithm can maintain a fast speed while providing accuracy suitable for edge devices, and the effect of chicken head temperature recognition is shown in Figure 22.

In Figure 22, the chicken head temperature is generally around 40°C, the normal temperature for a chicken head, indicating a high degree of temperature recognition accuracy. Moreover, the movement of the chicken and the robot has a minimal impact on the recognition of the chicken head temperature, with the maximum difference in the temperature recognition of the same laying hen's head being 0.4°C and the minimum being 0.1°C. The dataset shows good performance in chicken head temperature recognition, which can provide a strong preference for the refined management and prevention of diseases in the henhouse.

4. Conclusions

References

1. Khanal, R.; Choi, Y.; Lee, J. , Transforming Poultry Farming: A Pyramid Vision Transformer Approach for Accurate Chicken Counting in Smart Farm Environments. Sens. 2024, 24, 2977. [Google Scholar] [CrossRef] [PubMed]
2. Geffen, O.; Yitzhaky, Y.; Barchilon, N.; Druyan, S.; Halachmi, I. , A machine vision system to detect and count laying hens in battery cages. Anim. 2020, 14, 2628–2634. [Google Scholar] [CrossRef] [PubMed]
3. Bakar, M.; Ker, P. J.; Tang, S. G. H.; Baharuddin, M. Z.; Lee, H. J.; Omar, A. R. , Translating conventional wisdom on chicken comb color into automated monitoring of disease-infected chicken using chromaticity-based machine learning models. Front. Vet. Sci. 2023, 10. [Google Scholar] [CrossRef] [PubMed]
4. Yu, Z. W.; Wan, L. Q.; Yousaf, K.; Lin, H.; Zhang, J.; Jiao, H. C.; Yan, G. Q.; Song, Z. H.; Tian, F. Y. , An enhancement algorithm for head characteristics of caged chickens detection based on cyclic consistent migration neural network. Poult. Sci. 2024, (6), 103663. [Google Scholar] [CrossRef] [PubMed]
5. Wu, Z. L.; Zhang, T. M.; Fang, C.; Yang, J. K.; Ma, C.; Zheng, H. K.; Zhao, H. Z. , Super-resolution fusion optimization for poultry detection: a multi-object chicken detection method. J. Anim. Sci. 2023, 101, 1–15. [Google Scholar] [CrossRef]
6. Campbell, M.; Miller, P.; Díaz-Chito, K.; Hong, X.; McLaughlin, N.; Parvinzamir, F.; Del Rincón, J. M.; O'Connell, N. , A computer vision approach to monitor activity in commercial broiler chickens using trajectory-based clustering analysis. Comput. Electron. Agric. 2024, 217, 108591. [Google Scholar] [CrossRef]
7. Nasiri, A.; Yoder, J.; Zhao, Y.; Hawkins, S.; Prado, M.; Gan, H. , Pose estimation-based lameness recognition in broiler using CNN-LSTM network. Comput. Electron. Agric. 2022, 197, 106931. [Google Scholar] [CrossRef]
8. Yao, Y. Z.; Yu, H. Y.; Mu, J.; Li, J.; Pu, H. B. , Estimation of the Gender Ratio of Chickens Based on Computer Vision: Dataset and Exploration. Entropy 2020, 22, 719. [Google Scholar] [CrossRef] [PubMed]
9. Adebayo, S.; Aworinde, H. O.; Akinwunmi, A. O.; Alabi, O. M.; Ayandiji, A.; Sakpere, A. B.; Adeyemo, A.; Oyebamiji, A. K.; Olaide, O.; Kizito, E. , Enhancing poultry health management through machine learning-based analysis of vocalization signals dataset. Data Brief 2023, 50, 109528. [Google Scholar] [CrossRef]
10. Aworinde, H. O.; Adebayo, S.; Akinwunmi, A. O.; Alabi, O. M.; Ayandiji, A.; Sakpere, A. B.; Oyebamiji, A. K.; Olaide, O.; Kizito, E.; Olawuyi, A. J. , Poultry fecal imagery dataset for health status prediction: A case of South-West Nigeria. Data Brief 2023, 50, 109517. [Google Scholar] [CrossRef]
11. Han, H.; Xue, X.; Li, Q.; Gao, H.; Wang, R.; Jiang, R.; Ren, Z.; Meng, R.; Li, M.; Guo, Y.; Liu, Y.; Ma, W. , Pig-ear detection from the thermal infrared image based on improved YOLOv8n. Intell. Rob. 2024, 4, 20–38. [Google Scholar] [CrossRef]
12. Dlesk, A.; Vach, K.; Pavelka, K. , Photogrammetric Co-Processing of Thermal Infrared Images and RGB Images. Sens. 2022, 22, 1655. [Google Scholar] [CrossRef]
13. Hongwei, L.; Yang, J.; Manlu, L.; Xinbin, Z.; Jianwen, H.; Haoxiang, S. , Path planning with obstacle avoidance for soft robots based on improved particle swarm optimization algorithm. Intell. Rob. 2023, 3, 565–80. [Google Scholar]
14. Xinxing, C.; Yuxuan, W.; Chuheng, C.; Yuquan, L.; Chenglong, F. , Towards environment perception for walking-aid robots: an improved staircase shape feature extraction method. Intell. Rob. 2024, 4, 179–95. [Google Scholar]
15. Tan, X. J.; Yin, C. C.; Li, X. X.; Cai, M. R.; Chen, W. H.; Liu, Z.; Wang, J. S.; Han, Y. X. , SY-Track: A tracking tool for measuring chicken flock activity level. Comput. Electron. Agric. 2024, 217, 108603. [Google Scholar] [CrossRef]
16. Ma, W. H.; Wang, K.; Li, J. W.; Yang, S. X.; Li, J. F.; Song, L. P.; Li, Q. F. , Infrared and Visible Image Fusion Technology and Application: A Review. Sens. 2023, 23, 599. [Google Scholar] [CrossRef] [PubMed]
17. Fan, J.; Zheng, E.; He, Y.; Yang, J. , A Cross-View Geo-Localization Algorithm Using UAV Image and Satellite Image. Sens. 2024, 24, 3719. [Google Scholar] [CrossRef]
18. Lv, W.; Xu, S.; Zhao, Y.; Wang, G.; Wei, J.; Cui, C.; Du, Y.; Dang, Q.; Liu, Y. DETRs Beat YOLOs on Real-time Object Detection. ArXiv 2023, abs/2304.08069.
19. Terven, J. R.; Esparza, D. M. C. A Comprehensive Review of YOLO: From YOLOv1 to YOLOv8 and Beyond. ArXiv 2023, abs/2304.00501.
20. Varghese, R.; S, M. In YOLOv8: A Novel Object Detection Algorithm with Enhanced Performance and Robustness, 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), 18-19 April 2024, 2024; 2024; pp 1-6.
21. Wang, C.-Y.; Yeh, I.-H.; Liao, H. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. ArXiv 2024, abs/2402.13616.
22. Wang, A.; Chen, H.; Liu, L.; Chen, K.; Lin, Z.; Han, J.; Ding, G. YOLOv10: Real-Time End-to-End Object Detection. ArXiv 2024, abs/2405.14458.
23. Rezatofighi, H.; Tsoi, N.; Gwak, J.; Sadeghian, A.; Reid, I.; Savarese, S. In Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15-20 June 2019, 2019; 2019; pp 658-666. [CrossRef]