This section presents the quantitative and qualitative experiment results regarding the contour profile inspection, a machine learning analysis of classifications, a comparison of the NIIM between the CMM contact method, and a discussion.
4.1. Contour Profile Inspection
The results of the contour profile inspection are conducted by generating the signature spectrum evaluation, involving the examination of 60 signatures for each profile, encompassing four distinct materials. They are comparing the spectrum of the calibration piece versus the spectrum of the reference. Through the subtraction of the two spectrum signals to generate a third spectrum signal that defines the profile variation.
The analysis of an aluminum sample is renowned for its superior dimensional precision, making it an ideal material for evaluating profile accuracy. The examination of signature profiles
,
, and
, generate the spectrum signals
shown in
Figure 20,
Figure 21 and
Figure 22.
The values depicted in
Figure 23,
Figure 24 and
Figure 25 reveal the result of the third signal spectrums for each profile signature. The maximum peaks:
= 0.623 for
,
=0.556 for
, and
=0.489 for
indicates significant form deviations within each signature. However, they fall within the tolerance zone limits. The
numerical representations serve as a quantitative indicator, offering a precise contour profile inspection of the machined parts in each signature
. The significance of
values lies in the ability of the NIIM to quantitatively assess the line profile tolerance and characterize the contour profile quality, giving an impact of machining on the accuracy of each profile type, ensuring a thorough understanding of the impact of the NIIM on profile inspection accuracy.
Form deviation results are presented in a probability plot, demonstrating a systematic approach that guarantees close correspondence between the derived inspection and the genuine physical attributes of the profile. The results obtained from the analysis of each contour based on the study of its shape deviation are shown in the following subsection.
4.1.1. Form Deviation Analysis on Curve Profile
Figure 26 outlines the trends in form deviation, showing the correlation with
values obtained from the
. The materials involved in the analysis include aluminum, nylon, delrin, and steel.
It is crucial to highlight that serves as an indicator for quantifying the form deviations in the curve profiles of these materials. A reduced value indicates a closer adherence to the ideal curve, implying minimal form deviations. Conversely, a higher value suggests more pronounced deviations, potentially exceeding the tolerance limits.
Aluminum, with a mean of =0.1726, exhibits the least form deviations among the materials under consideration. Results show that the curve profile of aluminum closely matches the ideal curve, reflecting a high degree of accuracy and precision in its manufacturing.
Nylon, with a mean =0.7359, shows moderate form deviations compared to aluminum. While nylon may have some imperfections in its curve profile, the deviation is still within an acceptable tolerance, and the material can be considered suitable for certain applications.
With a mean =0.6067, Delrin falls within a similar range as nylon regarding form deviations. Results imply that delrin also exhibits a moderate level of accuracy in its curve profile, making it a viable option for specific applications where such deviations are acceptable.
On the other hand, steel stands out of tolerance with a relatively higher mean value of 1.16, indicating more significant form deviations than the other materials. Results may be attributed to the inherent characteristics of steel or the specific manufacturing processes involved, which result in a curve profile that deviates noticeably from the ideal.
In summary, aluminum demonstrates the least deviations, nylon and delrin exhibit moderate levels, and steel shows a higher form deviation degree.
4.1.2. Form Deviation Analysis on Line Profile
The line profile analysis presented in
Figure 27 outlines the trends in form deviation, showing the correlation with
values obtained from the
. The materials involved in this analysis include aluminum, nylon, delrin, and steel.
Starting with aluminum, which has = 0.3614, the analysis reveals moderate form deviations in its line profile. Results show that aluminum maintains relatively good accuracy in its line profile, but there are discernible deviations from the ideal form. This information is crucial for applications where precision in the line profile is a critical factor.
Nylon, with = 0.2518, exhibits lower form deviations than aluminum. Results show that nylon’s line profile is closer to the ideal form, indicating higher accuracy and potentially making it suitable for applications where stringent form requirements are essential.
Delrin stands out with a significantly higher = 1.078, signaling a more substantial level of form deviations in its line profile. This information is essential for applications where precision and conformity to the ideal form are critical, as the deviations in delrin may impact its suitability for certain use cases.
Steel, with = 0.9205, demonstrates considerable form deviations in its line profile. Results show that the line profile of steel deviates noticeably from the ideal form, indicating potential challenges in applications that require high precision and accuracy.
In summary, the line profile analysis demonstrates the least deviations in nylon, the strong correlation between nylon and aluminum, and the highest form of delrin deformation. The correlation with changing machining conditions emphasizes the need for a comprehensive understanding of material properties and machining parameters to achieve the desired line profile tolerances in various applications.
4.1.3. Form Deviation Analysis on Slope Profile
The slope profile analysis, as depicted in
Figure 28 outlines the trends in form deviation, showing the correlation with
values obtained from the
. The materials involved in this analysis include aluminum, nylon, delrin, and steel.
Beginning with aluminum, which has = 0.3771, the slope profile analysis reveals moderate form deviations. Results show that the slope profile of aluminum exhibits noticeable variations from the ideal form, emphasizing the importance of understanding these deviations in applications where slope precision is critical.
Nylon, with = 0.8373, exhibits higher form deviations in its slope profile than aluminum. Results indicate that nylon is out of tolerance for essential applications where slope accuracy is paramount. The nylon deviations may impact suitability for certain use cases requiring precise slope control.
With = 0.5769, Delrin presents a moderate range of form deviations in its slope profile. Understanding these deviations is crucial for applications that demand a specific slope precision, as delrin may display variations that must be considered in the design and manufacturing processes.
Steel, with = 0.5521, demonstrates a relatively low level of form deviations in its slope profile. Results show that the slope profile of Steel is closer to the ideal form, making it potentially well-suited for applications requiring precise slope control.
Similar to the previous analyses, the correlation between varying machining conditions and distinct levels of profile form deviations is highlighted in the slope profile analysis. Results demonstrate correlation and underscore the importance of considering the influence of machining parameters, such as cutting speeds, feeds, and tool geometries, on the slope precision of these materials. Adjustments in machining conditions may be explored in future research to minimize slope deviations and enhance the overall accuracy of slope profiles.
4.2. Machine Learning Results
The machine learning models rFFNN, mFFNN, kFFNN, CNNG, CNNP, SVM, LSTM, AlexNet, ResNet, VGG-16, and Inception were evaluated to determine which feature vector and network are the best option for shape and quality contour profile classification.
According to [
3,
4,
6,
15,
16] y [
38], the metrics used to know the performance is accuracy for training and test (TAcc and PAcc, respectively), and Confusion matrices, defined in [
13,
14]. Also, we include computational cost metrics such as frame per second (FPS) and the number of parameters (NPam) to know which network is feasible for real-time processing in industrial environments for inspection applications.
Contour profile Classification. The NIIM classifies contour calibration pieces into Curve, Line, and Slope categories in terms of quality. The feature vector
has a size of 508 elements. In contrast, the feature vectors
and
have a size of 150 because, according to our experiments, the frequencies
are enough to represent the shape in the frequency domain. The networks rFFNN, mFFNN, and kFFNN have six neurons in the hidden layers and were tested with five validation experiments.
Table 6 shows the results where the networks rFFNN, mFFNN, kFFNN, CNNG, and CNNP achieved 100% for training and testing. However, mFFNN and kFFNN are faster and have fewer parameters than the others.
Figure 29 presents the confusion matrices of rFFNN, mFFNN, kFFNN, CNNG, and CNNP, where the performance is the same. However, kFFNN presents less overfitting during the training. Regarding converging epochs, rFFNN requires 56 epochs, mFFNN 28, kFFNN 25, CNNG 15, and CNNP 10. CNNG and CNNP require fewer epochs to converge than the other networks, but they present more overfitting than kFFNN. SVM, LSTM, AlexNet, VGG-16, ResNet, and Inception networks require 300 epochs.
CNNG learns features based on the piece surface and the illumination of the vision system, while CNNP learns features of the profile shape. For example,
Figure 30 shows the second convolution activation maps, where the propagation of a curve image generates activation divided into the piece surface and the illumination.
Figure 31 shows the activation maps of the second convolution layer, where the propagation of a curve image generates activation with similar patterns to the Laplacian filtering.
rFFNN, mFFNN, CNNG, and CNNP achieve good results because they learn the shape features that have , , , and . Additionally, kFFNN generates good results because have form deviation properties derived from the three machined processes that generate Curve, Line, and Slope profiles. Added to this, kFFNN and mFFNN generate the lowest computational cost. The models SVM, LSTM, AlexNet, VGG-16, ResNet, and Inception generate less accuracy than the FFNNs and CNNs proposed in this work. SVM has three lineal discriminant functions, each with 251 parameters (one for each feature vector and the bias).
However, the low accuracy is due to the shape classes being overlapped in the frequency feature space. The LSTM has low accuracy because it interprets a false causal relation between frequencies. The CNNs AlexNet, VGG-16, ResNet, and Inception have many parameters that codify many feature maps during propagation with irrelevant information and properties for shape classification.
Quality classification. The NIIM classifies calibration pieces into normal and defect-calibration pieces. For task classification, the feature vector has a size of 508 and the feature vectors and have a size of 168 because, according to our experiments, the frequencies are enough to represent the quality of the contour profile in the frequency domain.
The contour profile signature was selected because, according to [
40], profile quality defects and form deviations affect wear resistance, tolerance accuracy, shape deformation, fatigue resistance, corrosion resistance, and fluid passage in the milling part.
An expert generates the ground truths for task classification by dividing the dataset considering the geometric dimension and tolerances according to ASME Y14.5.1M and ISO/TC 213 for machine learning purposes.
Concerning the metrics of the experiments and results, we add the F-measure (F1) metric because it is used in many works as used in [
13,
14] and is defined as the harmonic mean between precision and recall in binary classifications.
The NIIM considers the true positive when a normal piece is classified as normal. The networks rFFNN, mFFNN, and kFFNN have 48 neurons in the hidden layers, and they were tested with five validation experiments. Regarding converging epochs, rFFNN requires 69, mFFNN 100, kFFNN 110, CNNG 32, and CNNP 32. CNNG and CNNP require fewer epochs to converge than the other networks, but after 32 epochs, they cannot improve the accuracy. SVM, LSTM, AlexNet, VGG-16, ResNet, and Inception networks require 300 epochs.
Table 7 shows the results of the experiments in quality classification, where kFFNN presents the best performance and computational cost because
describes the form deviation generated by the machined process. rFFNN and mFFNN had low performance in the test and training because the input feature vectors of
and
concentrate most of their energy on shape features. Hence, these vectors feed the ANN with few form deviation features. The CNN networks had good accuracy during the training, but the accuracy during the test decreased significantly. This performance is because of overfitting generated by many features unrelated to the contour profile.
Figure 32 and
Figure 33 show the second layer activation maps of CNNG and CNNP trained considering the normal and defective classes, where the features learned are focused on the shape properties. Then, rFFNN, mFFNN, CNNG, and CNNP process shape rather than form deviation features because the energy of the features is mainly concentrated in shape features. kFFNN achieves good results and has the lowest computational cost because it has only deviation features that describe the milling process in the piece.
Figure 34 shows the contour profile classification confusion matrices. mFFNN and rFFNN classified all samples as normal, and CNNG and CNNP classified half of the defective samples as normal.
SVM, LSTM, AlexNet, VGG-16, ResNet, and Inception models generate less accuracy and F1 than the FFNN and CNNs proposed in the NIIM. SVM has low accuracy because the shape classes overlap in the frequency feature space. The LSTM has low accuracy because it interprets a false causal relation between frequencies. The CNNs AlexNet, VGG-16, ResNet, and Inception are networks with many parameters that codify many feature maps during propagation with irrelevant information and properties for quality classification.
4.2.1. Computational Cost
Table 6 and
Table 8 show that kFFNN and mFFNN achieve the best FPS and Number of variables, and
Table 9 presents the relation of NPam of CNNG, CNNP, and rFFNN against NPam of kFFNN and mFFNN. The differences in the computational costs are because
and
have 307,200 elements and many abstract features that the CNNs need to codify. The rFFNN processes the signals
, which have 600 to 640 elements and are time-domain signals with shape, form deviation, and resolution error features. On the other hand,
and
have 150 or 170 elements that represent the shape and form deviation features.
The computational performance of renowned networks such as AlexNet [
45], VGG-16, ResNet [
46], comes at a higher cost, resulting in accuracy levels below 85% for shape and quality classification. The results are attributed to the networks’ tendency to learn redundant features, which may not contribute significantly to the classification task. Moreover, their computational demands render them impractical for real-time processing, as the frames per second (FPS) achieved are not conducive to swift and efficient analysis.
According to the machine learning experiments, any ANN with , , or and generate good performance in shape classification, but regular results in quality classification. The results are obtained because these inputs concentrate the energy in shape information and have little energy in form deviation information. Then, the ANN cannot learn from the form deviation features.
On the other hand,
and the FFNN generate the best computational cost and good performance in shape and quality classification. These results are obtained because
have only deviation features, and the tolerance zone is represented with 100 frequency elements (it can be seen in
Section 2.3.3. The obtained performance of
is because the form deviation feature vector is a signature of the CNC milling process that generates the profile of the calibration piece.
Also, according to the discussion section, the feature vector generates high energy when the piece has a shape deformation and can be used to identify if the profile machine process accomplishes the geometric standard tolerances.