1. Introduction
Smokestack Plume Cloud (PC) rises due to momentum and buoyancy. Finally, the PC dissipates and is carried downwind at a constant height. This height is called plume rise height or Plume Rise (PR). PR calculation is not straightforward, and it is a substantial problem in predicting the dispersion of different harmful effluents into the air [
1]. PR contributes to 1) the distance pollutants carried downwind, 2) their concentration at the surface, where they are deposited in the green environment or inhaled by people, and 3) the amounts of greenhouse gases mixed into the upper troposphere. Therefore, accurate measurement of the PR is of concern for research and operational applications such as air-quality transport models, local environment assessment cases and global climate models [
2].
The parameterizations of PR prediction were developed in the 1960s by Briggs [
3,
4]. Dimensional analysis was used to estimate the PR based on smokestack parameters and meteorological measurements in different atmospheric conditions. Early observations of PR were used to test and rectify the parameterizations developed using dimensional analysis [
5]. Wind tunnel studies and field observations using technologies including film photography, theodolites, and cloud-height searchlights [
6] were several calibration techniques utilized in this domain. There are also three-dimensional air-quality models using parameterization equations, including GEM-MACH [
7], CAMx [
8], and CMAQ [
9].
Some studies tested the parameterizations of PR prediction in the 1970s and 1980s by comparing them to actual observations and demonstrated that the Briggs equations overestimate the PR [
10,
11,
12,
13]. In 1993, an aircraft-based measurement was done to measure
emissions of a power plant which indicated an overestimation of about 400 m [
14]. Although these earlier studies showed some degree of overestimation, in 2002, Webster et al. [
15] performed surface measurements and concluded that the Briggs parameterizations tend to underestimate PR. In 2013, as part of the Canada-Alberta Joint Oil Sands Monitoring (JOSM) Plan, an aerial measurement study was done in northern Alberta’s Athabasca oil sands region to study the dispersion and chemical processing of emitted pollutants [
16,
17,
18]. The project consisted of 84 flight hours of an instrumented Convair aircraft over 21 flights designed to measure pollutants emissions, study the transformation of chemicals downwind of the industry, and verify satellite measurements of pollutants and greenhouse gases in the region. Using aircraft-based measurements and reported smokestack parameters and meteorological data, it was demonstrated that the Briggs equations significantly underestimate PR at this location.
Given the results of [
16,
17,
18] and the gap of more than 30 years since the Briggs equations were developed and previously tested, there is a need for further testing and possible modification of the Briggs equations based on modern observation techniques. In recent decades, there have been many significant advancements in environmental monitoring activities over industrial regions for safety and pollution prevention [
19,
20,
21,
22]. Moreover, several smoke border detection and recognition models have been introduced recently using digital image analysis, such as wavelet and support vector machines [
23], LBP and LBPV pyramids [
24], multi-scale partitions with AdaBoost [
25], and high-order local ternary patterns [
26] which are well-performed and impressive. These improvements have led to the development of instrumentation which can be deployed near any smokestack to give information on pollutant dispersion and potential exposure to people downwind. This information will be based on actual real-time observation, i.e. digital images, as opposed to potentially erroneous and decades-old parameterizations. Due to the similarity of our work to smoke recognition on the one hand, and on the other, the unavailability of PC recognition research, smoke recognition studies will be reviewed in the following.
To find the smoke within an image or a video frame, either a rough location of smoke is identified using bounding boxes called smoke border detection [
27], or pixels are identified and classified in detail, named smoke recognition [
29]. Due to the translucent edges of smoke clouds, the recognition task needs far more accuracy than border detection. Traditional smoke recognition methods utilize manual features, which lead to low accuracy recognition results due to a large variety of smoke appearances. These low-level features consist of motion characteristic analysis of the smoke [
30], smoke colour [
31], and smoke shape [
32]. In another research, [
33] took advantage of the Gaussian Mixture Model (GMM) to detect the motion region of the smoke and [
34] combined rough set and region growing methods as a smoke recognition algorithm which seems to be a time-consuming algorithm due to the computational burden of the region growing process. Since using colour information is less effective due to the similarity of smoke colour to its surrounding environment, the combination of motion and colour characteristics is considered for smoke recognition [
35]. Some algorithms utilize infrared images and video frames in their experiments [
36], which are not easily accessible and can increase the project’s costs. Moreover, using digital images makes the algorithm more flexible as it can be used with more hardware. On the other hand, some smokes are too close to the background temperature to be captured by the near red-channel wavelength. A higher-order dynamical system introduced in 2017 used particle swarm optimization for smoke pattern analysis [
37]. However, this approach had a low border detection rate and high computational complexity.
In recent years, deep learning-based methods, especially Convolutional Neural Network (CNN) based methods, have led to significant results in semantic segmentation [
38] and object recognition [
39]. Similarly, these methods are widely used in smoke border detection and recognition [
40] with different architectures such as three-layer CNN [
41], generative adversarial network (GAN) [
42], and two-path Fully Convolutional Network (FCN) [
28]. Recently, a count prior embedding method was proposed for smoke recognition to extract information about the counts of different pixels (smoke and non-smoke) [
43]. Experimental results showed an improvement in the recognition performance of these studies. However, the high computational complexities of these huge models are an obstacle to their use in PR real-time observations.
We have proposed a novel framework using Deep Convolutional Neural Network (DCNN) algorithms to measure PR. Our approach comprises three stages: 1) recognizing the PC region using an improved Mask R-CNN, 2) extracting the PC’s Neutral Buoyancy Point (NBP) from the centerline of the recognized PC, and 3) transforming the PC’s geometric measurement from an image-scale to real-world scale.
This strategy accurately recognizes the PC and measures PR in real-time. Here, we reinforce the bounding box loss function in Region Proposal Network (RPN) [
46,
47] through engaging a new regularization to the loss function. This regularizer restricts the search domain of RPN to the smokestack exit. In other words, it minimizes the distance between the proposed bounding boxes and the desired smokestack exit, called smokestack exit loss (
). The proposed method is also computationally economical because it generates only a limited number of anchor boxes swarmed across the desired smokestack exit. Consequently, the main contributions of this paper can be summarized as follows:
Proposing Deep Plume Rise Network (DPRNet), a deep learning method for PR measurements, by incorporating PC recognition and image processing-based measurements. We have provided a reproducible algorithm to recognize PCs from RGB images accurately.
To the best of our knowledge, this paper estimates the PCs’ neutral buoyancy coordinates for the first time, which is of the essence in environmental studies. This online information can help update related criteria, such as the live air-quality health index (AQHI).
A pixel-level recognition dataset, Deep Plume Rise Dataset (DPRD), containing: 1) 2500 fine segments of PCs, 2) The upper and lower boundaries of PCs, 3) The image coordinates of smokestack exit, 4) The centerlines and NBP image coordinates of PCs, is presented. As is expected, the DPRD dataset includes one class, namely PC. Widely-used DCNN-based smoke recognition methods are employed to evaluate our dataset. Furthermore, this newly generated dataset was used for PR measurements.
This paper is organized as follows—
Section 2 briefly explains the theoretical information used in our proposed framework.
Section 3 describes our proposed framework for measuring the PR of a desired smokestack. Then,
Section 4 present our dataset collection procedure, under-study site, experimental results of the proposed method and evaluation results using different metrics, and PR and PR distance calculations. Finally, this research’s conclusions, findings, and future studies are described in
Section 5.
4. Experimental results and discussion
In this section, we describe our image datasets and the industrial area in which these image datasets have been collected and shared. Also, we will explain the validation metrics used to compare our proposed method with the other competitive methods in smoke border detection and recognition. Then, our discussion falls into two last sections, named comparison with existing smoke recognition methods and plume rise measurement, in which the performance of the proposed method is evaluated, and the PR is calculated based on our "DPRNet," respectively. To validate the performance of our proposed method, we used a computer equipped with Core i9, 3.70 GHz/4.90 GHz, 20 MB cache CPU, 64GB RAM and NVIDIA GeForce RTX 3080,10 GB graphic card. The total training time of the network was about one hour using Python 3.8 with PyTorch Deep Learning framework. Finally, for the geometric transformation and image processing analysis, we used MATLAB R2022b software.
4.1. Site description
The imaging system was deployed on a meteorological tower with a clear sightline to the desired smokestack operated by the Wood Buffalo Environment Association (WBEA). It is located outside the Syncrude oil sands processing facility north of Fort McMurray, Alberta, Canada.
Figure 9 represents the satellite images, the location of the camera, and the desired smokestack.
WBEA operates a 10-meter-tall meteorological tower with a clear sightline to the smokestack at Syncrude (
https://wbea.org/stations/buffalo-viewpoint). The camera system is mounted on this tower above the tree canopy because they are on a hill sloped downward from the tower location, and the biggest smokestack and its PC are always visible. The system consists of a digital camera with shutter control and a camera housing for weather protection with interior heating for window defrost and de-icing.
The Syncrude processing facility has six main smokestacks. The tallest one is about 183 m, and the heights of the other five are between 31 m to 76 m. To isolate a single smoke plume rise, we have concentrated on the area’s tallest one, which can help find the PR for one plume source. All six smokestacks are listed in
Table 1. Wind directions during the capturing period were determined from the Mildred Lake Air Monitoring Station (
https://wbea.org/stations/mildred-lake), located at the Mildred Lake airstrip (AMS02: Latitude:
, Longitude:
), approximately 5 km from the Syncrude facility.
4.2. Deep Plume Rise Dataset (DPRD)
The greatest challenge in using deep learning for PC recognition is inadequate annotated images for training. Hence, creating image datasets for PC recognition for research and industry purposes is invaluable. For this study, 96 images were captured daily; for the first part of the project, 35K images were collected from January 2019 to December 2019. The collected images demonstrated various types of plume shapes in different atmospheric conditions. The dataset has been classified into day, night, and cloudy/foggy conditions. The collected dataset revealed that among 96 images captured daily, we have 48-day and 48-night images. There were some outlier images for different reasons, such as camera handle shaking, auto-focus problems, disturbing smoke and severe snow and hail. Furthermore, some PCs could not be recognized from their background, even by visual image inspection. As a consequence, among 35K collected images, 10684 were valid. Note that about 8110 images were captured when the facility was not working.
This paper introduces a new benchmark, DPRD, including a 2500 annotated dataset. DPRD contains PC upper and lower borders, smokestack exit image coordinates, PC centerline, and NBP image coordinates. 60% of DPRD is considered training data and 40% is used for validation and testing. Rows (a) and (b) in
Figure 10 show sample images from the region and their corresponding ground truth, which are generated by the "Labelme" graphical image annotation tool at
https://github.com/wkentaro/labelme. We tried to select images of different atmospheric conditions, such as clear daytime, nighttime, cloudy, and foggy, to represent the results of different situations.
4.3. Model validation metrics
The performance of the methods in question is evaluated using the metrics of accuracy, recall, precision and F1 score. These metrics are defined using four values of True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN) obtained from the confusion matrix of each introduced method [
58]. The accuracy validation metric is the ratio of observations predicted correctly to the total observations. In our application, the model’s accuracy represents how accurately our model can recognize the PC pixels. This criterion is valid as long as the values of FP and FN are almost the same [
58]. Otherwise, other validation metrics should be considered. The foreground pixel coverage of the sample images is revealed in
Figure 10, confirming that accuracy is not suitable for this study. Recall or sensitivity is the ratio of positive observations predicted correctly to all actual observations. Recall shows how many PC pixels are labelled among all the actual PC pixels. The recall is obtained as follows,
Precision is the ratio of positive observations which are predicted correctly to all observations which are predicted as positive. This metric represents how many PC pixels exist among all the pixels labelled as PC. Therefore, a low rate of FP can achieve high precision. This validation metric is obtained as follows,
As it is implied from Equations
13 and
14, precision and recall take either FP or FN into account. The last validation measure in this paper, the F1 score, considers both FP and FN as a weighted average of recall and precision metrics. Unlike accuracy, this metric is more useful when FP and FN are not the same as in our study. Our FP is less than FN, or the amount of non-actual PC pixels predicted as PC pixels is less than that of actual PC pixels predicted as non-PC pixels. Therefore, the F1 score helps us look at both recall and precision validation metrics as follows,
4.4. Comparison with existing smoke recognition methods
In this section, we evaluate the performance of DPRNet and compare it with several competitors. To choose suitable smoke recognition methods for comparison, we considered both the identification accuracy and computational complexity of the reviewed approaches, which led to the selection of DeepLabv3+ [
56], FCN [
55], and regular Mask R-CNN. Our proposed DPRNet is evaluated using three metrics introduced in
Section 4.3.
As is clear from
Table 2, DPRNet performs much better than competitive methods for 90 test images selected from various day, night, foggy, and cloudy conditions. In detail, the recall and precision metrics express the reasonable difference between the models, which shows the effectiveness of the proposed model in recognizing the actual PC pixels. Compared to the rivals, the more considerable value of the F1 score guarantees that DPRNet outperforms the other three methods and shows the efficacy of this method. Among our competitive methods, DeepLabv3 performed better regarding all validation metrics, and Mask R-CNN had the worst performance.
Besides these average values, the detailed statistics for each model are given in
Figure 11 in terms of each used validation metric. At a glance, our proposed method shows the most robustness in all circumstances. Of competitors, Mask R-CNN and FCN have the worst performance, whereas, DeepLabv3 has the best efficiency slightly.
To further validate our DPRNet performance, we compared the models over the day, night and foggy & cloudy data sets in terms of different validation metrics, which are given in
Figure 12. It can be observed that all methods, except Mask R-CNN, have acceptable performance using day and night datasets. Even with night precision, FCN is better than our proposed method. However, as discussed in
Section 4.3, this metric can only partially convey the merit of a model individually, and it needs to be analyzed with the F1 score. Our proposed DPRNet outperforms the other rival methods by recognizing roughly all of the PC pixels correctly. Most datasets are related to cloudy and foggy conditions and are frequently seen within image batches. The strength of our DPRNet is its robust performance in this case, which is of paramount importance in our application. The DPRNet could improve the recall metric by 66%, 58%, and 87% on average in cloudy and foggy conditions relative to FCN, DeepLabv3, and Mask R-CNN frameworks, respectively, which means that the proposed method is able to find the PC regions appropriately, using
. This capability produces high-quality image recognition with a more complicated mixture of PCs and the sky behind. These high recall values help us meet our research application requirement, in which we should identify the entire PC stream for PR distance measurement.
To demonstrate the qualitative results of the proposed method, we show some visual results to compare competitive methods.
Figure 13 depicts these recognition results. The first two rows represent the input images and their corresponding ground truths, respectively, and the other rows give the output of different models. We tried to visualize samples from all classes such that the first two images are related to cloudy/foggy conditions, the second two are from the nighttime dataset, and the last two are obtained from our daytime dataset. It is observed that DPRNet outperformed the other methods by attaining high accuracy of PC localization and, consequently, correctly recognizing the desired smokestack PC.
4.5. Plume rise measurement
As discussed in
Section 3, DPRNet gives PC border detection and recognition. Then, we take advantage of the NBP image coordinates and the wind direction information from the meteorological tower to obtain PR real-life measurements through geometric transformations.
Figure 14 illustrates the asymptotic curve for four PC images and the automatically chosen point NBP, where the PC reaches neutral buoyancy. Apart from the PR and PR distance values of each sample PC, estimated by the proposed framework (tabulated in
Table 3), the averaged hourly measured wind directions at the image sampling times are given as prior information of this study. These realistic PR and PR distance values within an extended period are required for future studies.
Figure 1.
Region proposal network. Red rectangles illustrate the proposal regions on the feature map of the input image.
Figure 1.
Region proposal network. Red rectangles illustrate the proposal regions on the feature map of the input image.
Figure 2.
PR measurements system framework. and are the NBP coordinates in the image scale. Similarly, and represent the NBP coordinates in the real-life scale. and are PR and PR distance in real-life scale.
Figure 2.
PR measurements system framework. and are the NBP coordinates in the image scale. Similarly, and represent the NBP coordinates in the real-life scale. and are PR and PR distance in real-life scale.
Figure 3.
DPRNet architecture. The supplemental modules are shown in green, and the dashed blue rectangle is dismissed in the inference time.
Figure 3.
DPRNet architecture. The supplemental modules are shown in green, and the dashed blue rectangle is dismissed in the inference time.
Figure 4.
Sample PC segments with eight boundary points.
Figure 4.
Sample PC segments with eight boundary points.
Figure 5.
NBP extraction framework. The red curve represents the centerline of the PC. The cyan and yellow lines, respectively, display the upper and lower boundaries of the PC. Green dashes demonstrate the asymptotic curve, and the magenta point is NBP.
Figure 5.
NBP extraction framework. The red curve represents the centerline of the PC. The cyan and yellow lines, respectively, display the upper and lower boundaries of the PC. Green dashes demonstrate the asymptotic curve, and the magenta point is NBP.
Figure 7.
The schematic top view of the region. is the depth of the in real-life, is the camera’s field of view, and denotes the deviation from the camera-smokestack line.
Figure 7.
The schematic top view of the region. is the depth of the in real-life, is the camera’s field of view, and denotes the deviation from the camera-smokestack line.
Figure 8.
Smokestack location schemes. Smokestack exit, S; image center, O; PC centerline, CL; NBP image coordinates, ; depth of the point in the real world, ; and the yellow arrow shows the wind direction.
Figure 8.
Smokestack location schemes. Smokestack exit, S; image center, O; PC centerline, CL; NBP image coordinates, ; depth of the point in the real world, ; and the yellow arrow shows the wind direction.
Figure 9.
Imaging situation. Camera station, C; and smokestack position, S. The abc coordinate system is only for differentiating the side and camera views and is not used as a coordinate reference system.
Figure 9.
Imaging situation. Camera station, C; and smokestack position, S. The abc coordinate system is only for differentiating the side and camera views and is not used as a coordinate reference system.
Figure 10.
Sample images (up) and their corresponding ground truth (down) from our DPR dataset are listed as (a) Clear daytime, (b)&(c) cloudy day, and (d)&(e) clear nighttime.
Figure 10.
Sample images (up) and their corresponding ground truth (down) from our DPR dataset are listed as (a) Clear daytime, (b)&(c) cloudy day, and (d)&(e) clear nighttime.
Figure 11.
Performance of different methods regarding some test images (a) recall, (b) precision and (c) F1 score metrics.
Figure 11.
Performance of different methods regarding some test images (a) recall, (b) precision and (c) F1 score metrics.
Figure 12.
Detailed comparison of methods over three datasets employing (a) recall, (b) precision and (c) F1 score metrics.
Figure 12.
Detailed comparison of methods over three datasets employing (a) recall, (b) precision and (c) F1 score metrics.
Figure 13.
Qualitative results of recognition tasks listed as: (a) Input image, (b) corresponding ground truth, (c) results of Mask R-CNN, (d) FCN, (e) results of DeepLabv3, and (f) results of DPRNet.
Figure 13.
Qualitative results of recognition tasks listed as: (a) Input image, (b) corresponding ground truth, (c) results of Mask R-CNN, (d) FCN, (e) results of DeepLabv3, and (f) results of DPRNet.
Figure 14.
DPRNet and image measurement results. In column (c), the red curve represents the meandering of the PC. The cyan and yellow lines, respectively, illustrate the upper and lower boundaries of the PC. Green dashes show the asymptotic curve; the magenta point is NBP.
Figure 14.
DPRNet and image measurement results. In column (c), the red curve represents the meandering of the PC. The cyan and yellow lines, respectively, illustrate the upper and lower boundaries of the PC. Green dashes show the asymptotic curve; the magenta point is NBP.
Table 1.
Syncrude smokestacks information, including location, smokestack height (), smokestack diameter (), the effluent velocity at the smokestack exit (), and effluent temperature at the smokestack exit (). The velocities and temperatures are averages for the entire capturing period.
Table 1.
Syncrude smokestacks information, including location, smokestack height (), smokestack diameter (), the effluent velocity at the smokestack exit (), and effluent temperature at the smokestack exit (). The velocities and temperatures are averages for the entire capturing period.
Reported ID |
Latitude |
Longitude |
(m) |
(m) |
() |
(K) |
Syn. 12908 |
57.041 |
-111.616 |
183.0 |
7.9 |
12.0 |
427.9 |
Syn. 12909 |
57.048 |
-111.613 |
76.2 |
6.6 |
10.1 |
350.7 |
Syn. 13219 |
57.296 |
-111.506 |
30.5 |
5.2 |
8.8 |
355.0 |
Syn. 16914 |
57.046 |
-111.602 |
45.7 |
1.9 |
12.0 |
643.4 |
Syn. 16915 |
57.046 |
-111.604 |
31.0 |
5.0 |
9.0 |
454.5 |
Syn. 16916 |
57.297 |
-111.505 |
31.0 |
5.2 |
9.2 |
355.0 |
Table 2.
Comparison of different methods for PC recognition using average validation metrics values.
Table 2.
Comparison of different methods for PC recognition using average validation metrics values.
Model |
Recall |
Precision |
F1 score |
Mask R-CNN |
0.556 |
0.727 |
0.607 |
FCN |
0.591 |
0.859 |
0.599 |
DeepLabv3 |
0.654 |
0.892 |
0.721 |
DPRNet |
0.846 |
0.925 |
0.881 |
Table 3.
PR and PR distance values of each of the four PC images from
Figure 14.
Table 3.
PR and PR distance values of each of the four PC images from
Figure 14.
Image |
Date |
Time |
(deg.) |
(deg.) |
(m) |
(m) |
I1 |
2019-11-08 |
18-00-13 |
12.16 |
-239.8 |
177 |
1685 |
I2 |
2019-11-09 |
15-00-13 |
3.46 |
-248.5 |
450.3 |
3287 |
I3 |
2019-11-14 |
10-00-16 |
10.41 |
-241.6 |
266.8 |
2280 |
I4 |
2019-11-16 |
11-00-12 |
10.83 |
-241.1 |
300.5 |
2905 |