1. Introduction
Numerous physiological activities in leaves, including photosynthesis, respiration, and transpiration, are primarily regulated by nitrogen [
1,
2,
3]. Nitrogen is also closely related to chlorophyll concentration, light utilization efficiency, and net crop production [
4,
5,
6]. Besides being a frequently limiting factor for plant growth [
7,
8,
9], nitrogen is a crucial input in the cycle of ecological processes [
10,
11]. Leaf nitrogen content has also been suggested as one of the crucial bio-diversity variables for monitoring of the progress towards the Aichi Biodiversity Targets by the remote sensing and ecology communities, who acknowledge the significant role of leaf nitrogen in biodiversity and ecosystem functioning [
12,
13].
Despite being a relatively minor element of leaves (up to 6.5-7%), nitrogen has been reliably recovered using leaf and canopy-level hyperspectral data [
14,
15]. By offering contiguous, narrow spectral band data, hyperspectral data are able to identify the nitrogen's small absorption properties. Compared to the conventional destructive sample procedures, this provides an effective and economical way to assess leaf nitrogen. The spectra from leaf powder, dry leaves, and fresh leaves were employed in previous research on the determination of nitrogen concentration in vegetation, as well as estimates at the canopy level [
16,
17,
18]. The mask of the significant water absorption [
17,
19], the confounding effects coming from canopy structure, illumination/viewing geometry, and background [
20,
21], and other obstacles make it difficult to retrieve nitrogen at the canopy level.
To improve the nitrogen content estimation performance, numerous methods have been employed. One method is spectral transformation, which includes employing first/second derivatives and reflectance log transformation [
22,
23]. There are also other studies which includes continuum removal [
17,
24], water removal [
25,
26], and wavelet analysis [
27]. Numerous studies have been proposed to estimate the nitrogen content over forests [
18,
23,
28,
29,
30], grasslands [
31,
32,
33], and crop ecosystems [
34,
35,
36]. A variety of artificial intelligence techniques, including support vector regression, neural networks, and Bayesian model averaging [
32,
36,
37], as well as traditional regression techniques like stepwise multiple linear regression and partial least square regression, are used to retrieve nitrogen concentration.
Vegetation indices are one of the simplest and most popular empirical methodologies for estimating the biochemical contents of leaves, such as nitrogen. The main sources of nitrogen in leaf cells are proteins and chlorophylls [
38]. Since there is a strong correlation between nitrogen and chlorophyll in a variety of species [
1,
2,
39], nitrogen has been estimated using vegetation indices which are proposed and used for chlorophyll estimation [
40]. For measuring chlorophyll, spectral wavelengths around 550 nm and 700 nm, as well as the red-edge area (680-780 nm), have been used [
35,
41], leading to a significant variety of indices [
42,
43,
44]. In contrast to chlorophyll, there are fewer studies that offer particular indices for nitrogen estimate; the majority of these indices were established for crops [
45,
46,
47,
48], while just a small number were developed for forests [
49].
Given that canopy structure is the primary cause of changes in canopy reflectance, calculation of foliar nitrogen using canopy spectral data is complicated. According to the study in [
7], the NIR reflectance (800–850 nm) and canopy foliar mass–based nitrogen concentration (%N) have a significant association that can be utilized to predict nitrogen where in [
50] the researchers pointed out that the association between NIR reflectance and canopy structure. The study in [
51] suggested that the biological associations between nitrogen and structural characteristics that affect NIR scattering and reflectance served as the foundation for their ideas. Additionally, in [
14] and [
52], scientists indicated that the canopy structure and leaf characteristics may co-vary among plant functional types, contradicting the study in [
53] who claimed that the %N-NIR correlation is inherently false.
The purpose of the study in [
54] was to assess how well 32 vegetation indicators collected from airborne hyperspectral imaging performed when used to calculate canopy foliar nitrogen in a mixed temperate forest. For comparison, the widely used partial least squares regression was carried. These vegetation indicators can be divided into three groups, most of which are connected to the biochemical and structural characteristics of vegetation (e.g., nitrogen, chlorophyll, and leaf area index (LAI)). The nitrogen indicators are selected based on how nitrogen absorption characteristics' physical underpinnings affect canopy reflectance. The biological connections between nitrogen, chlorophyll, and canopy structure were used to justify the inclusion of the structural and chlorophyll indices in this study. Nitrogen (N) losses and the ensuing environmental issues are what define the production of vegetable crops [
55,
56,
57]. The most frequent environmental issues include nitrous oxide (N2O) emissions, ground and surface water contamination, and eutrophication of surface waters [
58,
59]. These issues are frequently a result of the extensive use of N fertilizer [
60], which typically exceeds the requirement of the crops [
57,
61,
62], which is done to ensure optimal growth and production. In order to decrease N contamination of water bodies by intensive vegetable production, it is necessary to understand crop nitrogen requirements and match crop demand with nitrogen supply [
57,
63,
64].
There are numerous techniques available for tracking crop nitrogen status [
57,
65]. Leaf nutritional analysis is a common method, but it is time-consuming and labor-intensive in the lab, and it typically cannot quantify the temporal and geographical variability of nitrogen status [
66,
67]. These are significant drawbacks since it is much easier to match the supply of nitrogen to crop requirements when one is aware of the temporal and spatial variability of crop nitrogen status [
68].
Optical sensors are tools that quickly, accurately, and nondestructively monitor the crop's nitrogen status in the field [
57,
69]. They make it possible to regularly evaluate a crop and evaluate spatial variability. Canopy reflectance sensors, which are among the proximal optical sensors, have two highly advantageous characteristics in that they can monitor huge portions of a crop while they are in motion [
70]. Field crops' nitrogen status can be evaluated via measures of crop reflectance [
65]. These measures are based on the differential reflection of light wavelengths [
57], which, depending on crop nitrogen status [
69], are absorbed and reflected by the crop in varying amounts. Typically, red, green, and near-infrared light wave lengths are employed for nitrogen estimation [
65]. Recently, the red-edge has been suggested as a solution for nitrogen estimation to the red band's apparent saturation [
71,
72].
The nitrogen nutrition index (NNI) [
67,
73], is another commonly utilized strategy. The critical crop nitrogen content [
74,
75] is the lowest crop nitrogen content required for non-limiting growth, and it is used to calculate NNI by dividing the actual crop nitrogen content by it. Any variation from 1 indicates either excess nitrogen (i.e., NNI>1) or insufficient nitrogen (i.e., NNI1) crop status, with values of NNI equal to 1 indicating adequate nitrogen feeding [
76].
Due to the high cost of SWIR band sensors, a regression based method should be developed which maps the VNIR band indexes to SWIR band indexes which has more capability to measure the crop nitrogen status like [113]. The majority of studies have been done on cereal crops like wheat [
12,
79] and rice [
80,
81]; very few have been done on vegetable crops like sweet pepper.
In [
83], the crop nitrogen status of sweet pepper was estimated using eight vegetation indices that were computed from canopy reflectance data taken with two separate proximate sensors. First, crop NNI calibration regression models were fitted for each vegetation index. Second, a different dataset was used to validate these regression equations. Thirdly, sufficient values for each vegetative index for optimal nitrogen nutrition, for the main phenological stages of sweet pepper crops, were obtained utilizing the validated equations between vegetation indices and crop NNI.
The findings of the study [
84] supported the use of the normalized difference vegetation index (NDVI) as a useful tool for determining the nitrogen status of cotton leaves at various growth stages. Using vegetation indices, the study in [
85] calculated the nitrogen nutrition index (NNI), canopy nitrogen density (CND), and leaf nitrogen content of winter wheat over the course of the entire growth period. This study demonstrated that the correlations between each nitrogen index and the Vogelmann red-edge index (VOG), simple ratio pigment index (SRPI), modified red-edge simple ratio index, and red-edge position based on linear interpolation method (REPliner) were not significantly influenced by growth period, and the estimation model R
2 for CND was higher than 81%. The estimation model's accuracy was higher than NNI, however it would become saturated if CND is calculated using just one vegetation index. The red-edge chlorophyll index, CIred-edge was demonstrated by [
86] to be responsive to the canopy structure. The correlation between the nitrogen content of cotton leaves and several spectral ratio measures was examined in [
87] who also conducted a cluster analysis based on prediction accuracy and overall accuracy.
The ratio of the red-edge position to the near-infrared band was shown to have a pretty high prediction accuracy and overall accuracy. The estimation of the winter wheat spectral index was investigated by [
88] in a variety of environments, seasons, varieties, and growth stages. According to their findings, the growth stage had a significant impact on the performance of various vegetation indices and the choice of a sensitive wavelength for plant nitrogen concentration (PNC) estimation. The simple ratio of reflectivity at 370 nm and 400 nm (R
370/R
400) displayed the most consistent estimation accuracy in an indoor experiment (R
2 = 0.58) and field experiment (R
2 = 0.51). According to the studies, there are obvious changes in the relevant spectral index for different crops, or for various kinds and ecological zones of the same crop, when employing the spectral index to estimate crop nitrogen [
89,
90].
Additionally, compared to employing sensitive spectral features alone or vegetation indexes, modeling techniques like deep machine learning can produce greater prediction effects [
91]. These techniques can also be used to monitor agricultural nutrients and growth indicators [
92,
93,
94]. Support vector machine regression (SVR) was shown to be the most effective method for assessing crop nutrient contents in [
95] which evaluated artificial neural network and SVR methods. The authors proposed that the creation of models with large sample sizes is appropriate for an artificial neural network. Hyperspectral reflectance of leaves was used in [
90] to study the generalized partial least-squares regression (PLSR) model, and this approach was successful in retrieving leaf nitrogen concentration (r = 0.85).
The study in [
96] is based on data for two types of drip-irrigated cotton at various growth stages from April 2019 to September 2020. The data include canopy nitrogen density (CND) and leaf nitrogen concentration (LNC) values. Pearson's correlation analysis was used to determine which of the 30 hyperspectral vegetation indexes and the two nitrogen indexes (LNC and CND) that were used in the three modeling techniques of simple multiple linear regression (MLR), PLSR, and support vector regression (SVR) were relatively stable. The models were employed to investigate the possibility of measuring the nitrogen nutrient status of cotton in each growth period based on a multi-vegetation index in order to give theoretical background for the application of remote sensing technology in cotton nutrition monitoring and diagnostics.
Based on the previous studies conducted in the literature to estimate the nitrogen content of the plant, the contributions of this study are,
- -
Using the radiance values provided by Hyperion data directly without applying any atmospheric correction.
- -
Developing a proper deep model which transforms VNIR-only vegetation indexes to NDNI with a high correlation.
- -
Removing the necessity to have high cost special cameras like SWIR to measure the nitrogen content of the crop.
- -
Enabling the farmers follow the nitrogen content of the crop progressively and decide when to/not to fertilize.
2. Materials and Methods
Image data of Hyperion sensor was used in this study. As a push-broom hyperspectral instrument, Hyperion is housed on the EO-1 satellite which is depicted in
Figure 1.
With a 10-nm bandwidth spanning from 357 nm to 2576 nm, it collects 220 different spectral channels. All bands have a spatial resolution of (30x30) m
2. The VNIR band, SWIR band, and VNIR-SWIR band indices can all be estimated from the same image data because Hyperion includes both the VNIR and SWIR bands. There is no need for a geometric correction because the Hyperion images have already been rectified geometrically. In addition no atmospheric correction is applied on the data. This is due to the fact that atmospheric correction tools cannot achieve a perfect reconstruction which affects the results negatively. The properties of Hyperion sensor is given in
Table 1.
Different test images were collected from Hyperion data of
Harran region based in South-West of Turkey. Google Earth and Earth explorer views of
Harran and Hyperion image are shown in
Figure 2.
Harran Plain is a region that starts from the southeast of the city of Şanlıurfa and extends to the Syrian border. It is one of the plains with very fertile soil. It is located between 36°43'-37°08' N parallels and 38°57'-39°55' east meridians in the Upper Mesopotamian lands in Şanlıurfa province.
Mainly corn and cotton farming is done. In the Harran Plain, under the influence of the Mediterranean climate, the continental climate is dominant. Winters are cold and rainy, summers are very hot and dry. Annual precipitation is 365 mm, annual evaporation is 1,848 mm, and annual average temperature is 17.2 °C.
Hyperion image downloaded from Earth Explorer covers roughly 900 km
2 which is both required and sufficient to investigate the reliability of the proposed method. The QGIS Geographic Information Software Tool is used to determine the area of the Hyperion images [
98].
Figure 3 shows the image data taken from
Harran region in which a large amount of corn, cotton and wheat planting is done by farmers. Image acquisition time is
2016-08-08 when the crop is dense and mature. Image latitude and longitude values and the scale of the map are given on the figure as well.
Figure 4 shows a drone camera view and a ground image of a region from Harran taken in July, 2019.
In
Harran region, to show the vegetation density and the spreading over the area, an NDVI estimation is applied firstly. The Hyperion image and the resulted NDVI index map is shown in
Figure 5. Hyperion image has a spatial resolution of 3241x1241, totally 4,022,081 pixels where 2,859,262 pixels are the black regions surrounding the target region. Therefore, vegetation index analysis is implemented on 1,162,819 pixels. Each pixel corresponds to 30x30 m
2 ground area. By recalling that NDVI values can be change from -1 to 1, from the
Table 2, it can be seen that the number of pixels have a high NDVI value like > 0.75 is lower, relatively. An important reason for this is the ground spatial resolution of the Hyperion sensor in which one pixel covers 30x30 m
2. This is relatively a large area in which spectral mixing occurs, therefore the spectra of soil, water and vegetation is mixed which may reduce the NDVI indices [
100]. Nonetheless the data is powerful and exhibits a good distribution which enables an accurate analysis of vegetation indices.
To increase the accuracy, the pixels with NDVI values higher than 0.2 is taken into account. This is due to typical vegetation pixel has an NDVI greater than 0.2.
The study in [
101] states that, NDNI can be used effectively for estimation of the nitrogen content of the vegetation. Since, in this study, we develop a deep model which establish a correlation between VNIR band vegetation indices and NDNI, the most important vegetation indices which hold information about nitrogen content of the plants are estimated. These indices are: NDVI [
102], GNDVI [
103], EVI [
104], GOSAVI [
105], GSAVI [
105], MCARI2 [
106] and VREI2 [
107]. To establish the correlation, NDNI [
108] is estimated as well.
Table 3 shows the corresponding indices, the bands or wavelengths and the equations which are used to estimate them.
The black regions on Hyperion data is excluded first. Then, each vegetation indices are estimated for the pixels left.
to be number of pixels in each index map, a vector of index values is created as shown in equation 1, where
is the index map of the corresponding vegetation index and
is the estimated index value at pixel i.
Then, a new index data is created as shown in equation 2, where
is the estimated index for (b). pixel of vegetation index (a). (i.e. 3
rd pixel of NDVI map.)
Since this study uses 7 vegetation index from VNIR region, totally 7 are created at the beginning. Therefore the size of the data matrix is (7x1,162,819). After handling data matrix, a further analysis is done.
- -
Due to the division by 0, some index values are calculated as infinite and/or NaN. Therefore, those kind of pixels are found and the corresponding column is deleted.
- -
Another analysis is also done for the pixels having abnormally large vegetation index. Therefore the index values which has an absolute value above 5 are also deleted from the data.
- -
Finally 1,113,529 pixel values are used and to normalize the effect of the environment at the time of the capturing, each index row in the data is normalized between -1 and 1. To normalize the data the normalize function of Matlab is used with a ‘range’ parameter.
Similarly the above operations are also applied on NDNI Map. Finally, the input data has the shape of (7x1113529) and the output data has (1x1113529).
To train a model which matches the input data to the output data,
Matlab deep learning toolbox [
109] is used. By using this tool, a deep neural network is designed with 4 dense (hidden) layers, each with 25 neurons.
Figure 6 shows the designed neural network. Number of input neurons is 7 due to the shape of the input data and 1 for output, hence it is the NDNI index value.
Data is split into train, validation and test datasets with the default ratio of 0.7:0.15:0.15. Number of epochs is set as 1000. Training function and adaption learning functions are set as TRAINLM (Levenberg-Marquardt) and LEARNGDM. LEARNGDM is a gradient descent algorithm with momentum. The cost will be pushed farther to go around a saddle point by adding a momentum element to the gradient descent, even though the current gradient is insignificant. Performance function is MSE. Training is done on CPU on a Windows PC with 8 GB Ram and 11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz processor. The reason not to use GPU is that in Matlab, TRAINLM function is not supported.
The Levenberg-Marquardt [
110] technique, which was developed for minimizing functions that are sums of squares of nonlinear functions, is derived from Newton's method [
111]. Levenberg-Marquardt algorithm is designed to minimize sum-of-square error functions of the form (3). In (3),
is the error in the kth instance and
is a vector with element at k. A Taylor series can be used to expand the error vector to first order if the difference between the old and new weight vectors is modest.
As a result, the error function can be expressed as
After minimizing the error function with respect to the updated weight vector, (6) can be written.
where,
Since the Hessian for the sum-of-square error function is as shown in (8),
By ignoring the second term, Hessian can be written as:
For nonlinear networks, updating the weights therefore entails the inverse Hessian. Since the Hessian is based on first order derivatives with respect to the network weights, which can be easily handled by back propagation, it can be calculated rather quickly. Although iterative application of the updating formula to reduce the error function is an option, this may provide a step size that is too big, invalidating the linear approximation that the method is based on.
The Levenberg-Marquardt approach minimizes the error function while maintaining a small step size to guarantee the accuracy of the linear approximation. Utilizing a form's customized error function allows for this.
where
is a parameter adjusting the step size. When the modified error is minimized with respect to
, (11) is handled.
With very large values of , Levenberg-Marquardt approaches standard gradient descent, whereas for very small values , approaches to the Newton method.
This study aims to investigate the relation between VNIR-only vegetation indexes and NDNI. For that purpose linear regression is employed by using a deep neural network. Linear regression can be explained as follows:
where
In equation 12,
is the dependent (outcome) variable,
is y-intercept,
is the slope of the regression line,
is the independent variable, and
is the error term. The calculation of linear regression includes 3 steps:
First, the values of formula components a and b are found by using , , , and
Then the values derived in the first step are substituted into and
Finally, and values are used with the formula Y = + X + ɛ to establish the linear relationship between and variables.
Since a deep neural network is used in this study, the calculation is done by neural network tool defined in Matlab and linear regression is employed.