Prerpints.org logo
Preprint
Article

Automated Photogrammetric Tool for Landslide Recognition and Volume Calculation Using Time-Lapse Imagery

Altmetrics

Downloads

128

Views

38

Comments

0

A peer-reviewed article of this preprint also exists.

Submitted:

10 July 2024

Posted:

11 July 2024

You are already at the latest version

Alerts
Abstract
Digital photogrammetry has attracted widespread attention in the field of geotechnical and geological survey due to its low-cost, ease of use and contactless mode. In this work, with the purpose of studying the progressive block surficial detachments of a landslide we developed a monitoring system based on fixed multi-view time-lapse cameras. Thanks to a new-developed photogrammetric algorithm based on the comparison of photo sequences through a structural similarity metric and the computation of the disparity map of two convergent views we can quickly detect the occurrence of collapse events, determine their location and calculate the collapse volume. With the field data obtained at the Perarolo landslide site (Belluno Province, Italy), we have preliminary tested the effectiveness of the algorithm and its accuracy in the volume calculation. The method of quickly and automatically obtaining collapse information proposed in this paper can extend the potentials of landslide monitoring system based on videos or photo sequence and it will be of great significance for further research on the link between the frequency of collapse events and the driving factors.
Keywords: 
Subject: Engineering  -   Civil Engineering

1. Introduction

In recent decades, digital photogrammetry has emerged as a powerful and cost-effective method for 3D reconstruction of objects and surfaces across various scientific and engineering disciplines. Due to advantages such as ease of use and the capability of mounting cameras on drones, its application has expanded to engineering geology. This technology offers an affordable and efficient survey and monitoring solution compared to other terrestrial approaches like laser scanning and radar interferometry. Most photogrammetric applications in this field focus on the morphological 3D reconstruction of slope surfaces and rock masses [1,2,3]. More recently, its use has been extended to landslide displacement monitoring [4,5,6]. The occurrence of sudden collapses and local surface sliding on nearly vertical slopes represents precursor signals of landslide reactivation. These events provide critical information for understanding landslide behavior, assessing stability conditions, and forecasting potential evolution. Conventional techniques, such as topographic surveys, generally fail to capture these events due to excessive deformations associated with collapses and the disruption of optical targets. Moreover, these methods often provide sparse spatial information, limited to the locations of optical prisms. Radar interferometry, while effective for detecting gradual movements, is inadequate for capturing large sudden displacements. Similarly, laser scanner surveys can estimate missing volumes but face challenges in long-term monitoring. In this context, time-lapse close-range photogrammetry offers a viable alternative. It allows for prolonged observation of landslides, automatically obtaining detailed optical and spatial information from images. By comparing and analyzing image sequences from multiple cameras, local slip events and collapsed areas can be quickly identified. The volumes of collapsed material can be estimated based on the relative positions of the cameras. This data can then be used to hypothesize correlations between detachment events and driving factors such as rainfall and piezometric levels. Given the importance of automatic detection and susceptibility mapping for understanding landslide characteristics and risk assessment, numerous intelligent methods based on image analysis have been proposed [7,8,9]. Lei et al. [10] proposed an end-to-end change detection algorithm using a symmetric fully convolutional network. Lv et al. [11] introduced a landslide detection method based on multiscale segmentation and object-based majority voting of images. Convolutional neural network-based methods have been widely adopted in this field [9,12,13]. Additionally, Lu et al. [14] presented an object-oriented change detection method for rapid landslide mapping. While these algorithms can detect landslides over large areas using aerial or satellite images, they typically operate at low temporal resolution and often fail to detect detachments on nearly vertical slopes. In contrast, our approach focuses on terrestrial photo sequences at high spatial and medium-high temporal resolution. Terrestrial photos offer the advantage of adjustable spatial resolution (e.g., by using different lenses or changing the camera distance) and temporal resolution.
In this paper, we introduce a novel algorithm that automatically analyzes and processes sequences of terrestrial images from multiple cameras. Utilizing the Structural Similarity Index Measure (SSIM) algorithm [15] and a series of convolution filters, our program identifies collapse events and determines the collapsed areas. The stereo configuration of the cameras enables the calculation of the 3D shape and volume of the detached mass [16,17,18]. This stereo-camera system has been prototyped and tested in the field at the Perarolo landslide site, where spontaneous slope collapses frequently occur. The system, comprising three cameras and a large spotlight, was installed on the opposite side of the valley. Nearly a year of daily and nightly photo sequences were analyzed. Additionally, three large-scale slope stabilization operations using blasting and excavators at the same site were employed to evaluate the effectiveness of the collapse detection method and the accuracy of volume estimation, which was compared to laser scanner surveys.

2. Methods

2.1. Architecture of the Algorithm

Our program is designed to work with fixed multi-view cameras to perform image acquisition, collapse detection, and calculation of collapsed areas and volumes in a fully automatic and continuous manner, without any third-party intervention or manual operation. As shown in Figure 1, the process consists of two main phases: collapse recognition on the image plane through image comparison at different time steps, and identification and calculation of the collapsed area and volume. Before initiating the collapse recognition phase, wooded and grassy areas, if present, must be excluded from the analysis. The color and texture of these areas can change continuously due to wind and seasonal vegetation growth, hindering accurate comparison between successive images. This operation can be performed using a deep learning mask [19]. It should be noted that this limitation naturally restricts the method’s applicability in fully vegetated areas.
The second pre-processing step involves filtering out areas covered by shadows, which could introduce collapse artifacts, particularly due to changes in sun position over time [20]. A detailed analysis of this aspect will be provided in Section 2.3.
The core of the algorithm uses the Structural Similarity Index Measure (SSIM) to compare pairs of images in sequence, generating a structural similarity map at each time interval [21,22]. To reduce noise caused by natural small variations in color intensity, brightness, and contrast, image convolution filters are applied to the similarity map. We employed a cascade of Gaussian and median convolution filters with different patch sizes for this purpose [23]. Based on the statistics of pixel values in the final structural similarity map, potential collapse areas can be identified, and a unique structural similarity index is obtained. This index is used to determine whether a collapse occurred between the images taken at times t(i) and t(i+1). When the structural similarity index exceeds a certain threshold, the program signals the presence of a collapse and automatically initiates the calculation of the collapsed volume using images from a second camera. This second phase of the process involves comparing two point-clouds generated from the 3D reconstruction of the slope before and after the collapse. The details of this phase will be discussed in Section 2.4.

2.2. Structural Similarity Algorithm Approach

The Structural Similarity Index Measure (SSIM) algorithm is commonly used for assessing image quality and evaluating the performance of image compression algorithms and systems [24]. In this study, we propose using SSIM as a local measure of structural changes to detect collapses in a sequence of images taken from a fixed camera oriented towards the slope. The SSIM calculates the Structural Similarity Index (SSI) for each pixel in the image and generates a Structural Similarity Map (SSM) by comparing a target photo with a reference photo [22,25]. The SSI is derived from the variation of three components between the two images or portions of images x and y: the luminance term, the contrast term, and the structural term, according to the following metric:
S S I ( x , y ) = [ l ( x , y ) ] α ∙ [ c ( x , y ) ] β ∙ [ s ( x , y ) ] γ
where
l ( x , y ) = 2 μ x μ y + C 1 μ x 2 + μ y 2 + C 1
c ( x , y ) = 2 σ x σ y + C 2 σ x 2 + σ y 2 + C 2
s ( x , y ) = σ x y + C 3 σ x σ y + C 3
and μ x , μ y , σ x , σ y and σ x y are the local means, standard deviations, and cross-covariances for images x and y. The parameters α, β, γ are exponents for the luminance, contrast, and structural terms, respectively.
The Structural Similarity Map (SSM) is a grayscale map (values ranging from 0 to 1) with the same dimensions as the target image in SSIM. It consists of the SSI values for each pixel in the target image. Figure 2 illustrates an example of an SSM applied to two consecutive images of a landslide taken at different time steps: darker pixels indicate the smallest similarity values, representing areas with a higher probability of erosion or collapse, while lighter pixels indicate areas with high SSI values, signifying minimal structural and color changes. At first glance, it is evident that vegetated parts are classified as dark areas, which could lead to erroneous collapse detection. Additionally, due to the complex and changing environmental conditions of the landslide scene, factors such as brightness variation and shadows—primarily caused by clouds, fog, and sun position—can significantly affect the structural and chromatic information of the image. This can result in artifacts in the assessment of collapse areas. Therefore, it was deemed necessary to investigate the impact of these variables on the calculation of the SSI in detail, using sets of images where no collapse events occurred.

2.2.1. Effect of Shadows

Shadow is well known as one of the main sources of error in the analysis of time-lapse aerial and satellite images, and its detection remains a challenging task [20,26,27,28]. Two types of shadows may affect the images: cast shadows, produced by sunlight being blocked by asperities of the slope, which are regular and have a fairly constant periodicity, and shadows produced by clouds, which are generally less sharp and unpredictable. Essentially, there are property-based and model-based methods for identifying and extracting shadows cast on the ground from digital images [27,28]. The first category of methods uses information about colors and textures in the images for shadow segmentation and can be applied to most shadows. The second category uses external models based on timing, latitude, longitude, and digital elevation models to estimate the sun’s position and forecast shadow casting. The latter approach is suitable for analyzing zenithal images and is particularly effective if elevation information can be integrated [29]. In our application, we will employ the first type of shadow detection methods.
To evaluate the influence of shadows cast by the sun on the SSI, we selected a study case involving five sunny days from 27/07/2020 to 31/07/2020, choosing consecutive hourly images taken from 11 AM to 5 PM. For these five sets of images, we computed the SSI using the 3 PM shot as the reference image. In each group of images, the shadow cast on the slope gradually decreases from 11 AM to 3 PM, with the images taken at 4 PM and 5 PM having almost no shadows, as shown in Figure 3.
As shown in Figure 4, the SSI is very high and changes little when the reference image is compared with images with less shadow cast (Δt=−1,+1,+2), while it significantly decreases and changes greatly when the reference image is compared with images with strong shadows. We also found in Figure 4 that as the shadow in the images gradually decreases (Δt=−4,−3,−2,−1), the difference between the SSIM results with and without the shadow filter decreases significantly. Moreover, the results of the five groups of tests maintained the same trend. The overall trend and the standard deviation shown in Figure 4 suggest that shadows have a significant impact on SSIM.

2.2.1. Effect of Illuminance

To evaluate the effect of illuminance on the structural similarity metric, we conducted five groups of image sequence comparison tests using photos with different illuminance levels. Vegetation and shadow filters were applied in these tests. Each group of tests comprised consecutive night images and day images taken within the same time interval. From 23/02/2021 to 28/02/2021, we selected four night-images (taken at 11 PM, 1 AM, 3 AM, 5 AM) and four day-images (taken at 11 AM, 1 PM, 3 PM, 5 PM) as test images to compare with the reference image (taken at 8 AM) in each group. Figure 5 shows an example set of images used in one of these tests. The SSI between the target image and the reference image is illustrated in Figure 6a. It can be observed that the SSI changes little when the reference image is compared with the night images (t=23, 01, 03, 05). This is due to the artificial illuminance provided by the spotlight, which ensures a constant level of brightness. Shadows cast are fixed and limited within the region of interest, since the illuminator is close to the cameras and directed almost along the same line-of-sight. On the other hand, the varying illuminance of the day images (t=11, 13, 15, 17) has a significant effect on SSI due to the substantial changes in sunlight intensity and direction, as discussed earlier. Moreover, the results of the five groups of tests consistently showed the same trend. It is worth mentioning that the reference images of Group 5 have different illuminance levels compared to the reference images of the other groups (see Figure 6b), resulting in a significantly lower SSI for Group 5 compared to the others.
These conclusions highlight the significant impact of illuminance on SSIM. Considering this influence, it is preferable to use images with stable illuminance (e.g., night images) for comparison in collapse detection. Addressing changes in illuminance on the landslide site during daytime monitoring poses a challenging problem, and thus, it is necessary to develop filters to mitigate the influence of illuminance changes, which will be addressed in the next section.

2.3. Image Filters

Several filters are used in our program to enhance the accuracy of the collapse detection phase from images. These filters can be broadly categorized into two groups: image pre-processing filters and noise filters applied directly to the Structural Similarity Map (SSM). The image pre-processing filters include a vegetation filter and a shadow filter. The vegetation filter removes areas covered by vegetation, which were visually identified and sampled from a set of pre-learning images. These sampled areas were used to create a deep learning mask [30]. Similarly, the shadow filter removes areas covered by shadows, which were also visually identified and sampled from a set of pre-learning images. These sampled areas were used to create a color mask. These filters are applied before comparing two images to prevent false-positive identification of collapsed areas due to movement of vegetation and changes in shadows.
The filters applied to the SSM include various image convolution filters, which smooth the noise generated by SSIM to avoid detection of small spots with low SSI and to consolidate detected collapsed areas. In this case, we used multiple median filters with kernel sizes of 23x23 and 7x7 pixels to preserve edges of the SSM [31,32,33], threshold filters [34], and Gaussian filters with a 19x19 kernel and standard deviation of 3 [35,36] for final smoothing. Figure 7 illustrates the specific roles of these filters in image comparison by showing the evolution of the SSM at each step.

2.4. Calculation of Collapsed Volumes

2.4.1. 3D Surface Reconstruction and Disparity Calculation

Once a collapse is detected, the program automatically calculates the collapsed volume. The first step in this phase involves the 3D reconstruction of the slope surface before and after the collapse using fixed calibrated cameras that enable a stereoscopic view of the scene [37,38,39]. Each camera must be calibrated with a series of shots taken from different angles to calculate the intrinsic parameters, including the focal length distortion parameters, as well as the extrinsic parameters, which include the position and orientation of each image [40]. The extrinsic parameters require the system to be geolocalized. This is achieved using fixed points and colored balls specially positioned on the landslide, whose exact positions are determined through a GPS survey [41]. The final image used for calibration is taken from the camera’s operational position to obtain its accurate pose.
For the reconstruction of the scene based on only two images, the principles of stereophotogrammetry were used. Given the large area to be monitored and its distance from the cameras, a larger baseline with two slightly converging lines of sight has been chosen. Using the intrinsic and extrinsic parameters of the two cameras, the stereo parameters can be derived [42,43]. This includes the intrinsic parameters of the individual cameras and a roto-translation matrix that allows conversion from the reference system of the left camera to the reference system of the right camera. This matrix provides the relative position between the two cameras.
The second step is to obtain undistorted and rectified images from the left and right cameras using the stereo camera parameters and considering an epipolar geometry[44,45,46]. To ensure the stereo images are well-aligned and the 3D reconstruction is as accurate as possible, it is necessary to correct for any image oscillations. This correction is achieved by aligning fixed points in the scene using the calibration image as a reference.
The third and final step involves calculating the disparity value of each pixel in the rectified image to obtain the disparity map, which contains the depth information of each pixel in space. Using the disparity map and stereo camera parameters [47,48,49], the 3D position of each point is calculated, resulting in the 3D point cloud.

2.4.2. Point Cloud Comparison and Volume Calculation

After the 3D reconstruction, the next step is to combine the information of the collapse area obtained with SSIM and the point cloud of the entire landslide. As described in the workflow of Figure 8, the first step involves using a median filter with a 10-by-10 kernel to remove the noise generated in the 3D reconstruction stage. To determine the collapse volume, we compute the difference between the point clouds before and after the collapse and apply a threshold filter with a 0.5m threshold. Subsequently, a median filter with a 21-by-21 kernel is used to remove any remaining noise. Finally, we search for continuous areas in the binary map and use the largest area as the approximate collapsed area.
Next, we dilate the boundary of the approximate collapsed area and fill the invalid points within the collapsed area on the depth information map (disparity map or Z-axis coordinate map), as illustrated in Figure 9. We then compare the point clouds again to determine the exact collapsed area and its pixel coordinates. This step is crucial because invalid points in the disparity map can cause some real collapsed areas to go unrecognized; hence, identifying the lost real collapsed areas near the boundaries is essential. Figure 10 shows an example from the Perarolo landslide site.
Based on the pixel coordinates of the collapsed area on the rectified image, we can identify the points on the surface of the collapsed area. As shown in Figure 11, by using all the point clouds on the surface of the collapsed area before and after the collapse, we can create the alpha shape representing the collapsed bodies and easily estimate the collapsed volumes [50,51,52]. This approach also allows us to visualize the spatial shape of the collapsed bodies and calculate their 3D coordinates.

3. Case Study and Results

3.1. Perarolo Landslide Site

The Perarolo Landslide, also known as the Sant’Andrea Landslide, is located in the Cadore area of the Belluno province in northeastern Italy [53]. This area is one of the most geohazard-prone regions in the southern Alpine mountains. The coordinates of the landslide are 46°23′44″N and 12°21′23″E close to the Alpi Carniche. The landslide is part of an older landslide on the southern flank of Mount Zucco, covering approximately 72,000 square meters and extending from an elevation of 490 to 580 meters above sea level.
As shown in Figure 12, the active portion of the landslide is near the left bank of the Boite River. A village is on the right bank along the river, with its central area located downstream relative to the landslide. Additionally, about 100 meters southeast from the bottom of the landslide, there is a road (SP42) that ascends the gentle slope on the southeast side of the landslide. This road features a bend adjacent to the northeast corner of the upper part of the landslide. In the 1980s, a small railway line ran along the upper section of the landslide. However, when landslide activity was detected, the railway was relocated to a tunnel inside the mountain. Furthermore, the Piave River flows southwest on the east side of the landslide body, intersecting with the Boite River approximately 150 meters southeast of the landslide’s base. Behind the village, there is a hillside with a height comparable to that of the landslide, where the cameras used for visual monitoring in this study are installed. As shown in Figure 13, the camera system consists of and three Canon EOS 1300D cameras, each mounted in a covered iron box and equipped with remote connection and a solar charging panel. Moreover, a searchlight is also present for night photos acquisition.

3.2. Image-Based Collapse Detection at the Perarolo Landslide Site

To confirm the capability of our photogrammetric tool based on SSIM to detect collapses, we conducted a test using an image sequence acquired at the Perarolo landslide site from July 2020 to July 2021. One image was taken per day, except during the winter months when the slope was partially or totally covered by snow, and interferometric data and total station data were unavailable. The trend of SSI between the images on day i and i+1 is shown in Figure 14, where sudden changes above the threshold indicate the occurrence of a collapse, in 4 different periods (from 08/07/2020 to 30/09/2020; from 01/10/2020 to 01/12/2020; from 15/02/2021 to 30/04/2021; from 30/04/2021 to 19/07/2021.). We defined that when the SSI is lower than 0.9998, a collapse has occurred between the two images. This threshold is calculated based on the minimum area of detached mass that we aim to detect, depending on the ground pixel size. Considering the camera resolution, the distance from the slope surface, and the mean angle between the line of sight and surface normal, an SSI lower than 0.9998 corresponds to a collapsed area larger than about 7 m². As shown in Figure 14, the program detected 13 distinct collapse events during the monitoring period. These events are consistent with on-site observations and sudden acceleration of displacement rates obtained from radar interferometry just before the events.
The first nine events correspond to small natural collapses, as shown in Figure 15b. The comparison between Figure 15a,b allows the identification of changes in the slope’s condition before and after the event of 28/02/2021. Figure 15c-f illustrate other significant collapses that occurred in the Perarolo landslide area during June-July 2021. As mentioned earlier, a major natural collapse happened on 09/06/2021; the algorithm detected this extreme condition, assigning an SSI of 0.9135 (Figure 15c). Subsequently, on 25/06/2021, an artificial collapse was induced by explosives to restore the safety condition of the slope (SSI = 0.9449, Figure 15d). Finally, in July, anthropic activities were carried out, where a bulldozer reshaped the toe of the landslide. These operations were also correctly detected by the algorithm, with SSIs of 0.9936 and 0.9941 for July 13 and 14, respectively (Figures 15e-f).
For all the cases, the program identified the location of the collapsed area on the original images, as shown in Figure 16. We can assess its accuracy by comparing the original images obtained before and after the events. Since image availability may not be ensured on some days due to bad weather conditions or other technical reasons, we tested the accuracy of our method by comparing pairs of images taken at different time intervals and computing the SSI, essentially estimating a temporal correlation function. For this purpose, we used a daily sequence of 25 images before a collapse event and 25 images after the collapse, from 06/08/2020 to 24/09/2020, to calculate the SSI between images on day i and i+Δt. As shown in Figure 16, among the possible pairs of images, they can be categorized according to the Δt, and grouped into two categories: pairs of images crossing the collapse event (shown in red in Figure 17 and called “with collapse”), and pairs not crossing the event (shown in black and called “without collapse”).
With boxplots using whiskers at 1.5 times the interquartile range, we found that the median of the “with collapse” group fluctuates little with the change of Δt, and there are very few outliers, as shown in Figure 18a. It was found that almost all the SSI values in the “with collapse” group were lower than the threshold value, indicating that the lack of images for several days after a collapse does not affect its detection. On the other hand, in the boxplot of image pairs “without collapse” (Figure 18b), we can see that there are cases where the first quartile (Q1) is below the threshold when Δt≥14 days. When Δt=20 days, the median is also below the threshold. This means that the time interval between two consecutive images should be less than 14 days to ensure a 75% probability of detection. When the interval is 20 days, this probability drops to 50%

3.3. Volume Calculation in Perarolo Landslide Site

Our photogrammetric tool automatically calculates the collapse volume for each detected event. Table 1 illustrates the volumes of collapse events occurring after 09/04/2021, calculated using our photogrammetric tool. To assess the accuracy of this approach, we conducted a couple of laser scanner surveys before and after each of the collapses on 09/06/2021 and 25/06/2021. From the relative point clouds, we measured the collapse volumes using CloudCompare software (see Figure 19), evaluating the distance in vertical direction between the slope surfaces before and after each event. We compared them with the results of our photogrammetric approach, obtaining that the relative errors were found to be 6.6% and 3.1%, respectively.

4. Discussion

The results of the case study at the landslide site demonstrate the capability of our photogrammetric tool in rapid collapse detection and volume calculation. This small-scale time-lapse photogrammetric tool allows for quick collection of collapse information for landslides that continue to collapse, which is crucial for understanding landslide characteristics, back-analysis of collapse causes, and further research toward developing a collapse prediction tool. The fully automatic, non-contact, and low-cost features of this technology make it easily acceptable and suitable for widespread adoption.
However, the results from the tests over multiple days in Section 3.2 have shown that using the current threshold of 0.9998, comparing two images with an interval of more than 14 days may cause the program to fail. It is important to understand the maximum allowable interval between two consecutive images for reliable program operation, especially when natural environmental factors such as fog can prevent image acquisition. In this case, we found that the probability of program failure is 50% when the interval is 20 days. For other probabilities (75% or 90%), we plan to conduct tests with longer image sequences that include a collapse event in the middle, as soon as suitable data becomes available in the future. To further improve the accuracy of collapse detection, it is necessary to study the SSIM specifically for geotechnical materials and utilize machine learning algorithms to identify the most suitable image sequences. These efforts are currently ongoing.

5. Conclusions

This paper presents a new automatic photogrammetric tool designed for rapid collapse detection and calculation of collapsed volume. Collapse detection is achieved through the structural similarity comparison of consecutive image sequences. We investigated the impact of natural environmental factors at the landslide site on the classic image structural similarity algorithm (SSIM) and developed multiple filters within the tool to eliminate noise and optimize detection accuracy. The algorithm for automatic calculation of collapsed volume relies on 3D reconstruction and point cloud comparison. By analyzing image sequences taken over one year at the Perarolo landslide site, along with laser scanner surveys, we validated the tool’s effectiveness under various conditions.
However, further analyses are necessary to refine this tool and obtain precise information on the dynamics of the Perarolo landslide and other camera-instrumented landslide sites. Specifically, ongoing work includes studying the SSIM specific to geotechnical materials and applying machine learning algorithms to enhance the selection of the most suitable image sequences.

Author Contributions

Conceptualization, F.G., Z.L. and L.B.; methodology, Z.L., F.G., A.P. and L.B.; software, Z.L. and F.G.; validation, L.B., Z.L. and A.P.; formal analysis, Z.L., F.G. and L.B.; investigation, L.B., Z.L. and A.P.; resources, F.G. and L.B.; data curation, Z.L. and F.G.; writing—original draft preparation, L.B., F.G. and Z.L; writing—review and editing, L.B. and A.P.; visualization, Z.L. and A.P.; supervision, F.G. and L.B.; project administration, F.G. and L.B.; funding acquisition, F.G. and L.B. All authors have read and agreed to the published version of the manuscript.

Funding

The research was financially supported by the Fondazione Cariverona through the research grant titled “Monitoring of Natural Hazards and Protective Structures Using Computer Vision Techniques for Environmental Safety and Preservation” and by the Veneto Region through the grant “ Scientific Support for the Characterization of Hydrogeological Risk and the Evaluation of the Effectiveness of Interventions Related to the Landslide Phenomenon of Busa del Cristo in Perarolo di Cadore (BL) through the Development of Predictive Geo-Hydrological Models”.

Conflicts of Interest

The authors declare no conflicts of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Sturzenegger, M.; Stead, D. Close-Range Terrestrial Digital Photogrammetry and Terrestrial Laser Scanning for Discontinuity Characterization on Rock Cuts. Eng Geol 2009, 106. [CrossRef]
  2. Stumpf, A.; Malet, J.P.; Allemand, P.; Ulrich, P. Surface Reconstruction and Landslide Displacement Measurements with Pléiades Satellite Images. ISPRS Journal of Photogrammetry and Remote Sensing 2014, 95. [CrossRef]
  3. Livio, F.A.; Bovo, F.; Gabrieli, F.; Gambillara, R.; Rossato, S.; Martin, S.; Michetti, A.M. Stability Analysis of a Landslide Scarp by Means of Virtual Outcrops: The Mt. Peron Niche Area (Masiere Di Vedana Rock Avalanche, Eastern Southern Alps). Front Earth Sci (Lausanne) 2022, 10. [CrossRef]
  4. Antonello, M.; Gabrieli, F.; Cola, S.; Menegatti, E. Automated Landslide Monitoring through a Low-Cost Stereo Vision System. In Proceedings of the CEUR Workshop Proceedings; 2013; Vol. 1107.
  5. Travelletti, J.; Delacourt, C.; Allemand, P.; Malet, J.P.; Schmittbuhl, J.; Toussaint, R.; Bastard, M. Correlation of Multi-Temporal Ground-Based Optical Images for Landslide Monitoring: Application, Potential and Limitations. ISPRS Journal of Photogrammetry and Remote Sensing 2012, 70. [CrossRef]
  6. Gabrieli, F.; Corain, L.; Vettore, L. A Low-Cost Landslide Displacement Activity Assessment from Time-Lapse Photogrammetry and Rainfall Data: Application to the Tessina Landslide Site. Geomorphology 2016, 269. [CrossRef]
  7. Ding, A.; Zhang, Q.; Zhou, X.; Dai, B. Automatic Recognition of Landslide Based on CNN and Texture Change Detection. In Proceedings of the Proceedings - 2016 31st Youth Academic Annual Conference of Chinese Association of Automation, YAC 2016; 2017.
  8. Ghorbanzadeh, O.; Meena, S.R.; Blaschke, T.; Aryal, J. UAV-Based Slope Failure Detection Using Deep-Learning Convolutional Neural Networks. Remote Sens (Basel) 2019, 11. [CrossRef]
  9. Ghorbanzadeh, O.; Blaschke, T.; Gholamnia, K.; Meena, S.R.; Tiede, D.; Aryal, J. Evaluation of Different Machine Learning Methods and Deep-Learning Convolutional Neural Networks for Landslide Detection. Remote Sens (Basel) 2019, 11. [CrossRef]
  10. Lei, T.; Zhang, Q.; Xue, D.; Chen, T.; Meng, H.; Nandi, A.K. End-to-End Change Detection Using a Symmetric Fully Convolutional Network for Landslide Mapping. In Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; 2019; Vol. 2019-May.
  11. Lv, Z.Y.; Shi, W.; Zhang, X.; Benediktsson, J.A. Landslide Inventory Mapping from Bitemporal High-Resolution Remote Sensing Images Using Change Detection and Multiscale Segmentation. IEEE J Sel Top Appl Earth Obs Remote Sens 2018, 11. [CrossRef]
  12. Amit, S.N.K.B.; Aoki, Y. Disaster Detection from Aerial Imagery with Convolutional Neural Network. In Proceedings of the Proceedings - International Electronics Symposium on Knowledge Creation and Intelligent Computing, IES-KCIC 2017; 2017; Vol. 2017-January.
  13. Ji, S.; Shen, Y.; Lu, M.; Zhang, Y. Building Instance Change Detection from Large-Scale Aerial Images Using Convolutional Neural Networks and Simulated Samples. Remote Sens (Basel) 2019, 11. [CrossRef]
  14. Lu, P.; Stumpf, A.; Kerle, N.; Casagli, N. Object-Oriented Change Detection for Landslide Rapid Mapping. IEEE Geoscience and Remote Sensing Letters 2011, 8. [CrossRef]
  15. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 2004, 13, 600–612. [CrossRef]
  16. Blanch, X.; Abellan, A.; Guinau, M. Point Cloud Stacking: A Workflow to Enhance 3D Monitoring Capabilities Using Time-Lapse Cameras. Remote Sens (Basel) 2020, 12. [CrossRef]
  17. Giacomini, A.; Thoeni, K.; Santise, M.; Diotri, F.; Booth, S.; Fityus, S.; Roncella, R. Temporal-Spatial Frequency Rockfall Data from Open-Pit Highwalls Using a Low-Cost Monitoring System. Remote Sens (Basel) 2020, 12. [CrossRef]
  18. Blanch, X.; Eltner, A.; Guinau, M.; Abellan, A. Multi-Epoch and Multi-Imagery (Memi) Photogrammetric Workflow for Enhanced Change Detection Using Time-Lapse Cameras. Remote Sens (Basel) 2021, 13. [CrossRef]
  19. Shen, B.; Chen, S.; Yin, J.; Mao, H. Image Recognition of Green Weeds in Cotton Fields Based on Color Feature. Nongye Gongcheng Xuebao/Transactions of the Chinese Society of Agricultural Engineering 2009, 25. [CrossRef]
  20. Chen, Q.; Zhang, G.; Yang, X.; Li, S.; Li, Y.; Wang, H.H. Single Image Shadow Detection and Removal Based on Feature Fusion and Multiple Dictionary Learning. Multimed Tools Appl 2018, 77.
  21. Wang, Z.; Bovik, A.C.; Sheikh, H.R.; Simoncelli, E.P. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE Transactions on Image Processing 2004, 13, 600–612. [CrossRef]
  22. Wang, Z.; Bovik, A.; Sheikh, H. Structural Similarity Based Image Quality Assessment. Digital Video Image Quality and Perceptual Coding, Ser. Series in Signal Processing and Communications 2005. [CrossRef]
  23. Lim, J.S. Two-Dimensional Signal and Image Processing. Englewood Cliffs, NJ, Prentice Hall 1990, 710.
  24. Brunet, D.; Vrscay, E.R.; Wang, Z. On the Mathematical Properties of the Structural Similarity Index. IEEE Transactions on Image Processing 2012, 21. [CrossRef]
  25. Wang, Z.; Simoncelli, E.P.; Bovik, A.C. Multiscale Structural Similarity for Image Quality Assessment. In Proceedings of the The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003; 2003; Vol. 2, pp. 1398-1402 Vol.2.
  26. Liasis, G.; Stavrou, S. Satellite Images Analysis for Shadow Detection and Building Height Estimation. ISPRS Journal of Photogrammetry and Remote Sensing 2016, 119. [CrossRef]
  27. Arévalo, V.; González, J.; Ambrosio, G. Shadow Detection in Colour High-Resolution Satellite Images. Int J Remote Sens 2008, 29. [CrossRef]
  28. Adeline, K.R.M.; Chen, M.; Briottet, X.; Pang, S.K.; Paparoditis, N. Shadow Detection in Very High Spatial Resolution Aerial Images: A Comparative Study. ISPRS Journal of Photogrammetry and Remote Sensing 2013, 80. [CrossRef]
  29. Li, F.; Jupp, D.L.B.; Thankappan, M.; Lymburner, L.; Mueller, N.; Lewis, A.; Held, A. A Physics-Based Atmospheric and BRDF Correction for Landsat Data over Mountainous Terrain. Remote Sens Environ 2012, 124. [CrossRef]
  30. Hua, S.; Shi, P. GrabCut Color Image Segmentation Based on Region of Interest. In Proceedings of the Proceedings - 2014 7th International Congress on Image and Signal Processing, CISP 2014; 2014.
  31. Ko, S.J.; Lee, Y.H. Center Weighted Median Filters and Their Applications to Image Enhancement. IEEE Trans Circuits Syst 1991, 38. [CrossRef]
  32. Hwang, H.; Haddad, R.A. Adaptive Median Filters: New Algorithms and Results. IEEE Transactions on Image Processing 1995, 4. [CrossRef]
  33. Shrestha, S. Image Denoising Using New Adaptive Based Median Filter. Signal Image Process 2014, 5. [CrossRef]
  34. Reddi, S.S.; Rudin, S.F.; Keshavan, H.R. An Optimal Multiple Threshold Scheme for Image Segmentation. IEEE Trans Syst Man Cybern 1984, SMC-14. [CrossRef]
  35. Deng, G.; Cahill, L.W. Adaptive Gaussian Filter for Noise Reduction and Edge Detection. In Proceedings of the IEEE Nuclear Science Symposium & Medical Imaging Conference; 1994.
  36. Shin, D.H.; Park, R.H.; Yang, S.; Jung, J.H. Block-Based Noise Estimation Using Adaptive Gaussian Filtering. IEEE Transactions on Consumer Electronics 2005, 51. [CrossRef]
  37. Tack, F.; Buyuksalih, G.; Goossens, R. 3D Building Reconstruction Based on given Ground Plan Information and Surface Models Extracted from Spaceborne Imagery. ISPRS Journal of Photogrammetry and Remote Sensing 2012, 67. [CrossRef]
  38. Liu, S.; Zhao, L.; Li, J. The Applications and Summary of Three Dimensional Reconstruction Based on Stereo Vision. In Proceedings of the Proceedings of the 2012 International Conference on Industrial Control and Electronics Engineering, ICICEE 2012; 2012.
  39. Zhang, Z. A Flexible New Technique for Camera Calibration. IEEE Trans Pattern Anal Mach Intell 2000, 22. [CrossRef]
  40. Luhmann, T.; Fraser, C.; Maas, H.G. Sensor Modelling and Camera Calibration for Close-Range Photogrammetry. ISPRS Journal of Photogrammetry and Remote Sensing 2016, 115.
  41. Colomina, I.; Molina, P. Unmanned Aerial Systems for Photogrammetry and Remote Sensing: A Review. ISPRS Journal of Photogrammetry and Remote Sensing 2014, 92.
  42. Heikkila, J.; Silven, O. Four-Step Camera Calibration Procedure with Implicit Image Correction. In Proceedings of the Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 1997.
  43. Scaioni, M.; Crippa, J.; Longoni, L.; Papini, M.; Zanzi, L. IMAGE-BASED RECONSTRUCTION and ANALYSIS of DYNAMIC SCENES in A LANDSLIDE SIMULATION FACILITY. In Proceedings of the ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences; 2017; Vol. 4.
  44. Kjær-Nielsen, A.; Jensen, L.B.W.; SøSrensen, A.S.; Krüger, N. A Real-Time Embedded System for Stereo Vision Preprocessing Using an FPGA. In Proceedings of the Proceedings - 2008 International Conference on Reconfigurable Computing and FPGAs, ReConFig 2008; 2008.
  45. Junger, C.; Hess, A.; Rosenberger, M.; Notni, G. FPGA-Based Lens Undistortion and Image Rectification for Stereo Vision Applications.; 2019.
  46. Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; 2004;
  47. Hamzah, R.A.; Ibrahim, H. Literature Survey on Stereo Vision Disparity Map Algorithms. J Sens 2016, 2016.
  48. Bleyer, M.; Gelautz, M. A Layered Stereo Matching Algorithm Using Image Segmentation and Global Visibility Constraints. ISPRS Journal of Photogrammetry and Remote Sensing 2005, 59. [CrossRef]
  49. Georgoulas, C.; Kotoulas, L.; Sirakoulis, G.C.; Andreadis, I.; Gasteratos, A. Real-Time Disparity Map Computation Module. Microprocess Microsyst 2008, 32. [CrossRef]
  50. Xu, X.; Harada, K. Automatic Surface Reconstruction with Alpha-Shape Method. Visual Computer 2003, 19. [CrossRef]
  51. Hadas, E.; Borkowski, A.; Estornell, J.; Tymkow, P. Automatic Estimation of Olive Tree Dendrometric Parameters Based on Airborne Laser Scanning Data Using Alpha-Shape and Principal Component Analysis. GIsci Remote Sens 2017, 54. [CrossRef]
  52. Carrea, D.; Abellan, A.; Derron, M.H.; Gauvin, N.; Jaboyedoff, M. Matlab Virtual Toolbox for Retrospective Rockfall Source Detection and Volume Estimation Using 3d Point Clouds: A Case Study of a Subalpine Molasse Cliff. Geosciences (Switzerland) 2021, 11. [CrossRef]
  53. Brezzi, L.; Carraro, E.; Pasa, D.; Teza, G.; Cola, S.; Galgaro, A. Post-Collapse Evolution of a Rapid Landslide from Sequential Analysis with FE and SPH-Based Models. Geosciences (Switzerland) 2021, 11. [CrossRef]
Preprints 111751 g001
Figure 1. Scheme of the various steps composing the collapse detection algorithm.
Figure 1. Scheme of the various steps composing the collapse detection algorithm.
Preprints 111751 g001
Preprints 111751 g002
Figure 2. Example of the input images of the monitoring area used for comparison and the corresponding Structural Similarity Map (SSM) generated by the SSIM algorithm.
Figure 2. Example of the input images of the monitoring area used for comparison and the corresponding Structural Similarity Map (SSM) generated by the SSIM algorithm.
Preprints 111751 g002
Preprints 111751 g003
Figure 3. Example set of images used for the shadow test. Consecutive hourly images taken from 11 AM to 5 PM on sunny days from 27/07/2020 to 31/07/2020. The shadow cast on the slope gradually decreases from 11 AM to 3 PM, with minimal shadows in the images at 4 PM and 5 PM.
Figure 3. Example set of images used for the shadow test. Consecutive hourly images taken from 11 AM to 5 PM on sunny days from 27/07/2020 to 31/07/2020. The shadow cast on the slope gradually decreases from 11 AM to 3 PM, with minimal shadows in the images at 4 PM and 5 PM.
Preprints 111751 g003
Preprints 111751 g004
Figure 4. Influence of shadow cast on the Structural Similarity Index (SSI). The SSI values for images with varying degrees of shadow cast compared to a reference image taken at 3 PM. The difference (∆t) indicates the time difference from the reference image.
Figure 4. Influence of shadow cast on the Structural Similarity Index (SSI). The SSI values for images with varying degrees of shadow cast compared to a reference image taken at 3 PM. The difference (∆t) indicates the time difference from the reference image.
Preprints 111751 g004
Preprints 111751 g005
Figure 5. Images used in one group (Group 1) of image sequence comparison tests. The images from all groups show a consistent trend of illuminance change.
Figure 5. Images used in one group (Group 1) of image sequence comparison tests. The images from all groups show a consistent trend of illuminance change.
Preprints 111751 g005
Preprints 111751 g006aPreprints 111751 g006b
Figure 6. (a) SSI between reference image and target images taken at different times within one day. (b) Reference images used in image sequence comparison tests to identify the influence of illuminance. The reference images in Group 5 have different illuminance levels compared to the reference images in the other groups.
Figure 6. (a) SSI between reference image and target images taken at different times within one day. (b) Reference images used in image sequence comparison tests to identify the influence of illuminance. The reference images in Group 5 have different illuminance levels compared to the reference images in the other groups.
Preprints 111751 g006aPreprints 111751 g006b
Preprints 111751 g007
Figure 7. Scheme of filter application in the algorithm, highlighting the effects of each filtering step. The image illustrates the application of various filters in the image comparison program. SSI is lowest when no filters are applied and increases with the addition of filters.
Figure 7. Scheme of filter application in the algorithm, highlighting the effects of each filtering step. The image illustrates the application of various filters in the image comparison program. SSI is lowest when no filters are applied and increases with the addition of filters.
Preprints 111751 g007
Preprints 111751 g008
Figure 8. Steps of the algorithm for calculating the volume of collapsed material.
Figure 8. Steps of the algorithm for calculating the volume of collapsed material.
Preprints 111751 g008
Preprints 111751 g009
Figure 9. Method for identifying and filling invalid points with the nearest valid points in the collapsed area.
Figure 9. Method for identifying and filling invalid points with the nearest valid points in the collapsed area.
Preprints 111751 g009
Preprints 111751 g010
Figure 10. Process of refining the approximate collapse area to determine the exact collapse area, illustrated with data from the Perarolo landslide site collapse (Belluno Provice, Italy) on 09/06/2021.
Figure 10. Process of refining the approximate collapse area to determine the exact collapse area, illustrated with data from the Perarolo landslide site collapse (Belluno Provice, Italy) on 09/06/2021.
Preprints 111751 g010
Preprints 111751 g011
Figure 11. Method for creating an Alpha shape to represent collapsed bodies for volume calculation, demonstrated with data from the Perarolo landslide site collapse on 09/06/2021.
Figure 11. Method for creating an Alpha shape to represent collapsed bodies for volume calculation, demonstrated with data from the Perarolo landslide site collapse on 09/06/2021.
Preprints 111751 g011
Preprints 111751 g012
Figure 12. Overview of the Perarolo landslide site, showing the location of the monitoring system, the village on the right bank of the Boite river, and other geographic features.
Figure 12. Overview of the Perarolo landslide site, showing the location of the monitoring system, the village on the right bank of the Boite river, and other geographic features.
Preprints 111751 g012
Preprints 111751 g013
Figure 13. (a) Searchlight for night photos; (b) view of one of the three photographic system used for monitoring the Perarolo landslide; (c) detailed view of the hardware components of the system for the automatic acquisition and transmission of the time-lapse images.
Figure 13. (a) Searchlight for night photos; (b) view of one of the three photographic system used for monitoring the Perarolo landslide; (c) detailed view of the hardware components of the system for the automatic acquisition and transmission of the time-lapse images.
Preprints 111751 g013
Preprints 111751 g014
Figure 14. The trend of SSI between consecutive daily images over the monitoring period where values below the threshold (red dots) indicate collapse events. (a) from 09/07/2020 to 30/09/2020; (b) from 01/10/2020 to 01/12/2020; (c) from 16/02/2021 to 30/04/2021; (d) from 01/05/2021 to 19/07/2021.
Figure 14. The trend of SSI between consecutive daily images over the monitoring period where values below the threshold (red dots) indicate collapse events. (a) from 09/07/2020 to 30/09/2020; (b) from 01/10/2020 to 01/12/2020; (c) from 16/02/2021 to 30/04/2021; (d) from 01/05/2021 to 19/07/2021.
Preprints 111751 g014
Preprints 111751 g015
Figure 15. Images of the major collapse events recorded on the main landslide scarp. (a) Slope condition on 09/06/2021; (b) after the major natural collapse on 09/06/2021, detected with an SSI of 0.9135; (c) after the artificial collapse induced by explosives on 25/06/2021 to restore slope safety, detected with an SSI of 0.9449; (d) after the anthropic activity on 13/07/2021, where a bulldozer reshaped the toe of the landslide, detected with an SSI of 0.9936; (e) during the continuation of anthropic activity on 14/07/2021, detected with an SSI of 0.9941.
Figure 15. Images of the major collapse events recorded on the main landslide scarp. (a) Slope condition on 09/06/2021; (b) after the major natural collapse on 09/06/2021, detected with an SSI of 0.9135; (c) after the artificial collapse induced by explosives on 25/06/2021 to restore slope safety, detected with an SSI of 0.9449; (d) after the anthropic activity on 13/07/2021, where a bulldozer reshaped the toe of the landslide, detected with an SSI of 0.9936; (e) during the continuation of anthropic activity on 14/07/2021, detected with an SSI of 0.9941.
Preprints 111751 g015
Preprints 111751 g016
Figure 16. Location of the collapsed area identified on the original images (in black, a, b), showing the accuracy of the photogrammetric tool by comparing images (c) before and (d) after the events.
Figure 16. Location of the collapsed area identified on the original images (in black, a, b), showing the accuracy of the photogrammetric tool by comparing images (c) before and (d) after the events.
Preprints 111751 g016
Preprints 111751 g017
Figure 17. Test to verify the accuracy of our program at different time lags. Using 50 consecutive images taken from 06/08/2020 to 24/09/2020, a collapse event occurred between 30/08/2020 and 31/08/2020. The red connectors identify the pairs of images in which a collapse is detected, while the black ones represent the pairs of images without collapse.
Figure 17. Test to verify the accuracy of our program at different time lags. Using 50 consecutive images taken from 06/08/2020 to 24/09/2020, a collapse event occurred between 30/08/2020 and 31/08/2020. The red connectors identify the pairs of images in which a collapse is detected, while the black ones represent the pairs of images without collapse.
Preprints 111751 g017
Preprints 111751 g018
Figure 18. Boxplot showing the distribution of structural similarity index (SSI) values at different time lags (∆t) for image pairs with (a) and without (b) collapse events. The whiskers extend to 1.5 times the interquartile range (IQR).
Figure 18. Boxplot showing the distribution of structural similarity index (SSI) values at different time lags (∆t) for image pairs with (a) and without (b) collapse events. The whiskers extend to 1.5 times the interquartile range (IQR).
Preprints 111751 g018
Preprints 111751 g019
Figure 19. Comparison of collapse volume measured using laser scanner and our photogrammetric algorithms during the collapse event on (a) 09/06/2021 and (b) 25/06/2021, using CloudCompare software. The contour plots indicate the vertical distance, in z-direction, between the slope surfaces before and after each collapse.
Figure 19. Comparison of collapse volume measured using laser scanner and our photogrammetric algorithms during the collapse event on (a) 09/06/2021 and (b) 25/06/2021, using CloudCompare software. The contour plots indicate the vertical distance, in z-direction, between the slope surfaces before and after each collapse.
Preprints 111751 g019
Table 1. Volume of some collapse events detected after 09/04/2021 calculated by our photogrammetric tool.
Table 1. Volume of some collapse events detected after 09/04/2021 calculated by our photogrammetric tool.
Date Type of Collapse Volume (m3) SSI
09/04/2021 Natural 353.2 0.9971
09/06/2021 Natural 8558.7 0.9135
25/06/2021 Explosive 2739.6 0.9449
13/07/2021 Anthropic activity 217.9 0.9936
14/07/2021 Anthropic activity 457.9 0.9941
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
Prerpints.org logo

Preprints.org is a free preprint server supported by MDPI in Basel, Switzerland.

facebook logotwitter logolinkedin logo
MDPI Initiatives
Important Links
Subscribe

Disclaimer

Terms of Use

Privacy Policy

© 2024 MDPI (Basel, Switzerland) unless otherwise stated