1. Introduction

2. Main Methods

3. Materials and Methods

• calibration of the camera is used to estimate its intrinsic parameters, including radial distortion,
• calibration of the projector consists of estimating the geometric parameters of the cone (vertex ${\mathit{O}}_{\mathcal{P}}$, direction $\mathit{d}$, half-angle of opening $\alpha$),
• local 3D reconstruction based on the camera / projector triangulation leading to a 3D point cloud expressed in $\mathcal{C}$, the camera frame.

3.1. Cone Parameters

3.1.1. Cone of Revolution

The cone of revolution, referred to as a cone in the remainder of this document, is associated with its surface of revolution and not with its solid, as may be the case in certain applications. The terms we will use to define the relationship between a point and the cone are as follows:

• a point belonging to the cone is a point which belongs to the surface of the cone,
• a point inside the cone is a point which belongs to the solid bounded by the surface of the cone,
• a point outside the cone is a point which does not belong to the solid bounded by the surface of the cone.

This surface is generated by the revolution of a line, called the generatrix, which we will note g and which passes through a vertex, in this case ${\mathit{O}}_{\mathcal{P}}$.

The revolution takes place around a fixed axis which also passes through the vertex and which turns out to be an axis of symmetry of the cone, the direction of which is defined by the unit vector $\mathit{d}$.

If a plane not passing through ${\mathit{O}}_{\mathcal{P}}$ is orthogonal to the axis of symmetry, then its intersection with the cone is a circle.

Thus, the angle formed by g and the axis of symmetry is constant and corresponds to the half-angle of the cone opening $\alpha \in ]0;\frac{\pi }{2}[$.

The cone is represented in Figure 5 with the orthonormal base $({\mathit{O}}_{\mathcal{P}},{\mathit{x}}_{\mathcal{P}},{\mathit{y}}_{\mathcal{P}},{\mathit{z}}_{\mathcal{P}})$, oriented such that ${\mathit{z}}_{\mathcal{P}}=\mathit{d}$, which defines the $\mathcal{P}$ reference frame associated with the cone.

The generatrix g has the unit direction vector $\mathit{w}$ which is set by a new angle $\theta \in [0;2\pi [$ (Figure 5).

In the cone frame $\mathcal{P}$, we can simply express the vector ${\mathit{w}}_{\left\{\mathcal{P}\right\}}$ which then depends only on the angles $\theta$ and $\alpha$ :

$${\mathit{w}}_{\left\{\mathcal{P}\right\}}=\left[\begin{array}{c}sin\left(\alpha \right)cos\left(\theta \right)\\ sin\left(\alpha \right)sin\left(\theta \right)\\ cos\left(\alpha \right)\end{array}\right]$$

(1)

This vector can be used to express all the points on the surface parametrically.

Let ${\mathit{X}}_{\left\{\mathcal{P}\right\}}={\left[\begin{array}{ccc}{x}_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}& y_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}& z_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}\end{array}\right]}^T$ be a point $\mathit{X}$ expressed in the reference frame $\mathcal{P}$. Then, if $\mathit{X}$ belongs to the cone we have :

$${\mathit{X}}_{\left\{\mathcal{P}\right\}}=l{\mathit{w}}_{\left\{\mathcal{P}\right\}}\iff \left\{\begin{array}{c}\begin{array}{cc}\hfill & x_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}=lsin\left(\alpha \right)cos\left(\theta \right)\hfill \\ \hfill & y_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}=lsin\left(\alpha \right)sin\left(\theta \right)\hfill \\ \hfill & z_{{\mathit{X}}_{\left\{\mathcal{P}\right\}}}=lcos\left(\alpha \right)\hfill \end{array}\hfill \end{array}\right.$$

(2)

with l a real parameter whose absolute value defines the distance between ${\mathit{X}}_{\left\{\mathcal{P}\right\}}$ and the cone vertex. However, with $l\in \mathbb{R}$, the parametric equation defines a "double cone". However, since the cone models our light projector, we are only interested in the upper part of the "double cone", i.e. when the parameter $l\in {\mathbb{R}}^{+}$.

3.1.2. Orthogonal Distance

This parametrisation of the cone will help us to write the equation for the orthogonal distance between a point and the cone, an equation that will be useful for cone estimation.

Let $\mathit{M}$ be a point outside the cone and $\mathit{H}$ its orthogonal projection on the cone (see Figure 5).

The angle ${\theta }_H$ is the angle that gives the generatrix of the cone closest to $\mathit{M}$. This generatrix, called $g_H$, passes through $\mathit{H}$ and has the unit direction vector ${\mathit{w}}_H$ whose expression in the $\mathcal{P}$ reference frame is :

$${{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}=\left[\begin{array}{c}sin\left(\alpha \right)cos\left({\theta }_H\right)\\ sin\left(\alpha \right)sin\left({\theta }_H\right)\\ cos\left(\alpha \right)\end{array}\right]$$

If we also express $\mathit{M}$ in $\mathcal{P}$ with ${\mathit{M}}_{\left\{\mathcal{P}\right\}}={\left[\begin{array}{ccc}{x}_{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}& y_{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}& z_{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}\end{array}\right]}^T$, we obtain the following expressions:

$$\left\{\begin{array}{c}\begin{array}{cc}\hfill {\theta }_H& =arctan\left(\frac{y_{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}}{x_{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}}\right)\hfill \\ \hfill {\mathit{H}}_{\left\{\mathcal{P}\right\}}& =\left({{\mathit{M}}_{\left\{\mathcal{P}\right\}}}^T·{{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}\right){{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}\hfill \end{array}\hfill \end{array}\right.$$

(3)

If ${{\mathit{M}}_{\left\{\mathcal{P}\right\}}}^T·{{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}<0$ then the projected point is in the lower part of the cone. As we are only working with the upper part of the cone, when this condition is true, the nearest point is the vertex of the ${\mathit{O}}_{\mathcal{P}}$ cone (see example Figure 6). The orthogonal distance h is therefore :

$$h=\left\{\begin{array}{c}\begin{array}{cccc}\hfill & \sqrt{{({\mathit{M}}_{\left\{\mathcal{P}\right\}}-{\mathit{H}}_{\left\{\mathcal{P}\right\}})}^T({\mathit{M}}_{\left\{\mathcal{P}\right\}}-{\mathit{H}}_{\left\{\mathcal{P}\right\}})}\hfill & \hfill & \mathrm{si}{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}^T{{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}⩾0\hfill \\ \hfill & \sqrt{{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}^T{\mathit{M}}_{\left\{\mathcal{P}\right\}}}\hfill & \hfill & \mathrm{si}{{\mathit{M}}_{\left\{\mathcal{P}\right\}}}^T{{\mathit{w}}_H}_{\left\{\mathcal{P}\right\}}<0\hfill \end{array}\hfill \end{array}\right.$$

(4)

3.1.3. Quadratic form of the Cone

To obtain this relationship, we must first note that if the angle between the vectors $(\mathit{X}-{\mathit{O}}_{\mathcal{P}})$ and $\mathit{d}$ or $(\mathit{X}-{\mathit{O}}_{\mathcal{P}})$ and $-\mathit{d}$ is equal to the angle $\alpha$, then the point $\mathit{X}$ belongs to the cone. This is equivalent to the following scalar product:

$${(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\mathit{d}=\pm ∥\mathit{X}-{\mathit{O}}_{\mathcal{P}}∥cos\left(\alpha \right)$$

When squared, the expression becomes :

$$\begin{array}{cc}\hfill & {\left({(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\mathit{d}\right)}^2={\left(∥\mathit{X}-{\mathit{O}}_{\mathcal{P}}∥cos\left(\alpha \right)\right)}^2\hfill \\ \hfill \iff & \left({(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\mathit{d}\right)\left({(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\mathit{d}\right)={(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T(\mathit{X}-{\mathit{O}}_{\mathcal{P}}){cos}^2\left(\alpha \right)\hfill \\ \hfill \iff & {(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\mathit{d}{\mathit{d}}^T(\mathit{X}-{\mathit{O}}_{\mathcal{P}})-{(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T(\mathit{X}-{\mathit{O}}_{\mathcal{P}}){cos}^2\left(\alpha \right)=0\hfill \\ \hfill \iff & {(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^T\underset{Q}{\underbrace{\left[\begin{array}{c}\mathit{d}{\mathit{d}}^T-{cos}^2\left(\alpha \right)I_3\end{array}\right]}}(\mathit{X}-{\mathit{O}}_{\mathcal{P}})=0\hfill \end{array}$$

We thus obtain a quadratic function of the cone in the general case, directly showing the parameters $\mathit{d}$, $\alpha$ and ${\mathit{O}}_{\mathcal{P}}$ which define it:

$$\mathcal{Q}\left(\mathit{X}\right)={(\mathit{X}-{\mathit{O}}_{\mathcal{P}})}^{T}Q(\mathit{X}-{\mathit{O}}_{\mathcal{P}})$$

(5)

This gives the relationships between the point $\mathit{X}$ and the cone:

3.2. Projector Calibration

Projector calibration is an essential step, since it involves estimating the parameters of the cone used in our method and in the equations we have seen so far. As a reminder, the parameters of the cone to be estimated are its vertex ${\mathit{O}}_{\mathcal{P}}$, its direction vector $\mathit{d}$ and its half-angle of aperture $\alpha$.

Since ${\mathit{O}}_{\mathcal{P}}$ and $\mathit{d}$ represent the relative pose of the cone with respect to the camera, the projector must be fixed with respect to the camera during calibration and during any experimentation. Furthermore, if the projector has a variable half-angle aperture, it is also necessary to ensure that this is fixed.

3.2.1. 3D Point Generation for Cone Estimation

The first step in the calibration method is to generate a cloud of 3D points belonging to the cone. To achieve it, the chosen method is to capture several light projection images on a flat surface.

This light projection is the result of the intersection between the projector cone and the flat surface, which is by definition a conic. In our case, it will be an ellipse because we want a closed shape. The projection of this ellipse into the image is also an ellipse.

If we can estimate the relative pose of this surface with respect to the camera and extract the light contour in the image, which is an ellipse, then we can obtain the real ellipse from the ellipse in the image.

To estimate the relative pose, we attached a chessboard pattern to the surface to estimate its relative pose. This relative pose is obtained by estimating the homography between the calibration pattern and the image plane [25]. As the camera is calibrated, rotation and translation can be estimated from the homography.

To extract the ellipse from the image, we again use the contour detection method presented in [26,27].

Then, from the contour points obtained, we estimate the best ellipse using the method of [28] which is based on least squares optimisation.

It is therefore possible to obtain n elliptical sections of the cone from n images taken at different distances from the surface. For each ellipse extracted in an image, we obtain the ellipse in the surface via the estimated homography between the image plane and this surface. The estimated relative pose allows us to obtain the elliptical section, i.e. the set of 3D points expressed in the camera frame of reference belonging to the ellipse of the surface.

Figure 8 shows an example where the camera has captured three images of an ellipse surrounding a chessboard pattern on the flat surface in three different poses. This is equivalent to obtaining three elliptical sections of the same cone.

Figure 7. Example of detecting the contour of a halo of light on a wall using a chessboard to estimate the plane pose. The contour is in green and the adjusted ellipse in blue.

Figure 7. Example of detecting the contour of a halo of light on a wall using a chessboard to estimate the plane pose. The contour is in green and the adjusted ellipse in blue.

Consequently, if we have obtained n elliptical sections where m 3D points are extracted per section, we generate a cloud of $n\times m$ 3D points belonging to the cone.

3.2.2. Cone Estimation

The aim is now to estimate the cone that best approximates the 3D point cloud. This is done by geometric fitting, i.e. by minimising the orthogonal distances between the 3D points and the cone.

Let $\mathit{p}$ be the vector containing the six parameters of the cone, which are :

• the three coordinates of its vertex ${\mathit{O}}_{\mathcal{P}}$,
• the two angles yaw and pitch in ZYX-Euler convention of its direction vector $\mathit{d}$, the angle roll not being necessary since a cone has an axis of symmetry,
• its opening half-angle $\alpha$.

To estimate the $\mathit{p}$ vector, we use the Levenberg-Marquardt (LM) iterative method [29]. For the first iteration, we need to choose an initial parameter vector ${\mathit{p}}_0$. This could be the null vector, for example. In practice, we will use measurements taken with a caliper for the various parameters of ${\mathit{p}}_0$.

At each iteration i, LM receives a solution vector ${\mathit{p}}_i$ and the function for calculating the vector of orthogonal distances to be minimised. Our function first calculates the rotation and translation of the cone relative to the camera from ${\mathit{O}}_{\mathcal{P}}$ and the yaw and pitch angles of the vector ${\mathit{p}}_i$. It then expresses the 3D points in the cone reference frame and calculates the orthogonal distance vector between the 3D points and the cone using the equation 4.

So from ${\mathit{p}}_i$ and this function, LM returns a new solution vector ${\mathit{p}}_{i+1}$.

When the difference between the sum of the squared orthogonal distances obtained with ${\mathit{p}}_j$ and the calculus obtained with ${\mathit{p}}_{j+1}$ is less than an arbitrary threshold, this means that LM has reached convergence at iteration j. The best solution in the least squares sense is therefore the vector ${\mathit{p}}_j$.

It is important to note that there are an infinite number of cones that share the same elliptical section. On the other hand, there is only one cone that passes through an elliptical section and through a point outside that section. Consequently, this minimisation can only work if we have at least two elliptical sections.

4. Experiments in a Waterless Aqueduct

5. Discussion

6. Conclusions

References

1. Gleick, P.H. Water in crisis. Pacific Institute for Studies in Dev., Environment & Security. Stockholm Env. Institute, Oxford Univ. Press. 473p 1993, 9, 1051–0761.
2. Massot-Campos, M.; Oliver-Codina, G. Optical sensors and methods for underwater 3D reconstruction. Sensors 2015, 15, 31525–31557. [Google Scholar] [CrossRef] [PubMed]
3. Hu, K.; Wang, T.; Shen, C.; Weng, C.; Zhou, F.; Xia, M.; Weng, L. Overview of Underwater 3D Reconstruction Technology Based on Optical Images. Journal of Marine Science and Engineering, 11, 949. [CrossRef]
4. Snyder, J. Doppler Velocity Log (DVL) navigation for observation-class ROVs. In Proceedings of the OCEANS 2010 MTS/IEEE SEATTLE; IEEE, 2010; pp. 1–9. [Google Scholar] [CrossRef]
5. Lee, C.M.; Lee, P.M.; Hong, S.W.; Kim, S.M.; Seong, W.; et al. Underwater navigation system based on inertial sensor and doppler velocity log using indirect feedback kalman filter. International Journal of Offshore and Polar Engineering 2005, 15. [Google Scholar]
6. Brown, C.J.; Blondel, P. Developments in the application of multibeam sonar backscatter for seafloor habitat mapping. Applied Acoustics 2009, 70, 1242–1247. [Google Scholar] [CrossRef]
7. Gary, M.; Fairfield, N.; Stone, W.C.; Wettergreen, D.; Kantor, G.; Sharp, J.M., Jr. 3D mapping and characterization of Sistema Zacatón from DEPTHX (DE ep P hreatic TH ermal e X plorer). In Sinkholes and the Engineering and Environmental Impacts of Karst; 2008; pp. 202–212. [CrossRef]
8. Kantor, G.; Fairfield, N.; Jonak, D.; Wettergreen, D. Experiments in navigation and mapping with a hovering AUV. In Proceedings of the Field and Service Robotics. Springer; 2008; pp. 115–124. [Google Scholar] [CrossRef]
9. Mallios, A.; Ridao, P.; Ribas, D.; Carreras, M.; Camilli, R. Toward autonomous exploration in confined underwater environments. Journal of Field Robotics 2016, 33, 994–1012. [Google Scholar] [CrossRef]
10. Hartley, R.I.; Sturm, P. Triangulation. Computer vision and image understanding 1997, 68, 146–157. [Google Scholar] [CrossRef]
11. Massot-Campos, M.; Oliver-Codina, G. One-shot underwater 3D reconstruction. In Proceedings of the 2014 IEEE Emerging Technology and Factory Automation (ETFA).; IEEE, 2014; pp. 1–4. [Google Scholar] [CrossRef]
12. Meline, A.; Triboulet, J.; Jouvencel, B. Comparative study of two 3D reconstruction methods for underwater archaeology. In Proceedings of the 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems; IEEE, 2012; pp. 740–745. [Google Scholar] [CrossRef]
13. Onmek, Y.; Triboulet, J.; Druon, S.; Meline, A.; Jouvencel, B. Evaluation of underwater 3D reconstruction methods for Archaeological Objects: Case study of Anchor at Mediterranean Sea. In Proceedings of the 2017 3rd International Conference on Control, Automation and Robotics (ICCAR); IEEE, 2017; pp. 394–398. [Google Scholar] [CrossRef]
14. Brandou, V.; Allais, A.G.; Perrier, M.; Malis, E.; Rives, P.; Sarrazin, J.; Sarradin, P.M. 3D reconstruction of natural underwater scenes using the stereovision system IRIS. In Proceedings of the OCEANS 2007-Europe; IEEE, 2007; pp. 1–6. [Google Scholar] [CrossRef]
15. Beall, C.; Lawrence, B.J.; Ila, V.; Dellaert, F. 3D reconstruction of underwater structures. In Proceedings of the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems; IEEE, 2010; pp. 4418–4423. [Google Scholar] [CrossRef]
16. Zhou, Y.; Li, Q.; Ye, Q.; Yu, D.; Yu, Z.; Liu, Y. A binocular vision-based underwater object size measurement paradigm: Calibration-Detection-Measurement (C-D-M). Measurement, 216, 112997. [CrossRef]
17. Weidner, N.; Rahman, S.; Li, A.Q.; Rekleitis, I. Underwater cave mapping using stereo vision. In Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA); IEEE, 2017; pp. 5709–5715. [Google Scholar] [CrossRef]
18. Rahman, S.; Li, A.Q.; Rekleitis, I. Svin2: an underwater slam system using sonar, visual, inertial, and depth sensor. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); IEEE, 2019; pp. 1861–1868. [Google Scholar] [CrossRef]
19. Wu, X.; Wang, J.; Li, J.; Zhang, X. Retrieval of siltation 3D properties in artificially created water conveyance tunnels using image-based 3D reconstruction. Measurement, 211, 112586. [CrossRef]
20. Fan, J.; Wang, X.; Zhou, C.; Ou, Y.; Jing, F.; Hou, Z. Development, Calibration, and Image Processing of Underwater Structured Light Vision System: A Survey. IEEE Transactions on Instrumentation and Measurement, 72, 1–18. [CrossRef]
21. Wang, W.; Joshi, B.; Burgdorfer, N.; Batsos, K.; Li, A.Q.; Mordohai, P.; Rekleitis, I. eal-Time Dense 3D Mapping of Underwater Environments. Technical report, arXiv, 2023. arXiv:2304.02704 [cs] type: article. [CrossRef]
22. Bräuer-Burchardt, C.; Munkelt, C.; Bleier, M.; Heinze, M.; Gebhart, I.; Kühmstedt, P.; Notni, G. Underwater 3D Scanning System for Cultural Heritage Documentation. Remote Sensing, 15, 1864. [CrossRef]
23. Yu, B.; Tibbetts, R.; Barua, T.; Morales, A.; Rekleitis, I.; Islam, M.J. Weakly Supervised Caveline Detection For AUV Navigation Inside Underwater Caves. [CrossRef]
24. Hu, C.; Zhu, S.; Song, W. Real-time Underwater 3D Reconstruction Based on Monocular Image. In Proceedings of the 2022 IEEE International Conference on Robotics and Biomimetics (ROBIO); 2022; pp. 1239–1244. [Google Scholar] [CrossRef]
25. Marchand, E.; Uchiyama, H.; Spindler, F. Pose estimation for augmented reality: a hands-on survey. IEEE transactions on visualization and computer graphics 2015, 22, 2633–2651. [Google Scholar] [CrossRef] [PubMed]
26. Massone, Q.; Druon, S.; Triboulet, J. An original 3D reconstruction method using a conical light and a camera in underwater caves. In Proceedings of the Proceedings of the 4th International Conference on Control and Computer Vision, 2021; pp. 126–134. [CrossRef]
27. Massone, Q.; Druon, S.; Breux, Y.; Triboulet, J. Contour-based approach for 3D mapping of underwater galleries. In Proceedings of the Global Oceans 2020: Singapore–US Gulf Coast; IEEE, 2020; pp. 1–6. [Google Scholar] [CrossRef]
28. Halır, R.; Flusser, J. Numerically stable direct least squares fitting of ellipses. In Proceedings of the Proc. 6th International Conference in Central Europe on Computer Graphics and Visualization. WSCG. Citeseer; 1998; Vol. 98, pp. 125–132. [Google Scholar]
29. Moré, J.J. The Levenberg-Marquardt algorithm: implementation and theory. In Numerical analysis; Springer, 1978; pp. 105–116. [CrossRef]
30. Zhang, Z. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence 2000, 22, 1330–1334. [Google Scholar] [CrossRef]
31. Fischler, M.A.; Bolles, R.C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM 1981, 24, 381–395. [Google Scholar] [CrossRef]