1. Introduction

2. Proposed Approach

2.1. Background theory

“An optical flow is the technique to calculates the displacement of brightness patterns from one image frame to another”[15] [20]. Several different approaches to optical flow estimation have been proposed, including correlation and energy-based methods. This paper concentrates on differential approach and gives comparison of the two differential techniques namely Lucas-Kanade and Horn-Schunck. These methods are widely used for flow estimation. Horn-Schunck algorithm and Lucas-Kanade algorithm are analysed and compared based on the results obtained.

2.1.1. Differential Techniques:

Differential techniques compute optical flow information from spatial and temporalvariations of the image brightness based on brightness constancy and temporal consistence assumptions[15],[16]. These assumptions result in a motion constraint estimated by the first order Taylor’s expansion of

$$C\left(X,t\right)=C(X-U_t,0)$$

(1)

where $U={(u,v)}^T$. The Taylor expansion of (1) or intensity conservation assumption

$$\frac{dC(X,t)}{dt}=0$$

(2)

implies that

$$\nabla C\left(X,t\right).U+C_t\left(X,t\right)=0$$

(3)

Where $\nabla C\left(X,t\right)={(C_x\left(X,t\right),C_y(X,t\left)\right)}^T$ and $\nabla C.U$ denotes the usual dot product. Equation (3) gives the normal component (U_n) of motion of constant intensity spatial contour as $U_n=s\mathit{n}$, where s is the normal speed and n is the normal direction given by.

$$s\left(X,t\right)=\frac{{-C}_t(X,t)}{‖\nabla C(X,t)‖}$$

$$n\left(X,t\right)=\frac{C_t(X,t)}{‖\nabla C(X,t)‖}$$

(4)

Second order differential methods bounds 2-D velocity using second order derivatives

$$\left[\begin{array}{cc}{C}_{xx}(X,t)& C_{yx}(X,t)\\ C_{xy}(X,t)& C_{yy}(X,t)\end{array}\right]\left[\begin{array}{c}{U}_1\\ U_2\end{array}\right]+\left[\begin{array}{c}{C}_{tx}(X,t)\\ C_{ty}(X,t)\end{array}\right]=\left[\begin{array}{c}0\\ 0\end{array}\right]$$

(5)

The conservation of equation (5) implies $\nabla C\left(X,t\right),d\nabla C(X,t)/dt=0$

The coefficients of these equations are combinations of spatial and temporal derivatives of the image brightness.

• A. Horn– Schunck method:

The Horn-Schunck method uses the assumption that flow is smooth over the whole image[17].Therefore solutions which show more smoothness are preferred. The minimum distortions in the flow are ensured by minimizing a global energy function. The energy function is given for two-dimensional imagery as[15],[18]:

$$\int {\left(C_{x}u+C_{y}v+C_t\right)}^2+{\phi }^2({‖\nabla u‖}^2+{‖\nabla v‖}^2)dx$$

(6)

where $C_x$,$C_y$ and $C_t$ are the derivatives of the pixel intensity values along the x, y and time t dimensions respectively$,U={(u\left(x,y\right),v(x,y\left)\right)}^T$ is the flow vector, and $\phi$ is a regularization constant. To get smoother flow the value $\phi$ is made larger[19]. Iterative equations (6) are used to minimize the global energy function and obtain image velocity in 2-D.

$$u^{J+1}=u^{-J}-\frac{C_x(C_x{u}^{-k}+C_y{v}^{-k}+C_t)}{{\phi }^2+{C_x}^2+{C_y}^2}$$

$$v^{J+1}=v^{-J}-\frac{C_y(C_x{u}^{-k}+C{v}^{-k}+C_t)}{{\phi }^2+{C_x}^2+{C_y}^2}$$

(7)

where J is the number of iteration.

The Horn and Schunck method use first-order differences to estimate intensity derivatives. The Horn-Schunck algorithm applied to Fig 3(a) and Fig 3(b) gives the images in Fig 3(c) and Fig 3(d). The velocities obtained through this method are (-0.16, -0.38).

• B. Lucas–Kanade method:

Although Horn-schunck algorithm is a complete algorithm for optical flow estimation but its iterative nature results in high computational cost[17]. The solution is provided by Lucas-Kanade methods which implements least square method to find the velocity minimizing the constraints errors, squared errors in this case[16],[18]. Major assumption here is constant flow in local neighborhood of the pixel under consideration. Velocity is obtained by solving basic optical flow equations for all the pixels in that neighborhood $\sigma$ satisfying the least square criterion

$${\sum }_{x\in \sigma }{D}^2\left(X\right){[\nabla C\left(X,t\right).U+C_t(X,t\left)\right]}^2$$

(8)

where $D\left(X\right)$ is an n×n diagonal matrix containing the weights. The diagonal matrix ensures that constraints have more influence on pixels at the center of the neighborhood than those at the periphery. The solution to (8) is given by

$$B^T{D}^{2}BU=B^T{D}^{2}b$$

(9)

where for n points $x_i\in \sigma$at a single time t,

$$\begin{gathered} B={[\nabla C(X_1,\dots ..,\nabla C\left(X_n\right)\left)\right]}^T \\ D=diag(D\left(X_1\right),\dots .,D(X_n\left)\right) \\ b=-{\left[C_t\right(X_1),\dots \dots ,C_t(X_n\left)\right]}^T \end{gathered}$$

The solution to (10) is $U={\left[B^T{D}^{2}B\right]}^{-1}{B}^T{D}^{2}b$ , the solution exists if$B^2{D}^{2}B$ is nonsingular matrix.

$$B^2{D}^{2}B=\left[\begin{array}{cc}\sum D^2\left(X\right){C_x}^2\left(X\right)& \sum D^2\left(X\right)C_x\left(X\right)C_y\left(X\right)\\ \sum D^2\left(X\right)C_y\left(X\right)C_x\left(X\right)& \sum D^2\left(X\right){C_y}^2\left(X\right)\end{array}\right]$$

(10)

$B^2{D}^2$ where all sums are taken over points in the neighborhood.

Equations (8) and (9) represent weighted least squares estimates of velocity components from estimates of normal velocities $U_n=sn$. Equation (10) is equivalent to

$$\sum _{x\in \sigma }{D}^2\left(X\right){d^2\left(X\right)[U.n\left(x\right)-s(X\left)\right]}^2$$

Where the coefficients $D^2\left(x\right)$ reflect that how good in the normal velocity estimates are; here $d\left(x\right)=‖\nabla C(X,t)‖$

To solve equation$B^{T}BU=B^T{D}^{2}b$, $B^T{B}^{\prime }s$ eigenvalues should satisfy the constraint ${\lambda }_1\ge {\lambda }_2>0$[17]. ${\lambda }_2$ should not be too small in order to avoid noise. Another constraint is, small value of $\raisebox{1ex}{${\lambda }_1$}\!\left/ \!\raisebox{-1ex}{${\lambda }_2$}\right.$ which if not satisfied leads to the aperture problems. This condition is also works for Corner detection[18].

Fig 4(c) and Fig 4(d) are the resultant images after application of Lucas-Kanade algorithm to Fig 4(a) and Fig 4(b). The velocities obtained through this method are (0.10, -0.63).

Experimentation

Experimentation is performed at around 4 pm in an evening at NUST College of E&ME Rawalpindi using a grid of 1m x 1m having four cells. The aerial platform that is used in experimentation is an X-configuration quad-rotor. It has an empty weight of 989grams and a payload carrying capacity of another 900 grams leading to a total takeoff and flying weight of approximately 2000 grams. The quadrotor uses 4 BLDC motors having thrust of 1280Nm each. The motors are controlled by a PD controller. The BLDCs has a maximum speed of 3200rpm. For imaging, the quad-rotor is equipped with LG G3 mobile phone having a 16 megapixels wide angle camera. Figure 2 shows the imaging platform.

A ball is placed first in the center of the grid that is taken as the starting point. The quad-rotor is made to hover over the grid at a constant altitude of 1.5 m above the ground. The ball is moved to different locations in the grid and the images are acquired. The images are then imported in MATLAB environment and are processed by the developed algorithm to calculate the change in position of the ball in different frames.

Figure 3 shows the experimental grid, whereas; Figure 4 shows the different positions of the ball in the grid. Figure 5 show the outcome of HSV filter and Figure 6 shows an inverted binarized image produce by the algorithm to calculate the change in position of the ball.

Figure 3. Lucas-Kanade.

Figure 3. Lucas-Kanade.

Figure 4. Horn-Schunck.

Figure 4. Horn-Schunck.

Figure 5. Lucas Knade(vector u).

Figure 5. Lucas Knade(vector u).

Figure 6. Lucas Knade(vector v).

Figure 6. Lucas Knade(vector v).

3. Experimental results

4. Conclusion

Figure 9. Difference of u.

Figure 9. Difference of u.

Figure 9. Difference of v.

Figure 9. Difference of v.

Figure 9. Future work also involves the implementation of a real-time tracking algorithm using an on-board processor.

Figure 9. Future work also involves the implementation of a real-time tracking algorithm using an on-board processor.

References

1. Counting Wildlife Manual, WWF-Southern Africa Regional Programme Office, Zimbabwe, January (2004).
2. M. Israel, “AUAV-based Roe Deer Fawn detection system”, Conference on Unmanned Aerial Vehicle in Geomatics, Zurich, Switzerland, (2011).
3. P. Christiansen, K. A. Steen, R. N. Jørgensen, H. Karstoft, “Automated Detection and Recognition of Wildlife Using Thermal Cameras”, Sensors, vol. 14 no. 8, pp 13778-13793, 2014. [CrossRef]
4. M. Leonardo, A. M. Jensen, C.Coopmans, M. McKee, Y. Chen “A Miniature Wildlife Tracking UAV Payload System Using Acoustic Biotelemetry”, International Design Engineering Technical Conferences, (2013).
5. M. Jensen, D. K Geller, YangQuanChen,“Monte Carlo Simulation Analysis of Tagged Fish Radio Tracking Performance by Swarming Unmanned Aerial Vehicles in Fractional Order Potential Fields”, Journal of Intelligent & Robotic Systems, vol. 74, no. 1-2, pp. 287-307, (2013). [CrossRef]
6. W. Selby, P. Corke, D.Rus, “Autonomous Aerial Navigation and Tracking of Marine Animals”,Australasian Conference on Robotics and Automation, (2011).
7. J. Linchant, C. Vermeulen, P. Lejeune, J. Semeki, J. Lisein, “Are unmanned aircraft systems (UASs) the future of wildlife monitoring? A review of accomplishments and challenges”, Laboratory of Tropical & Subtropical Forestry, Passage des Déportés, (2015). [CrossRef]
8. R. C. Gonzales, R. E. Woods, and S. L. Eddins. Digital image processing using MATLAB.Pearson Prentice Hall, (2004).
9. Helmut Mayer, Automatic Object Extraction from Aerial Imagery—A Survey Focusing on Buildings. January 28, (1999). [CrossRef]
10. Andr´e Fischer, Thomas H. Kolbe, Extracting Buildings from Aerial Images Using Hierarchical Aggregation in 2D and 3D. July 7, (1998).
11. Stuart L. Pimm,1,* Sky Alibhai. Emerging Technologies to Conserve Biodiversity.
12. Martin Wikelski1, Roland W. Kays . Going wild: what a global small-animal tracking system could do for experimental biologists. 26 October (2006). [CrossRef]
13. YunfeiFanga, Shengzhi Dub. Motion Based Animal Detection in Aerial Videos.
14. Siyi Li, Dit-Yan Yeung . Visual Object Tracking for Unmanned Aerial Vehicles: A Benchmark and New Motion Models. [CrossRef]
15. J. L. Barron, D. J. Fleet, and S. S. Beauchemin, “Performance of optical flow techniques,” Int. J. Comput. Vis., vol. 12, no. 1, pp. 43–77, 1994. [CrossRef]
16. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods,” Int. J. Comput. Vis., vol. 61, no. 3, pp. 1–21, 2005. [CrossRef]
17. M. G. Pinto, A. P. Moreira, P. G. Costa, and M. V. Correia, “Revisiting Lucas-Kanade and Horn-Schunck,” J. Comput. Eng. Informatics, vol. 1, no. 2, pp. 23–29, 2013. [CrossRef]
18. S. Negahdaripour, “Revised definition of optical flow: integration of radiometrie and geometric cues for dynamic scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 9, pp. 961–979, 1998.
19. L. Fennema and W. B. Thompson, “Velocity determination in scenes containing several moving objects,” Computer Graphics and Image Processing, vol. 9, no. 4. pp. 301–315, 1979. [CrossRef]
20. Kim, Nam-In, et al. "Piezoelectric Sensors Operating at Very High Temperatures and in Extreme Environments Made of Flexible Ultrawide-Bandgap Single-Crystalline AlN Thin Films." Advanced Functional Materials 33.10 (2023): 2212538. [CrossRef]