Energy Aware Camera Location Search Algorithm for Increasing Precision of Observation in Automated Manufacturing

Preprint

Article

Energy Aware Camera Location Search Algorithm for Increasing Precision of Observation in Automated Manufacturing

Altmetrics

Downloads

181

Views

Comments

A peer-reviewed article of this preprint also exists.

Rongfei Li^*,

Farhad Assadian^*

Rongfei Li^*,

Farhad Assadian^*

This version is not peer-reviewed

Submitted:

23 August 2024

Posted:

26 August 2024

You are already at the latest version

Alerts

Abstract

Visual servoing technology has been well developed and applied in many automated manufacturing tasks, especially in tools’ pose alignment. To access a full global view of tools, most applications adopt eye-to-hand configuration or eye-to-hand/eye-in-hand cooperation configura-tion in an automated manufacturing environment. All research papers mainly put efforts into developing control and observation architectures in various scenarios, but none of them has dis-cussed the importance of the camera’s location in eye-to-hand configuration. In a manufacturing environment, the quality of camera estimations may vary significantly from one observation loca-tion to another, as the combined effects of environmental conditions result in different noise levels of a single image shot at different locations. In this paper, we propose an algorithm for the camera’s moving policy so that it explores the camera workspace and searches for the optimal location where the images’ noise level is minimized. Also, this algorithm ensures the camera ends up at a subop-timal (if the optimal one is unreachable) location among the locations already searched, with limited energy available for moving the camera. An automated manufacturing application has been sim-ulated and the results show the success of this algorithm’s improvement of observation precision with limited energy.

Keywords:

Subject: Engineering - Control and Systems Engineering

1. Introduction

In automated industry, automatic alignment plays a critical role in many different applications, such as micromanipulation, autonomous welding, and industrial assembly [1,2,3]. Visual servoing is the technique commonly used in the tasks of alignment as it can guide robotic manipulators to their desired poses (positions and orientations) [4,5,6]. Visual servoing is the method of controlling a robot’s motion using real-time feedback from visual sensors that continuously extract image features [7].

According to the relative positions between cameras and the tool manipulators, visual servoing can be generally divided into two categories: eye-in-hand (EIH) configuration and eye-to-hand (ETH) configuration [8,9]. In EIH, a camera is mounted directly on a robot manipulator, in which case the motion of the robot induces motion of the camera, while in ETH, the camera is fixed in the workspace and observes the motion of the robot from a stationary configuration. Both configurations have their own merits and drawbacks regarding field of view limit and precision that oppose each other. EIH has a partial but precise view of the scene whereas ETH has a global but less precise view of it [10]. For complex tasks in automated manufacturing, one configuration of visual servoing is not adequate as those tasks require the camera to provide global views and to be maneuverable enough to explore the scene [10]. To take advantage of both stationary and robot-mounted sensors, two configurations can be employed in a cooperative scheme, which can be seen in some of the work [11,12,13].

However, we haven’t seen enough discussion on the importance of cameras’ location in the ETH or ETH/EIH cooperative configurations from any work published. To observe a single point in the workspace, there are an infinite number of poses that a fixed camera can be placed in the space. Among all possible poses of the camera that keep the same target within the field of view limit, the image noise at different locations may play a great role in the precision of observations. Many environmental conditions (such as illumination, temperature, etc.) affect the noise level of a single image [14]. In a manufacturing environment, the quality of estimations from a camera may vary significantly from one observation location to another. For instance, the precision of an estimation improves greatly if the camera moves from a location where it is placed within the shade of machines to a location that has better illumination. Also, places near machines may be surrounded by strong electrical signals that also introduce extra noise into the camera sensors.

It is intuitive to consider applying filters to reduce image noise, and therefore, to increase the precision of observations. The Gaussian white noise can be dealt with using spatial filters, e.g., the Gaussian filter, the Mean filter, and the Wiener filter [15]. In addition, wavelet methods [16,17] have the benefit of keeping more useful details but at the expense of computational complexity. However, filters that operate in the wavelet domain still inevitably filter out (or blur) some important high-frequency useful information of the original image, even though they succeed in preserving more edges, compared with spatial filters. Especially at locations where the signal-to-noise ratio (SNR) in an image is low, no matter what filters are applied, it is difficult to safeguard the edges of the noise-free image as well as reduce the noise to a desirable level. Thus, developing an algorithm that searches for locations of the camera where SNR is high is beneficial.

We can approach the denoising problem with multiple noisy images taken from the same perspective. This method is called signal averaging (or image averaging in the application of image processing) [18]. Assume images taken from the same perspective but at different times have the same noise levels. Then averaging multiple images at the same perspective will reduce the unwanted noise as well as retain all original image details. We can also assume the robot’s end effector holds the camera rigidly so that any shaking and drift are negligible when shooting pictures. Moreover, the number of images required for averaging is proportional to the ratio of an image’s original noise level to the reduced level that is desirable for observations. Therefore, the number of images required is a measurement of noise level in a single image. Furthermore, in the denoising process, the precise estimations require that the original image details are retained as much as possible. Based on previous statements, we decided to choose image averaging over all other denoising techniques in this work.

In this paper, we propose an algorithm that searches for the camera’s workspace to find an optimal location (if its orientation is fixed) of the camera so that a single image taken at this location has the smallest noise level among images taken at all locations in the space. With limited energy for moving the camera, this algorithm also ensures the camera ends up at a suboptimal (if the optimal pose is unreachable) location among the locations already searched.

2. Methods

2.1. Camera Location Search Process and Algorithm Flowchart

The proposed algorithm is aimed to search and move the camera to the location where the least number of pictures are needed to reduce the noise level of the averaged image to a degree

σ_{n o i s e_r e d u c e d}

. A flowchart describing the algorithm is shown in Figure 1.

A camera, mounted on a 6-joint robot manipulator, moves freely in 6 degrees of freedom (6 DoFs) in space. Assuming the camera is fixed in orientation towards the face of the object of interest, only the position (3 DoFs) of the camera can vary from the movement of the robot arm. The camera can only move to the locations within the maximum reach of the robot manipulator and the object can only be detectable within the view angle of the camera. Those constraints create a camera operational space (Figure 2a), which is then gridded with a certain resolution of distance. All the intersections of grids come up with a set of nodes (

S_{t o t a l}

), which are all possible candidates of locations that the camera may move and search utilizing the algorithm. Each node is indexed and denoted as

{N o d e}_{i n d e x} .

represents Euclidean distance (green line).

The algorithm iteratively commands the camera to take a picture at one location, generate the next target location, and move to that location until it finds the optimal location that requires the least number of pictures. In one iteration, the camera takes a photo at the current location

P^{C}

and after image processing, it produces a noisy image with an intensity matrix

I_{a} .

A previously developed algorithm [19] estimates the noise level

σ_{a}

across the image. Then, at the current location of the camera, the number of pictures

N^{C}

needed to reduce the noise level to

σ_{n o i s e_r e d u c e d}

is calculated by the equation:

N^{C} = {(\frac{σ_{a}}{σ_{n o i s e_r e d u c e d}})}^{2}

. Based on this information (

N^{C}

P^{C}

), the algorithm then generates the next target location of the camera,

P^{N e x t}

. If the next generated target position

P^{N e x t}

is the same as the current position

P^{C}

P^{C}

is the optimal location and the camera stays and observes the object at this location. Otherwise, the robot joint controller moves the robot joint angles to the targets

\bar{q}

(calculated from forward kinematics [20] (Appendix Equation (B1)–(B5))) so that the camera will move to the next target position,

P^{N e x t}

2.2. Core Alogorithm

The core of this algorithm is to generate the target locations to be explored in the camera’s operational space. The details of this algorithm are shown in the area circled with dashed lines in Figure 1. In this section, we will put efforts into the development of the picture number estimation function, the explorable nodes set, the stochastic modified evaluation function, the next location selection process, and the estimated energy cost function.

2.2.1. Picture Number Estimation Function

In each iteration of the algorithm, the camera moves to a new node with location

P^{C}

, and the algorithm evaluates the number of pictures

N^{C}

needed at this location. The node

({N o d e}_{i n d e x})

is then set to be explored and saved in a set

S_{e x p l o r e d} .

Each node that has been explored in iterations is one-on-one paired with the values: location, and picture numbers of that node. All pairs that are expressed as

{N o d e}_{i n d e x}^{e x p l o r e d}

: (

P_{i n d e x}^{e x p l o r e d}, N_{i n d e x}^{e x p l o r e d}

) are saved in a hash map

M_{e x p l o r e d}

. Then

M_{e x p l o r e d}

is used to develop an estimation function that can estimate the number of pictures needed for the rest unexplored nodes. The estimated picture number calculated for all nodes in space is denoted as

S_{\tilde{N}}

. Expressions of any element in

S_{\tilde{N}}

are:

For any node index

x

in the Total Node Set:

S_{t o t a l} = {{N o d e}_{x} | x \in (1,2, \dots m)}

, where

m

is the total number of nodes.

{\tilde{N}}_{x} = \{\begin{matrix} \sum_{i \in S_{i n d e x - e x p}} \frac{(\bar{E D} (P_{i}^{e x p}, P_{x})^(- K_{e s t}))}{\sum_{j \in S_{i n d e x - e x p}} ((\bar{E D} (P_{j}^{e x p}, P_{x})^(- K_{e s t}))} N_{i}^{e x p l o r e d} \\ N_{x}^{e x p l o r e d}, i f {N o d e}_{x} \in S_{e x p l o r e d} \end{matrix}, i f {N o d e}_{x} \notin S_{e x p l o r e d}

(1)

Then : S_{\tilde{N}} = {{\tilde{N}}_{x} | x \in (1,2, \dots m)}

(2)

where

K_{e s t}

is a positive constant parameter

S_{i n d e x - e x p} = {i n d e x | {N o d e}_{i n d e x} \in S_{e x p l o r e d}}

(3)

\bar{E D} (P_{a}, P_{b}) = \sqrt{{(P_{a x} - P_{b x})}^{2} + {(P_{a y} - P_{b y})}^{2} + {(P_{a z} - P_{b z})}^{2}} with P_{a}, P_{b} are 3 D locations of {N o d e}_{a} a n d {N o d e}_{b} : P_{a} = (P_{a x}, P_{a y}, P_{a z}) a n d P_{b} = (P_{b x}, P_{b y}, P_{b z})

(4)

Equation (1) is a weighted function concerning each explored node. If one explored node

{N o d e}_{i}^{e x p l o r e d}

is closer to the node

{N o d e}_{x}^{e x p l o r e d}

, its picture number

N_{i}^{e x p l o r e d}

has a larger weight (effect) on the estimation of

{\tilde{N}}_{x}

. Thus, a minus sign of

K_{e s t}

is utilized to indicate the negative correlation between the Euclidean distance and weight.

Equation (1) shows that at least some initial nodes are required to be explored before their values can be used to estimate the picture numbers in other nodes. For the estimation function to work properly by covering all nodes in the camera operational space, four initial nodes are selected in the following steps:

Find the first two nodes on the boundary of operational space so that the Euclidean distance between these two nodes is the largest.
Select other two nodes whose connection line is perpendicular to the previous connection line and the Euclidean distance between these two nodes is also the largest among all possible node pairs.
Move the camera to those four nodes in space with proper orientation. Take a single image at each location and estimate the number of pictures required at those locations. And save all four nodes in $S_{e x p l o r e d}$ and their values in $M_{e x p l o r e d} .$

Figure 2b shows the selection of four initial nodes in a camera operational space and shows an example of the use of the estimation function to estimate the picture number in one node.

2.2.2. Explorable Node Selection

A stage in the algorithm, Explorable Node Selection, generates a set of nodes

S_{e x p l o r a b l e},

which contains all the camera’s next explorable locations from the current location

P^{C}

with

a v a i l a b l e

current energy for moving

E_{b o u n d} .

The explorable set is found in the following steps (Figure 3):

With current energy bound $E_{b o u n d}$ , find all nodes in the space that can be reached from the current node $P^{c} .$ In other words, all feasible nodes are in the set:

$S_{f e a s i b l e} = {{N o d e}_{i n d e x} | E (P^{c}, P_{i n d e x}) \leq E_{b o u n d}}$

(5)

where $P$ is the position of a node and $E (P_{a}, P_{b})$ is the estimated energy cost from $P_{a} t o P_{b}$ .

When the current energy bound

E_{b o u n d}

is greater than a predefined energy threshold

E_{T}

(

E_{b o u n d} > E_{T}

), then the algorithm is safe to explore all nodes in the feasible range

S_{f e a s i b l e}

in the next iteration loop. In other words:

S_{e x p l o r a b l e} = S_{f e a s i b l e}, i f E_{b o u n d} > E_{T}

(6)

Note:

E (P_{a}, P_{b})

is generated from a dynamic model of a robot arm control system. The details of this function are developed in the following section of this paper.

2.: When $E_{b o u n d} \leq E_{T}$ , step 2 and step 3 are applied in finding the explorable set. In the feasible set, look at all explored nodes and find the one that has the smallest number of pictures. In other words:

$S_{f e a s & e x p} = {{N o d e}_{i n d e x} | {N o d e}_{i n d e x} \in S_{f e a s i b l e} \cap S_{e x p l o r e d}}$

(7)

$e x p_m i n = {i n d e x | N_{e x p_m i n} = \min (N_{i n d e x}| {N o d e}_{i n d e x} {\in S}_{f e a s & e x p})}$

(8)

where $S_{e x p l o r e d}$ is the set of all explored nodes.

$S_{f e a s & e x p}$ is the intersection set between $S_{f e a s i b l e}$ and $S_{e x p l o r e d}$ .

3.: The estimation function may not give accurate results for some unexplored nodes. Therefore, it is possible that the algorithm may make the camera end up in a node that has a large number of pictures in some iteration loops. Because of that, our algorithm needs to make sure at worst it has enough energy to go back to the best (minimum number of pictures) node that has been explored when the available energy is low ( $E_{b o u n d} \leq E_{T}$ ). This further reduces the feasible set $S_{f e a s i b l e} .$ The explorable set $S_{e x p l o r a b l e}$ can be written as:

$S_{e x p l o r a b l e} = \{{N o d e}_{i n d e x}| E_{1} + E_{2} \leq E_{b o u n d}\}, i f E_{b o u n d} \leq E_{T}$

(9)

$E_{1} = E (P^{c}, P_{i n d e x}), E_{2} = E (P_{i n d e x}, P_{e x p_m i n})$

(10)

Figure 3 illustrates the selection of explorable nodes set.

2.2.3. Stochastic Modified Evaluation

For all nodes in

S_{e x p l o r a b l e},

we set up a hash map

M_{e x p l o r a b l e}

. For each node, its location and number of required pictures for that node are given as:

{N o d e}_{i n d e x}^{e x p l o r a b l e}

: (

P_{i n d e x}^{e x p l o r a b l e}, {\tilde{N}}_{i n d e x}^{e x p l o r a b l e}

). The number of pictures required for nodes in

S_{e x p l o r a b l e}

are calculated from the estimation function in Equation (1).

The algorithm tends to select nodes located around the explored node that has the minimum number of pictures because, from Equation (1), those nodes tend to have the smallest estimated number of pictures. However, this process does not guarantee reaching the global minimum, therefore, minimizing the number of pictures. To sufficiently explore unknown nodes as well as exploit information from already explored nodes, a stochastic process is introduced to modify the picture number estimation function given in Equation (1).

Assume when the locations of a camera in space deviate with a smaller amount from each other, the difference of real picture number values at those locations is also smaller. Thus, among all unexplored nodes, the estimated picture number of an unexplored node, which is closer to the explored node, is more deterministic. To emphasize this feature, we make the estimated picture number follow a normal distribution with a mean

μ_{\tilde{N}}

being equal to the value from Equation (1) and a standard deviation

σ_{\tilde{N}}

. As an unexplored node is further away from explored nodes, the larger value of

σ_{\tilde{N}}

should be assumed, which means more indeterministic estimation (illustrated in Figure 4). The following equations summarize the above discussion:

For any nodes

{N o d e}_{i n d e x = x}^{e x p l o r a b l e}

S_{e x p l o r a b l e},

its picture number estimation should follow the following normal distributions:

P (Z_{x}) = N ({μ_{\tilde{N}}}_{x}, {σ_{\tilde{N}}}_{x})

(11)

{μ_{\tilde{N}}}_{x} = {\tilde{N}}_{x}^{e x p l o r a b l e}

(12)

{σ_{\tilde{N}}}_{x} = K_{s d} * {\bar{E D}}_{m i n}

(13)

where

P (Z_{x})

is the probability of a random variable

Z

{N o d e}_{x}^{e x p l o r a b l e}

{\tilde{N}}_{x}^{e x p l o r a b l e}

is the estimated value from Equation (1) of

{N o d e}_{x}^{e x p l o r a b l e} .

K_{s d}

is a constant parameter, and

{\bar{E D}}_{m i n}

is the smallest Euclidean distance among distances between that node and explored nodes.

As discussed above, the algorithm ensures that the camera at any iteration, always has enough energy to move back to the previous best node

N_{e x p_m i n}

. Therefore, a new distribution can be generated, which sets values bigger than the minimum (

N_{e x p_m i n}

) in the original distribution to be the minimum value. Then Equations (11) to (13) are modified with a new random variable

Z_{x}^{n e w}

and its expectations as follows:

Z_{x}^{n e w} = \{\begin{matrix} Z_{x}, i f Z_{x} < N_{e x p_m i n} \\ N_{e x p_m i n}, i f Z_{x} \geq N_{e x p_m i n} \end{matrix}

(14)

P (Z_{x}^{n e w}) = P (Z_{x}) = N ({μ_{\tilde{N}}}_{x}, {σ_{\tilde{N}}}_{x})

(15)

{\hat{N}}_{x}^{e x p l o r a b l e} = \sum [Z_{x}^{n e w} * P (Z_{x}^{n e w})]

(16)

where

{\hat{N}}_{x}^{e x p l o r a b l e}

is the stochastic modified estimated value for the node

{N o d e}_{x}^{e x p l o r a b l e}

Then a new hash map

{\hat{M}}_{e x p l o r a b l e}

is set up with pairs as

{N o d e}_{i n d e x}^{e x p l o r a b l e}

: (

P_{i n d e x}^{e x p l o r a b l e}, {\hat{N}}_{x}^{e x p l o r a b l e}

). The next target node to be explored is the one that has the smallest modified estimated value in the map:

i n d e x_{N e x t} = \{i n d e x| {\hat{N}}_{i n d e x_{N e x t}}^{e x p l o r a b l e} = \min ({\hat{N}}_{i n d e x}^{e x p l o r a b l e})}

(17)

P^{N e x t} = P_{i n d e x_{N e x t}}^{e x p l o r a b l e}

(18)

σ_{2}

) to differentiate possibility distributions. This approach distinguishes nodes that have the same estimations from equation (1) to have different likelihood to be explored in the next iteration.

2.2.4. Estimated Energy Cost Function

As discussed above, an estimated energy cost function

E (P_{a}, P_{b})

calculates the estimated energy cost from

P_{a} t o P_{b}

. This function is used to find feasible node set

S_{f e a s i b l e}

and explorable node set

S_{e x p l o r a b l e}

. In this section, the equations for the cost function are derived from the robot positioning-controlled system’s response. Also, trade-offs between energy cost and settling time of moving are discussed.

The energy of moving the camera between two locations from

P_{i n i t i a l}

P_{f i n a l}

, results from the energy cost of DC motors in 6 DOFs robot arm’s each joint rotating from joint angles

q_{i n i t i a l}

q_{f i n a l}

. Therefore,

E (P_{i n i t i a l}, P_{f i n a l})

can be described as the sum of the time integration of rotational power in each joint. That is:

E (P_{i n i t i a l}, P_{f i n a l}) = \sum_{i = 1}^{6} \int_{t_{0}^{i}}^{t_{f}^{i}} V^{i} \cdot I^{i} \cdot d t

(19)

where

V^{i}

is the voltage and

I^{i}

is the current in the DC motor’s circuit of the

i^{t h}

joint.

t_{0}^{i}

and

t_{f}^{i}

are initial time and final time of the

i^{t h}

joint moving from its initial angle

q_{i n i t i a l}^{i}

to its final angle

q_{f i n a l}^{i}

Without detailed derivation, the dynamic model of a 6dofs revolutionary robotic manipulator and DC motors can be expressed as [20]:

\frac{1}{r_{k}} {J_{m}}_{k} {\ddot{q}}_{k} + \sum_{j = 1}^{6} d_{j, k} {\ddot{q}}_{j} + \sum_{i, j}^{6} c_{i, j, k} (q) {\dot{q}}_{i} {\dot{q}}_{j} + {\frac{1}{r_{k}} B {\dot{q}}_{k} + Φ}_{k} (q) = \frac{K_{m}}{r_{k} R} V^{i}

(20)

J_{m} = J_{a} + J_{g}, {B = B}_{m} + \frac{K_{b} K_{m}}{R}

(21)

c_{i, j, k} = \frac{1}{2} (\frac{δ d_{k, j}}{δ q_{i}} + \frac{δ d_{k, i}}{δ q_{j}} - \frac{δ d_{i, j}}{δ q_{k}}), Φ_{k} = \frac{δ V}{δ q_{k}}

(22)

i, j, k \in (1, 2, 3, 4, 5, 6)

(23)

where Equations (20) and (21) express the

k^{t h}

joint dynamics equation.

q_{i} / q_{j} / q_{k}

is the joint revolute variable.

J_{a} a n d J_{g}

represent the moment of inertia of the motor, and the gear of the model.

r_{k}

is gear ratio at

k^{t h}

joint, and

B_{m}

is the damping effect of the gear.

d_{i, j}

represents the entry of inertial matrix of the robot manipulator at

i^{t h}

row and

j^{t h}

column.

c_{i, j, k}

is Christoffel symbols and for a fixed

k

, with

c_{i, j, k} = c_{j, i, k}

. And

Φ_{k}

is the derivative of potential energy V with respective to

k^{t h}

joint variance.

K_{m}

is the torque constant in

N - m / a m p,

K_{b}

is the back emf constant, and R is Armature Resistance.

Take

u^{i}

as the actuator input to the dynamic system (20) measured at

i^{t h}

joint from a designed controller. Therefore,

u^{i}

equals to the right-side of Equation (20). That is:

V^{i} = \frac{r R}{K_{m}} u^{i}

(24)

In addition, without derivation, the expression of current in the Laplace domain [20] is:

(L s + R) I^{i} (s) = V^{i} (s) - \frac{K_{b}}{r} s q^{i} (s)

(25)

Then : I^{i} (s) = \frac{1}{(L s + R)} V^{i} (s) - \frac{K_{b}}{r} \frac{s}{(L s + R)} q^{i} (s)

(26)

Take the inverse Laplace : I^{i} (t) = \frac{1}{L} e^{- \frac{R}{L} t} * V^{i} (t) - \frac{K_{b}}{r} (\frac{1}{L} - \frac{R}{L^{2}} e^{- \frac{R}{L} t}) * q^{i} (t)

(27)

where L is armature inductance

, K_{b}

is back emf constant, and

*

is the convolutional multiplication. Therefore, the instant current in the time domain

I^{i} (t)

is a function of the instant voltage

V^{i} (t)

and the instant angle

q (t) .

Various controller designs, such as PID controller [21], and Youla controller [22], of six DoFs revolute robotic manipulators have been well developed in many papers. In this paper, we use a previous Youla controller design [22] with feedback linearization (Figure 5) as the position controller of the robot manipulator that holds the camera.

From Figure 5, the actuator input

u^{i}

can be derived from a virtual input

v^{i}

so that:

u^{i} = M (q^{i} (t)) v^{i} + H (q^{i} (t), \dot{q^{i}} (t))

(28)

where M, and H are nonlinear functions of

q^{i} (t)

, and

\dot{q^{i}} (t)

, the first derivative of

q^{i} (t) .

Without showing the controller development in this paper, the 1 DoF controller transfer function

G_{c} (s)

and 1 DoF nominal plant

G_{p} (s)

with

v^{i}

as the virtual actuator input in the feedback linearization are expressed as follows:

G_{c} (s) = \frac{3 {τ_{i n}}^{2} s + 1}{{τ_{i n}}^{3} s + 3 {τ_{i n}}^{2}}

(29)

G_{p} (s) = \frac{1}{s^{2}}

(30)

where

τ_{i n}

is a constant parameter in the controller design.

Then the following transfer functions can be calculated as:

\frac{v^{i} (s)}{{q^{i}}_{f i n a l} (s)} = \frac{G_{c}}{1 + G_{c} G_{p}}

(31)

\frac{q^{i} (s)}{{q^{i}}_{f i n a l} (s)} = \frac{G_{c} G_{p}}{1 + G_{c} G_{p}}

(32)

\frac{\dot{q^{i}} (s)}{{q^{i}}_{f i n a l} (s)} = \frac{{s G}_{c} G_{p}}{1 + G_{c} G_{p}}

(33)

By taking inverse Laplace transform, and

{q^{i}}_{f i n a l}

as a step input, the following equations in the time domain are:

v^{i} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{1}{{τ_{i n}}^{4}} t^{2} e^{- \frac{t}{τ_{i n}}} + \frac{- 5}{{τ_{i n}}^{3}} t e^{- \frac{t}{τ_{i n}}} + \frac{3}{{τ_{i n}}^{2}} e^{- \frac{t}{τ_{i n}}})

(34)

q^{i} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{1}{{τ_{i n}}^{2}} t^{2} e^{- \frac{t}{τ_{i n}}} - \frac{1}{τ_{i n}} t e^{- \frac{t}{τ_{i n}}} - e^{- \frac{t}{τ_{i n}}} + 1) + {q^{i}}_{i n t i a l}

(35)

\dot{q^{i}} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{3}{{τ_{i n}}^{2}} t e^{- \frac{t}{τ_{i n}}} - \frac{1}{{τ_{i n}}^{3}} t^{2} e^{- \frac{t}{τ_{i n}}})

(36)

With expressions of

v^{i} (t), q^{i} (t),

and

\dot{q^{i}} (t)

, Equations (24) and (28) shows that:

V^{i} (t) = F ({q^{i}}_{i n t i a l}, {q^{i}}_{f i n a l}, t)

(37)

And combine Equations (27) and (37):

I^{i} (t) = G ({q^{i}}_{i n t i a l}, {q^{i}}_{f i n a l}, t)

(38)

where F, and G are nonlinear functions of

{q^{i}}_{i n i t i a l}, {q^{i}}_{f i n a l}, a n d t

It has been shown so far that the estimated energy cost

E (P_{i n i t i a l}, P_{f i n a l})

from location

P_{i n i t i a l}

P_{f i n a l}

is a function of

q_{i n i t i a l}

and

q_{f i n a l}

q_{i n i t i a l}

and

q_{f i n a l}

are derived from the inverse kinematics process [20] (Shown in Appendix Equations (B6)–(B13)), that is:

q_{i n i t i a l} = i n v e r k i n e m a t i c s (P_{i n i t i a l})

(39)

q_{f i n a l} = i n v e r k i n e m a t i c s (P_{f i n a l})

(40)

And q_{i n i t i a l} = [{q^{i}}_{i n i t i a l}| i \in (1,2, 3,4, 5,6)], q_{f i n a l} = [{q^{i}}_{f i n a l}| i \in (1,2, 3,4, 5,6)]

(41)

Development of Equations (34) to (36) assumes that the target angle of each joint

{q^{i}}_{f i n a l}

is a step input. A more realistic assumption is to set

{q^{i}}_{f i n a l}

as a delayed input.

{q^{i}}_{f i n a l} (t) = (1 - e^{- \frac{t}{τ_{d e l a y}}}) {q^{i}}_{f i n a l}

(42)

where

τ_{d e l a y}

is a time constant that measures the time delay of the target in the real positioning control.

τ_{d e l a y} \geq 0

, and when

τ_{d e l a y} = 0

, it indicates no delay exists in the input.

The Laplace form of (42) is:

{q^{i}}_{f i n a l} (s) = (\frac{1}{s} - \frac{1}{s + \frac{1}{τ_{d e l a y}}}) {q^{i}}_{f i n a l}

(43)

With new expression of

{q^{i}}_{f i n a l} (s)

, Equation (34) to (36) can be developed as:

v^{i} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{A_{v}}{{τ_{i n}}^{3}} t^{2} e^{- \frac{t}{τ_{i n}}} + B_{v} \frac{B_{v}}{{τ_{i n}}^{2}} t e^{- \frac{t}{τ_{i n}}} + \frac{C_{v}}{τ_{i n}} e^{- \frac{t}{τ_{i n}}} + \frac{D_{v}}{τ_{d e l a y}} e^{- \frac{t}{τ_{d e l a y}}})

(44)

A_{v} = \frac{1}{τ_{i n}} - \frac{- τ_{d e l a y}}{τ_{i n} (τ_{i n} - τ_{d e l a y})} B_{v} = \frac{- 5}{τ_{i n}} - \frac{7 τ_{i n} τ_{d e l a y} - 5 {τ_{d e l a y}}^{2}}{τ_{i n} {(τ_{i n} - τ_{d e l a y})}^{2}} C_{v} = \frac{3}{τ_{i n}} - \frac{τ_{d e l a y} (- 8 {τ_{i n}}^{2} + 9 τ_{i n} τ_{d e l a y} - 3 {τ_{d e l a y}}^{2})}{{τ_{i n} (τ_{i n} - τ_{d e l a y})}^{3}} D_{v} = - \frac{3 τ_{i n} τ_{d e l a y} - {τ_{d e l a y}}^{2}}{{(τ_{i n} - τ_{d e l a y})}^{3}}

q^{i} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{A_{Q}}{{τ_{i n}}^{3}} t^{2} e^{- \frac{t}{τ_{i n}}} + \frac{B_{Q}}{{τ_{i n}}^{2}} t e^{- \frac{t}{τ_{i n}}} + \frac{C_{Q}}{τ_{i n}} e^{- \frac{t}{τ_{i n}}} + \frac{D_{Q}}{τ_{d e l a y}} e^{- \frac{t}{τ_{d e l a y}}} + 1) + {q^{i}}_{i n t i a l}

(45)

A_{Q} = τ_{i n} + \frac{τ_{i n} τ_{d e l a y}}{(τ_{i n} - τ_{d e l a y})} B_{Q} = - τ_{i n} - \frac{3 {τ_{i n}}^{2} τ_{d e l a y} - τ_{i n} {τ_{d e l a y}}^{2}}{{(τ_{i n} - τ_{d e l a y})}^{2}} C_{Q} = - τ_{i n} - \frac{- 3 {τ_{i n}}^{2} {τ_{d e l a y}}^{2} + {τ_{i n} τ}_{d e l a y}^{3}}{{(τ_{i n} - τ_{d e l a y})}^{3}} D_{Q} = - \frac{3 τ_{i n} {τ_{d e l a y}}^{3} - {τ_{d e l a y}}^{4}}{{(τ_{i n} - τ_{d e l a y})}^{3}}

\dot{q^{i}} (t) = {(q^{i}}_{f i n a l} - {q^{i}}_{i n t i a l}) (\frac{A_{\dot{Q}}}{{τ_{i n}}^{4}} t^{2} e^{- \frac{t}{τ_{i n}}} + \frac{B_{\dot{Q}}}{{τ_{i n}}^{3}} t e^{- \frac{t}{τ_{i n}}} + \frac{C_{\dot{Q}}}{{τ_{i n}}^{2}} e^{- \frac{t}{τ_{i n}}} + \frac{D_{\dot{Q}}}{{τ_{d e l a y}}^{2}} e^{- \frac{t}{τ_{d e l a y}}})

(46)

A_{\dot{Q}} = - A_{Q} B_{\dot{Q}} = 2 A_{Q} - B_{Q} C_{\dot{Q}} = B_{Q} - C_{Q} D_{\dot{Q}} = - D_{Q}

A simulation scenario is set up to calculate how estimated energy cost changes with varying

τ_{d e l a y} .

Set

P_{i n i t i a l} = [- 0.30, 0.05, 1.20]

, and

P_{f i n a l} = [- 0.45, 0.45, 1.20]

and use the equations above. Figure 6 shows the response of one angle

q^{1} (t)

with varying

τ_{d e l a y}

and Table 1 presents the estimated energy cost:

E (P_{i n i t i a l}, P_{f i n a l})

and settling time:

t_{s}

with varying

τ_{d e l a y} .

Table 1 clearly shows a tradeoff between the estimated energy cost and settling time; reducing the energy cost of moving the camera inevitably increases the response time. This finding matches the results in Figure 6.

If the delay of input is given or can be measured, the estimated energy cost can be calculated through the process in this section. However, if the value is unknown, the delay constant must be chosen and decided based on the following criteria:

Select small $τ_{d e l a y}$ for a conservative algorithm that searches a small area but ensures it ends up with the minimum that has been explored.
Select large $τ_{d e l a y}$ for an aggressive algorithm that searches a large area but risks not ending up at the minimum that has been explored.

In this section, the estimated energy cost function is well developed. The parameter values of motors and gears used in the simulations are summarized in Appendix Table A4.

3. Results and Discussion

The developed algorithm has been simulated on an application in automated manufacturing (Figure 7). This is a multi-robot system composed of a visual system and a tool manipulation system. In the visual system, a camera is mounted on an elbow robot arm while a tool is held by the end-effector of the robot manipulator arm. The goal of the visual system is to provide a precise estimation of the tool pose so that the tool manipulator can control the pose with guidance from the visual system. The first step of the manufacturing process is to move the camera in the space and search for the best location so that the number of images required for averaging is minimized to reduce the image noises within an acceptable limit. The model of both robot manipulators is ABB IRB 4600 [23] and the model of the stereo camera we choose in this project is Zed 2 [24].

In this section, we first define the camera’s operational space in this application and then provide the simulation results of a made-up scenario.

3.1. Camera Operational Space

In the above discussion, a camera operational space is gridded with nodes and each node is a potential candidate of the camera’s optimal position. The camera operational space is defined by three geometric constraints below:

1. The camera can only be allowed to locations where fiducial markers that are attached to the tool are recognizable on the image frame. Therefore, the fiducial markers are in the angle of view of the camera, and the distance between the markers and the camera center is within a threshold.

2. The camera can only be allowed to locations within the reach of the robot arm.

3. The camera can only be allowed in locations where the visual system and the tool manipulation system do not physically interfere with each other.

From the specifications of the stereo camera and the robot arm (Table A1 and Table A3), the geometry and dimensions of the camera operational space are analyzed in the following part of the subsections.

3.1.1. Reachable and Dexterous Workspace of Two-Hybrid Systems

In the spherical wrist model of the robot arm, three rotational axes represent the pitch, roll, and yaw of the end-effector independently. As shown in Figure 8, those three axes intersect at point

H

. Point

P

is the location of the end-effector and it’s the place where the camera or tool is mounted. For the elbow manipulator mounted with the stereo camera, the camera is placed so that its optical center line coincides with axis C. The location of camera optical center C is determined and a good estimation of it can be found in this paper [25]. For now, assume the optical center is in the middle of

\bar{P J}

(set

\bar{P C} = 17 m m

). For the elbow manipulator mounted with the tool, the screwdriver is attached at the point

P

. Point J indicates the far end of the object (tool or camera) attached to the end effector. Point H is kept stationary no matter which axis rotates. Therefore, the task of positioning and rotation is decoupled in the spherical wrist robot model.

The idea of reachable and dexterous workspaces of an ideal elbow manipulator has been introduced in the paper [26]. An ideal elbow manipulator is a manipulator whose angles of rotation are free to move in the whole operational range [0, 2

π] .

However, a realistic elbow manipulator is limited to moving its joints within certain ranges of angle.

The workspace of an ideal elbow manipulator (Figure 9) is a sphere centered at the joint 1 of the manipulator, denoted as point

o

. The reachable workspace concerning the center o of a manipulator is the aggregate of all possible locations of the point

J

attached with the end-effector and is denoted as

W_{o} (J)

. The dexterous workspace to the center o of a manipulator is the aggregate of all possible locations that point

J

can reach all possible orientations of the end-effector and is denoted as

W_{o}^{d} (J)

. The reachable workspace of

H

is denoted as

W_{o} (H)

It is clearly shown in the figure:

W_{o}^{d} (J) \subseteq W_{o} (H) \subseteq W_{o} (J) .

(47)

For an ideal elbow manipulator, the radiuses of workspaces are expressed as below:

Radius of W_{o} (H) = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}}

(48)

Radius of W_{o} (J) = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} + L_{t} + \bar{P J}

(49)

{Radius of W}_{o}^{d} (J) = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} - L_{t} - \bar{P J}

(50)

where

L_{2},

L_{3}, a n d L_{4}

are links’ length of the manipulator.

L_{t}

is the length of the end-effector and

\bar{P J}

is the length of the object that mounts on the end-effector.

Because the camera and the tool must be able to rotate in all 3 DoFs when they are at the target position. So, the dexterous workspace is used as the working space of the visual system and the tool manipulation system. Dexterous workspace of the optical center is used for the visual system and Dexterous workspace of the far endpoint of the tool is used for the tool manipulator system.

Radius of W_{o}^{d} (V i s u a l) = r_{V} = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} - L_{t} - {\bar{P C}}_{c a m e r a} = 1.716 m (From Specification Appendix Table A 1)

(51)

Radius of W_{o}^{d} (T o o l) = r_{T} = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} - L_{t} - {\bar{P J}}_{t o o l} = 1.606 m (From Specification Appendix Table A 1)

(52)

Figure 9 shows the workspace setup for the hybrid system. The dexterous space of robot arm with camera is a sphere with radius of

r_{V}

and sphere center is

O_{V}

and the dextrous space of the robot arm with tools is a sphere the radius is

r_{T}

and center of this sphere is

O_{T} .

Two systems intersect with the ground with an angle

θ_{1}

and

θ_{2}

θ_{1} = \arcsin (\frac{L_{1}}{r_{V}}) = 16.77 °

(53)

θ_{2} = \arcsin (\frac{L_{1}}{r_{T}}) = 17.95 °

(54)

To avoid interference of the visual and the tool manipulation systems (as the third geometric constraint of the operational space), the reachable workspaces (the maximum reach) of two systems should have no overlap. The reachable workspaces of two systems are calculated from Equation (49) is:

Radius of W_{o} (V i s u a l) = r_{V r} = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} + L_{t} + {\bar{P J}}_{c a m e r a} = 2.044 m (From Specification Appendix Table A 1)

(55)

Radius of W_{o} (T o o l) = r_{T r} = L_{2} + \sqrt{{L_{3}}^{2} + {L_{4}}^{2}} + L_{t} + {\bar{P J}}_{t o o l} = 2.138 m (From Specification Appendix Table A 1)

(56)

Let

L_{V T}

is the distance between

O_{V}

and

O_{T}

. The following relationship must be satisfied:

L_{V T} \geq r_{V r} + r_{T r} = 4.182 m

(57)

The visual system is able to detect the tool when it gets close to the target pose (Just above the bolt). Also reference points are put near the tool’s target pose. Assume a marker is put at where the bolt is, and this marker represents as an approximated location of all interest points and reference points. Therefore, the marker should be within the dexterous space of the tool manipulation system. Set up a coordinate system with

{\bar{O}}_{V}

(the projection of

O_{V}

on the ground with

a_{1}

offset) as the origin. The projection of

O_{T}

on the ground with

a_{1}

offset is

{\bar{O}}_{T} .

Assume the marker is placed on the line

\bar{{\bar{O}}_{V} {\bar{O}}_{T}}

. The distance m from marker M to

{\bar{O}}_{V}

should be:

L_{M} \geq {a_{1} + L}_{V T} - r_{T} \cos (θ_{2}) \geq 2.829 m

(58)

3.1.2. Detectable Space for the Stereo Camera

From Appendix Table A3, the angle of view in width and height for both lens in stereo camera is

α

86.05 °

, and

β = 55.35 °

. The detectable space for each lens can be modeled as the inner area of a cone with angle

α

and

β .

And the overlapped space is the detectable space for the stereo camera. The model is shown below (Figure 10). Point C is the center of the baseline whose length is denoted as

b

or center of the camera system. The overlap area is also a cone with angles of vertex

α

and

β

with the offset

d

from baseline. Point Q is the vertex of the cone.

d = \frac{b / 2}{t a n (\frac{α}{2})} = 0.064 m

(59)

There is an upper bound of the distance between the object to the camera center; if the object is too far away from the camera center, the dimensions of projected images are too small to be measured. Suppose to have a clear image of the fiducial markers (circle shapes), it is required that the diameter of the projected image should takes at least 5-pixel numbers in the image frame:

N_{H} \geq 5 resolution

(60)

where

N_{H}

is the pixel number in the image frame.

Select the Zed camera’s mode so that its resolution is 1920*1082 and the image sensor size is 5.23mm X 2.94mm. Then numbers resolution in unit length is:

n = 367 resolution / mm

(61)

Then the range of markers’ diameter on the image frame is:

d_{H} = \frac{N_{H}}{n} \geq 0.0136 m m

(62)

Also assume in the inertial frame, the diameter of attached fiducial marks is:

D_{H} = 12 m m

(63)

Then from pinhole model of the camera, the range of distance between markers and camera center Z is:

Z = \frac{f_{u} D_{H}}{d_{H}} \leq 2.47 m = Z_{m a x}

(64)

where

Z_{m a x}

is the maximum depth of camera to detect the markers. This parameter defines the height of the cone in Figure 10. The detectable space (abiding the second geometric constraint) forms an elliptic cone with different angles of vertex.

3.1.3. Camera Operational Space Development

In Figure 11, set the camera lens so that it is always parallel to the face of the marker and the baseline is parallel to

{\bar{X}}_{V}

axis. In this case, lens is kept facing downward when detecting the marker on the ground. As discussed in the above section, the detectable space of the camera is modeled as a cone. This cone intersects with the horizontal plane and forms an oval. To have the camera detect the marker, the marker must be contained inside the oval.

An inertia coordinate system

{\bar{O}}_{V} {\bar{X}}_{V} {\bar{Y}}_{V} {\bar{Z}}_{V}

is set up with its origin seated at the bottom center of the visual system. Make point

C

as the location of the camera center and its coordinates in the inertial frame are (

X_{C}^{V}

Y_{C}^{V}

Z_{C}^{V}

Q

is the vertex of the detectable elliptic cone, and

Q ’

is the projection of

Q

on the horizontal plane.

M

is the marker.

The elliptic cone of the camera’s detectable space always intersects with the horizontal plane. Consider an extreme case when point

C

is located at the apex of the visual system workspace as shown in Figure 11. Then the

Z_{C}^{V}

coordinate of

C

at the inertial frame is:

Z_{C}^{V} {m a x} = L_{1} + r_{V} = 2.211 m \leq Z_{m a x}

(65)

where

Z_{m a x}

defined above is the maximum depth of camera to detect the markers. Therefore, the cone intersects with the horizontal plane when the camera center

C

is placed at any location of the workspace’s contour.

From expression of Cartesian coordinates in the inertia frame:

a_{Q'} = (Z_{C}^{V} - d) t a n (\frac{α}{2}), b_{Q'} = (Z_{C}^{V} - d) t a n (\frac{β}{2})

(66)

where

d

is the same offset in Equation (59).

a_{Q'}

and

b_{Q'}

are major radius and minor radius of the oval. Also coordinates of points

Q ’

and

M

are

(X_{C}^{V}, Y_{C}^{V}, 0)

and

(L_{M}, 0, 0)

Q ’

is the center of the oval. To ensure the marker

M

is inside the projected oval, it is required that:

\frac{{(X_{C}^{V} - L_{M})}^{2}}{{a_{Q'}}^{2}} + \frac{{(Y_{C}^{V})}^{2}}{{b_{Q'}}^{2}} \leq 1

(67)

Plug Equation (66) into Equation (67):

\frac{{(X_{C}^{V} - L_{M})}^{2}}{{t a n (\frac{α}{2})}^{2}} + \frac{{(Y_{C}^{V})}^{2}}{{t a n (\frac{β}{2})}^{2}} \leq {(Z_{C}^{V} - d)}^{2}

(68)

Inequality (68) is exactly a mathematic expression of all points

C

(

X_{C}^{V}

Y_{C}^{V}

Z_{C}^{V}

) within a cone whose opening is in the positive Z direction with its vertex at (

L_{M}

0, d

) and the opening parameters as

\tan (\frac{α}{2}) = 0.934 and \tan (\frac{β}{2}) = 0.524 .

Therefore, this inequality defines the second geometric constraint of the camera center. It forms an elliptical cone with its vertex located in Figure 12 at point E, which is the offset point of the marker

M

from the ground

.

C

(

X_{C}^{V}

Y_{C}^{V}

Z_{C}^{V}

) should also be within the sphere of the workspace (abiding the first geometric constraint), the camera operational space is presented as the overlap between the cone and the sphere (the shaded area in Figure 12).

From Figure 12, the marker should not be too far from the visual system, otherwise there is no overlap region between the cone and the sphere. The largest value of

L_{M}

, which is the distance from the marker to the coordinate center, occurs when the cone is tangent to the sphere as shown in Figure 13. To find the limit of

L_{M}

, draw a horizontal line across point

O_{V}

so that it intersects the surface of the cone at point A and the center line of the cone at point B as shown in Figure 13. Let

\bar{O_{V} A}

and

\bar{A B}

are line segments’ length.

Therefore, from the geometry, the upper limit of

L_{M}

is:

L_{M} < \bar{O_{V} A} + \bar{A B} + a_{1} = \frac{r_{V}}{\cos (\frac{α}{2})} + (L_{1} - d) \tan (\frac{α}{2}) + a_{1}

(69)

Combining with the lower bound of

L_{M}

in Equation (58), the marker M should be placed in the system so that:

2.829 m \leq L_{V T} - r_{T} \cos (θ_{2}) \leq L_{M} < \frac{r_{V}}{\cos (\frac{α}{2})} + (L_{1} - d) \tan (\frac{α}{2}) + a_{1} = 2.925 m

(70)

3.1.4. Mathematic Expression for Node Coordinates Within the Camera Operational Space

From Equation (70), the closer the marker’s distance

L_{M}

to its lower bound, the larger the operational space is. In general, larger operational space is preferred, because the camera has a larger space for searching the optimal location. Set

L_{M}

equals to its lower bound:

L_{M} = 2.83 m

(71)

Get the cross-section area at

{\bar{X}}_{V} - {\bar{Z}}_{V}

plane (Figure 14).

The function of semi-circle and the line EFG is:

({{\bar{Z}}_{V} - L_{1})}^{2} + {({\bar{X}}_{V} - a_{1})}^{2} = {r_{V}}^{2}

(72)

{\bar{X}}_{V} = - t a n (\frac{α}{2}) ({\bar{Z}}_{V} - d) + L_{M}

(73)

Combine Equations (72) and (73) and solve:

(1 + {t a n (\frac{α}{2})}^{2}) {\bar{Z}}_{V}^{2} - 2 ({t a n (\frac{α}{2})}^{2} d + t a n (\frac{α}{2}) (L_{M} - a_{1}) + L_{1}) {\bar{Z}}_{V} + {\tan (\frac{α}{2})}^{2} d^{2} + 2 \tan (\frac{α}{2}) (L_{M} - a_{1}) d + {(L_{M} - a_{1})}^{2} - {r_{V}}^{2} + {L_{1}}^{2} = 0

(74)

Solve Equation (74) by plugging numbers:

1.872 {\bar{Z}}_{V}^{2} - 6.061 {\bar{Z}}_{V} + 4.670 = 0

(75)

Solve : {\bar{Z}}_{V}_{m a x}, {\bar{Z}}_{V}_{m i n} = 1.972 m, 1.265 m

(76)

{\bar{Z}}_{V}_{m a x}

and

{\bar{Z}}_{V}_{m i n}

are the largest and smallest

{\bar{Z}}_{V}

coordinates of the camera center

C

that is within the camera operation space.

Take a value of

Z_{C}^{V}

(

{\bar{Z}}_{V}

coordinates of the camera center

C

) in the range (

{\bar{Z}}_{V}_{m i n}

{\bar{Z}}_{V}_{m a x}

) = (

1.265 m

1.972 m

). Set a plane

{\bar{Z}}_{V} = Z_{C}^{V}

, and that plane intersects the camera operational space and forms a shade area, which is the overlap between the circle and the oval, as shown in Figure 14. The inequality governing points

C

within sphere at a specific

Z_{C}^{V}

is:

({{\bar{X}}_{V} - a_{1})}^{2} + {\bar{Y}}_{V}^{2} \leq {r_{V}}^{2} - ({Z_{C}^{V} - L_{1})}^{2}

(77)

The inequality governing points

C

within cone at

Z_{C}^{V}

is:

\frac{{({\bar{X}}_{V} - L_{M})}^{2}}{{\tan (\frac{α}{2})}^{2}} + \frac{{({\bar{Y}}_{V})}^{2}}{{\tan (\frac{β}{2})}^{2}} \leq {(Z_{C}^{V} - d)}^{2}

(78)

By plotting those inequalities in Figure 15, the shaded area is where

X_{C}^{V}

and

Y_{C}^{V}

(

{\bar{X}}_{V}

and

{\bar{Y}}_{V}

coordinates of the camera center

C

) should be located within. The intersection of the two curves occurs when equality holds for (77) and (78), that is:

({{\bar{X}}_{V} - a_{1})}^{2} + {\bar{Y}}_{V}^{2} = {r_{V}}^{2} - ({Z_{C}^{V} - L_{1})}^{2}

(79)

\frac{{({\bar{X}}_{V} - L_{M})}^{2}}{{t an (\frac{α}{2})}^{2}} + \frac{{({\bar{Y}}_{V})}^{2}}{{\tan (\frac{β}{2})}^{2}} = {(Z_{C}^{V} - d)}^{2}

(80)

Solve by plugging numbers:

({\bar{Y}}_{V}_{m a x}, {\bar{Y}}_{V}_{m i n}) = \pm 0.5 * \sqrt{3.44 {* Z_{C}^{V}}^{2} - 2.0241 * Z_{C}^{V} - 29.802 + 2.451 * \sqrt{- 29.475 {* Z_{C}^{V}}^{2} + 23.707 * Z_{C}^{V} + 137.28}}

(81)

{\bar{Y}}_{V}_{m a x}

and

{\bar{Y}}_{V}_{m i n}

are the largest and the smallest

{\bar{Y}}_{V}

coordinates of the camera center

C

in the camera operation space with specific

Z_{C}^{V}

Take

Y_{C}^{V}

in the range (

{\bar{Y}}_{V}_{m i n}

{\bar{Y}}_{V}_{m a x}

), then for specific

X_{C}^{V}

{\bar{X}}_{V}_{m a x} = \sqrt{{r_{V}}^{2} - ({Z_{C}^{V} - L_{1})}^{2} - {Y_{C}^{V}}^{2}} + a_{1}

(82)

{\bar{X}}_{V}_{m i n} = L_{M} - \sqrt{{{(Z}_{C}^{V} - d)}^{2} {\tan (\frac{α}{2})}^{2} - {Y_{C}^{V}}^{2} \frac{{\tan (\frac{α}{2})}^{2}}{{\tan (\frac{β}{2})}^{2}}}

(83)

{\bar{X}}_{V}_{m a x}

and

{\bar{X}}_{V}_{m i n}

are the largest and the smallest

{\bar{X}}_{V}

coordinates of the camera center

C

in the camera operation space with specific

Z_{C}^{V}

and

Y_{C}^{V}

3.1.5. Numerical Solution of the Ideal Camera Operation Space

With known parameters from the camera and robot arm specifications, all possible location of the camera center

C

(

X_{C}^{V}

Y_{C}^{V}

Z_{C}^{V}

) inside the camera operational space can be derived from the following steps:

1. Calculate

{\bar{Z}}_{V}_{m a x}

and

{\bar{Z}}_{V}_{m i n}

by Equation (74), and mesh grid

Z_{C}^{V}

in (

{\bar{Z}}_{V}_{m i n}

{\bar{Z}}_{V}_{m a x}

2. Take a mesh value

Z_{C}^{V}

and calculate

{\bar{Y}}_{V}_{m a x}

and

{\bar{Y}}_{V}_{m i n}

by Equations (79) and (80), and mesh grid

Y_{C}^{V}

in (

{\bar{Y}}_{V}_{m i n}

{\bar{Y}}_{V}_{m a x}

3. Take a mesh

Y_{C}^{V}

and calculate

{\bar{X}}_{V}_{m a x}

and

{\bar{X}}_{V}_{m i n}

by Equations (82) and (83), and mesh grid

X_{C}^{V}

in (

{\bar{X}}_{V}_{m i n}

{\bar{X}}_{V}_{m a x}

Each above computed

C

(

X_{C}^{V}

Y_{C}^{V}

Z_{C}^{V}

) is inside the camera operation space. And those three steps completely account for all geometric constraints defined above. The number of mesh points (nodes available for searching) depends on the mesh grid size.

3.1.6. The Camera Operational Space from the Realistic Robot Manipulator

The above procedures of finding the camera operational space assumes the use of ideal robotic manipulators, whose joints are freely to move from

[0, 2 π]

. However, for realistic robotic manipulators, their revolutionary movement is limited within certain ranges of angles, and the ranges of motion in every joint of the specific robot manipulator: ABB IRB 4600 are listed in Appendix Table A2.

The geometric shape of the operational space for a realistic robotic manipulator is usually irregular (unlike an ideal manipulator whose operational space is the combination of a sphere and a cone) and the mathematical methods of directly finding the nodes within the space is computationally expensive. A faster way of generating realistic camera operational space is to decrease the ideal operational space by taking out nodes that are out of the operational ranges. Those outlier nodes can be found from the inverse kinematic model of the robotic manipulator. Figure 16 shows the decreased nodes from the ideal camera operational space to the realistic space.

3.2. Simulations of scenarios

A simulation in MATLAB is used to validate the proposed algorithm. In the simulation, the environment is set up with two minimums in the space (only one is the global minimum).

K_{e s t}

and

K_{s d}

are tunable parameters and set as

K_{e s t} = 5, a n d K_{s d} = 50 (m^{- 1}) .

The initial available energy bound is set to be

E_{b o u n d} =

12 (Watt Second (ws) or Joule). Figure 17 shows the sequences of nodes being explored in iteration steps by setting different energy threshold

E_{T}

By trials of different simulation runs, it can be concluded that:

When

E_{T}

is large (in this scenario,

{3 w s < E}_{T} \leq 12 w s),

the algorithm is conservative, so that it has only searched a small area that excludes the global minimum.

When

E_{T}

is in mid-range (in this scenario,

{0.5 w s \leq E}_{T} \leq 3 w s

), the algorithm drives the camera to the global minimum.

When

E_{T}

is small (in this scenario,

{0 w s \leq E}_{T} < 0.5 w s),

the algorithm is aggressive. Although it has searched a large amount of areas that includes the global minimum, it doesn’t have enough energy to move the camera back to the best location explored in the previous stages

The two tunable parameters

K_{e s t}

and

K_{s d}

are selected randomly in the simulation. Those parameters are used in the models to estimate the noise spatial distribution in the environment. Future research can focus on development of adaptive algorithms that tune

K_{e s t}

and

K_{s d}

over iterations based on errors between real measurements and estimations.

In addition, the numerical values of

E_{T}

are also picked randomly for testing. In real applications, a fixed

E_{T}

should be selected before the operation of this algorithm. The selection of

E_{T}

is related to the specific application scenario, size of camera’s operational space, and total energy available

E_{b o u n d}

at beginning. However, the general thumb of the rule is the following: choose small values of

E_{T}

when both

E_{b o u n d}

and operational space is large to encourage exploring over exploiting; otherwise, choose large

E_{T} .

4. Conclusion

In this article, we have developed an algorithm that a camera can explore the workspace in a manufacturing environment and search for a location so that images’ noise is minimized within the space that it can reach. This article also provides detailed development of the camera’s operational space for a specific application. Simulation results show how energy threshold

E_{T}

affects the behavior of the algorithm. A moderate threshold is suggested so that the algorithm can search for enough amount of area but enables to drive the camera back to the minimum with limited energy available. The analysis of

E_{T}

’s value in different applications is a focus of the future work. Future work may also include adaptive algorithms that update

K_{e s t}

and

K_{s d}

online. Also, orientations of the camera are fixed in this paper. In the future, we would like to develop advanced algorithms that provide not only the optimal of positions but also the orientations inside the space.

Appendix A

In this section, we will show the geometric model of a specific robot manipulator ABB IRB 4600 45/2.05[23] and a figure of a camera model: Zed 2 with dimensions [24]. This section also contains specification tables of robots’ dimensions, camera, and motor installed inside the joints of manipulators.

Figure A1. IRB ABB 4600 Model with attached frames.

Figure A2. Zed 2 stereo camera model with dimensions.

Table A1. Specification Table of ABB IRB 4600 45/2.05 Model (Dimensions).

Parameters	Values
Length of Link 1: $L_{1}$	495 mm
Length of Link 2: $L_{2}$	900 mm
Length of Link 3: $L_{3}$	175 mm
Length of Link 3: $L_{4}$	960 mm
Length of Link 1 offset: $a_{1}$	175 mm
Length of Spherical wrist: $L_{t}$	135 mm
Tool length (screwdriver): ${\bar{P J}}_{t o o l}$	127 mm

Table A2. Specification Table of ABB IRB 4600 45/2.05 Model (Axis Working range).

Axis Movement	Working range
Axis 1 rotation	+180 $°$ to -180 $°$
Axis 2 arm	+150 $°$ to -90 $°$
Axis 3 arm	+75 $°$ to -180 $°$
Axis 4 wrist	+400 $°$ to -400 $°$
Axis 5 bend	+120 $°$ to -125 $°$
Axis 6 turn	+400 $°$ to -400 $°$

Table A3. Specification Table of Stereo Camera Zed 2.

Parameters	Values
Focus length: f	2.8 mm
Baseline: B	120 mm
Weight: W	170g
Depth range:	0.5m-25m
Diagonal Sensor Size:	6mm
Sensor Format:	16:9
Sensor Size: W X H	5.23mm X 2.94mm
Angle of view in width: $α$	$86.09 °$
Angle of view in height: $β$	55.35 $°$

Table A4. Specification Table of Motors and gears.

Parameters	Values
DC Motor
Armature Resistance: $R$	0.03 $Ω$
Armature Inductance: $L$	0.1 mH
Back emf Constant: $K_{b}$	7 mv/rpm
Torque Constant: $K_{m}$	0.0674 N/A
Armature Moment of Inertia: $J_{a}$	0.09847 kg $m^{2}$
Gear
Gear ratio: $r$	200:1
Moment of Inertia: $J_{g}$	0.05 kg $m^{2}$
Damping ratio: $B_{m}$	0.06

Appendix B

In this section, we will show forward kinematics and inverse kinematics of the 6 DoFs revolute robot manipulators. The results are consistent with the model ABB IRB 4600.

Forward kinematics refers to the use of kinematic equations of a robot to compute the position of the end-effector from specified values for the joint angles and parameters. The equations are summarized in the below:

n_{x} = c_{1} s_{23} ({s_{4} s_{6} - c}_{4} c_{5} c_{6}) - s_{1} (s_{4} c_{5} c_{6} + c_{4} s_{6}) - c_{1} c_{23} s_{5} c_{6} n_{y} = s_{1} s_{23} ({s_{4} s_{6} - c}_{4} c_{5} c_{6}) + c_{1} (s_{4} c_{5} c_{6} + c_{4} s_{6}) - s_{1} c_{23} s_{5} c_{6} n_{z} = c_{23} ({s_{4} s_{6} - c}_{4} c_{5} c_{6}) + s_{23} s_{5} c_{6}

(B1)

s_{x} = c_{1} s_{23} ({s_{4} c_{6} + c}_{4} c_{5} c_{6}) {+ s}_{1} (s_{4} c_{5} s_{6} - c_{4} c_{6}) + c_{1} c_{23} s_{5} s_{6} s_{y} = s_{1} s_{23} ({s_{4} c_{6} + c}_{4} c_{5} c_{6}) - c_{1} (s_{4} c_{5} s_{6} - c_{4} c_{6}) + s_{1} c_{23} s_{5} s_{6} s_{z} = c_{23} ({s_{4} c_{6} + c}_{4} c_{5} c_{6}) - s_{23} s_{5} s_{6}

(B2)

a_{x} = {- c}_{1} s_{23} c_{4} s_{5} - s_{1} s_{4} s_{5} + c_{1} c_{23} c_{5} a_{y} = {- s}_{1} s_{23} c_{4} s_{5} + c_{1} s_{4} s_{5} + s_{1} c_{23} c_{5} a_{z} = c_{23} c_{4} s_{5} - s_{23} c_{5}

(B3)

d_{x} = L_{t} ({- c}_{1} s_{23} c_{4} s_{5} - s_{1} s_{4} s_{5} + c_{1} c_{23} c_{5}) + c_{1} (L_{3} s_{23} + L_{2} s_{2} + a_{1}) d_{y} = L_{t} ({- s}_{1} s_{23} c_{4} s_{5} + c_{1} s_{4} s_{5} + s_{1} c_{23} c_{5}) + s_{1} (L_{3} s_{23} + L_{2} s_{2} + a_{1}) d_{z} = L_{t} (c_{23} c_{4} s_{5} - s_{23} c_{5}) + L_{3} c_{23} + L_{2} c_{2} + L_{1}

(B4)

Note : c_{i} \equiv \cos (q_{i}), s_{i} \equiv \sin (q_{i}) c_{i, j} \equiv \cos (q_{i} + q_{j}), s_{i, j} \equiv \sin (q_{i} + q_{j}) i, j \in {1,2, 3,4, 5,6}

(B5)

where

{[n_{x}, n_{y}, n_{z}]}^{T}

{[s_{x}, s_{y}, s_{z}]}^{T}

and

{[a_{x}, a_{y}, a_{z}]}^{T}

are the end-effector’s directional vector of Yaw, Pitch and Roll in base frame

O_{0} X_{0} Y_{0} Z_{0}

(Figure A1). And

{[d_{x}, d_{y}, d_{z}]}^{T}

are the vector of absolute position of the center of the end-effector in base frame

O_{0} X_{0} Y_{0} Z_{0}

. For a specific model ABB IRB 4600-45/2.05 (Handling capacity: 45 kg/ Reach 2.05m) the dimensions and mass are summarized in Table A1.

Inverse kinematics refers to the mathematical process of calculating the variable joint angles needed to place the end-effector in a given position and orientation relative to the inertial base frame. The equations are summarized in the below:

p_{x} = d_{x} - L_{t} a_{x} p_{y} = d_{y} - L_{t} a_{y} p_{z} = d_{z} - L_{t} a_{z}

(B6)

q_{1} = a r c t a n (\frac{p_{y}}{p_{x}})

(B7)

q_{2} = \frac{p i}{2} - a r c c o s (\frac{{L_{2}}^{2} + {(\sqrt{{p_{x}}^{2} + {p_{y}}^{2}} - a_{1})}^{2} + {(p_{z} - L_{1})}^{2} - {L_{3}}^{2} - {L_{4}}^{2}}{2 L_{2} \sqrt{{L_{3}}^{2} + {L_{4}}^{2}}}) - a r c t a n (\frac{p_{z} - L_{1}}{\sqrt{{p_{x}}^{2} + {p_{y}}^{2}} - a_{1}})

(B8)

q_{3} = π - a r c c o s (\frac{{L_{2}}^{2} + {L_{3}}^{2} + {L_{4}}^{2} - {(\sqrt{{p_{x}}^{2} + {p_{y}}^{2}} - a_{1})}^{2} - {(p_{z} - L_{1})}^{2}}{2 L_{2} \sqrt{{L_{3}}^{2} + {L_{4}}^{2}}}) - a r c t a n (\frac{L_{4}}{L_{3}})

(B9)

q_{5} = a r c c o s (c_{1} c_{23} a_{x} + s_{1} c_{23} a_{y} - s_{23} a_{z})

(B10)

q_{4} = a r c t a n (\frac{s_{1} a_{x} - c_{1} a_{y}}{c_{1} s_{23} a_{x} + s_{1} s_{23} a_{y} + c_{23} a_{z}})

(B11)

q_{6} = - a r c t a n (\frac{c_{1} c_{23} s_{x} + s_{1} c_{23} s_{y} - s_{23} s_{z}}{c_{1} c_{23} n_{x} + s_{1} c_{23} n_{y} - s_{23} n_{z}})

(B12)

Note : c_{i} \equiv \cos (q_{i}), s_{i} \equiv \sin (q_{i}) c_{i, j} \equiv \cos (q_{i} + q_{j}), s_{i, j} \equiv \sin (q_{i} + q_{j}) i, j \in {1,2, 3,4, 5,6}

(B13)

where

{[n_{x}, n_{y}, n_{z}]}^{T}

{[s_{x}, s_{y}, s_{z}]}^{T}

{[a_{x}, a_{y}, a_{z}]}^{T}

and

{[d_{x}, d_{y}, d_{z}]}^{T}

have been defined above in the forward kinematic discussion.

References

W. Chang and C.-H. Wu, “Automated USB peg-in-hole assembly employing visual servoing,” in Proc. 3rd Int. Conf. Control, Autom. Robot. (ICCAR). Nagoya, Japan: IEEE. Apr. 2017, pp. 352–355.
J. Xu, K. Liu, Y. Pei, C. Yang, Y. Cheng, and Z. Liu, “A noncontact control strategy for circular peg-in-hole assembly guided by the 6-DOF robot based on hybrid vision,” IEEE Trans. Instrum. Meas., vol. 71,2022, Art. no. 3509815.
W. Zhu, H. Liu, and Y. Ke, “Sensor-based control using an image point and distance features for rivet-in-hole insertion,” IEEE Trans. Ind. Electron., vol. 67, no. 6, pp. 4692–4699, Jun. 2020.
F. Qin, D. Xu, D. Zhang, W. Pei, X. Han and S. Yu, "Automated Hooking of Biomedical Microelectrode Guided by Intelligent Microscopic Vision," in IEEE/ASME Transactions on Mechatronics, vol. 28, no. 5, pp. 2786-2798, Oct. 2023. [CrossRef]
Jung, Woo, Jin., Lee, Hyun, Jung. (2020). Automatic alignment type welding apparatus and welding method using the above auto-type welding apparatus.
P. Guo, Z. Zhang, L. Shi, and Y. Liu, “A contour-guided pose alignment method based on Gaussian mixture model for precision assembly,” Assem. Autom., vol. 41, no. 3, pp. 401–411, Jul. 2021.
F. Chaumette and S. Hutchinson, “Visual servo control Part I: Basic approaches,” IEEE Robotics & Automation Magazine, vol. 13, pp. 82–90, 2006.
Y. Ma, X. Liu, J. Zhang, D. Xu, D. Zhang, and W. Wu, “Robotic grasping and alignment for small size components assembly based on visual servoing,” Int. J. Adv. Manuf. Technol., vol. 106, nos. 11–12, pp. 4827–4843, Feb. 2020.
T. Hao, D. Xu and F. Qin, "Image-Based Visual Servoing for Position Alignment With Orthogonal Binocular Vision," in IEEE Transactions on Instrumentation and Measurement, vol. 72, pp. 1-10, 2023, Art no. 5019010. [CrossRef]
G. Flandin, F. Chaumette and E. Marchand, "Eye-in-hand/eye-to-hand cooperation for visual servoing," Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065), San Francisco, CA, USA, 2000, pp. 2741-2746 vol. [CrossRef]
Muis, A., & Ohnishi, K. (n.d.). Eye-to-hand approach on eye-in-hand configuration within real-time visual servoing. The 8th IEEE International Workshop on Advanced Motion Control, 2004. AMC ’04. [CrossRef]
Ruf, M. Tonko, R. Horaud, and H.-H. Nagel: “Visual Tracking of An End-Effector by Adaptive Kinematic Prediction”, Proc. of the Intl. ConJ on Intelligent Robots and Systems(IROS’97). vol. 2, pp.893-898,Grenoble, France, Sep.( 1997).
Jose Luis de Mena: “Virtual Environment for Development of Visual Servoing Control Algorithms”, Thesis, Lund Institute Technology, Sweden, May, (2002).
K. Irie, I. M. Woodhead, A. E. McKinnon and K. Unsworth, "Measured effects of temperature on illumination-independent camera noise," 2009 24th International Conference Image and Vision Computing New Zealand, Wellington, New Zealand, 2009, pp. 249-253. [CrossRef]
Patidar, P., Gupta, M., Srivastava, S., & Nagawat, A.K. Image De-noising by Various Filters for Different Noise. International Journal of Computer Applications. 2010; 9: 45-50.
Das, S., Saikia, J., Das, S., & Goñi, N. A COMPARATIVE STUDY OF DIFFERENT NOISE FILTERING TECHNIQUES IN DIGITAL IMAGES. 2015.
R. Zhao and H. Cui. Improved threshold denoising method based on wavelet transform. 2015 7th International Conference on Modelling, Identification and Control (ICMIC); 2015: pp. 1-4. [CrossRef]
Ng J., Goldberger J.J. Signal Averaging for Noise Reduction. In: Goldberger J., Ng J. (eds) Practical Signal and Image Processing in Clinical Cardiology. London: Springer; 2010. p. 69-77. [CrossRef]
G. Chen, F. Zhu and P. A. Heng. An Efficient Statistical Method for Image Noise Level Estimation. In: 2015 IEEE International Conference on Computer Vision (ICCV); 2015.p. 477-485. [CrossRef]
Mark, W.S., M.V. (1989). Robot Dynamics and control. John Wiley & Sons, Inc. 1989.
W. W. MARZOUK, A. ELBASET etc. (2019).” Modelling, Analysis and Simulation for A 6 Axis Arm Robot by PID Controller”, International Journal of Mechanical and Production Engineering Research and Development (IJMPERD) ISSN (P): 2249-6890; ISSN (E): 2249-8001 Vol. 9, Issue 4, Aug 2019, pp. 363-376. [CrossRef]
Li, R., & Assadian, F. Role of Uncertainty in Model Development and Control Design for a Manufacturing Process. In: Majid T., Pengzhong L, & Liang L, editors. Production Engineering and Robust Control. London: IntechOpen 2022. pp. 137-167. [CrossRef]
Anonymous. ABB IRB 4600 -40/2.55 Product Manual [Internet]. 2013. Available from: https://www.manualslib.com/manual/1449302/Abb-Irb-4600-40-2-55.html#manual [Accessed: 2024-7-17].
Anonymous. Stereolabs Docs: API Reference, Tutorials, and Integration. Available from: https://www.stereolabs.com/docs [Accessed: 2024-7-18].
Peer, P., & Solina, F. (2006). Where physically is the optical center? Pattern Recognition Letters, 27(10), 1117–1121. [CrossRef]
Vassilios D Tourassis, Dimitrios M Emiris. (1993) “A comparative study of ideal elbow and dual-elbow robot manipulators”, Mechanism and Machine Theory, Volume 28, Issue 3, Pages 357-373, ISSN 0094-114X. [CrossRef]

Figure 1. Algorithm flowchart.

Figure 2. (a): The operational space of the camera is an arbitrary closed 3D geometry in the space shown in the plot. The gridded area is in 3D (blue dashed lines). (b): Four initial nodes are chosen so that their connection lines (red dashed line) are perpendicular. The picture number of Node 5 can be estimated by Equation (1) from four initial explored nodes

\bar{E D}

represents Euclidean distance (green line).

\bar{E D}

represents Euclidean distance (green line).

Figure 3. Find the explorable set

S_{e x p l o r a b l e}

when the global minimum (red star) is inaccessible. (a). Step 1: Obtain the feasible range

S_{f e a s i b l e}

(green dashed line). (b). Step 2: Find the explored node with a minimum picture number in the feasible range. Initial nodes (blue stars) are explored (blue circles).

{N o d e}_{1}

{N o d e}_{e x p_m i n}

. (c). Step 3: Obtain explorable range when

E_{b o u n d}

\leq E_{T}

. The explorable range

S_{e x p l o r a b l e}

(blue dashed line) is a subset of

S_{f e a s i b l e}

Figure 3. Find the explorable set

S_{e x p l o r a b l e}

when the global minimum (red star) is inaccessible. (a). Step 1: Obtain the feasible range

S_{f e a s i b l e}

(green dashed line). (b). Step 2: Find the explored node with a minimum picture number in the feasible range. Initial nodes (blue stars) are explored (blue circles).

{N o d e}_{1}

{N o d e}_{e x p_m i n}

. (c). Step 3: Obtain explorable range when

E_{b o u n d}

\leq E_{T}

. The explorable range

S_{e x p l o r a b l e}

(blue dashed line) is a subset of

S_{f e a s i b l e}

Figure 4. Both case1 and case2 have the same estimated value of an unexplored node (green point) from Equation (1). However, the node in case 1 has a larger distance to explored nodes (red points) compared to that in case 2. Therefore, we set different standard deviations (

σ_{1}

σ_{1}

Figure 5. The block Diagram of feedback linearization Youla control design used for the joint control loop.

Figure 6. Response of

q^{1} (t)

with varying

τ_{d e l a y}

Figure 6. Response of

q^{1} (t)

with varying

τ_{d e l a y}

Figure 7. The topology of the multi-robotic system for accurate positioning control.

Figure 8. Spherical Wrist Model.

Figure 9. Workspace of an Ideal Elbow Manipulator.

Figure 9. The workspace configuration of the multi-robotic system.

Figure 10. The Detectable Space of the Stereo Camera.

Figure 11. The Detectable Elliptic Cone in the Inertial Visual System Frame.

Figure 12. Illustration of Camera Operational Space (the shaded area).

Figure 13. An Extreme Case of Marker Position.

Figure 14. The Camera Operational Space Cross-section at

{\bar{X}}_{V} - {\bar{Z}}_{V}

plane.

Figure 14. The Camera Operational Space Cross-section at

{\bar{X}}_{V} - {\bar{Z}}_{V}

plane.

Figure 15. The Camera Operational Space Cross-section at

{\bar{X}}_{V} - {\bar{Y}}_{V}

plane.

Figure 15. The Camera Operational Space Cross-section at

{\bar{X}}_{V} - {\bar{Y}}_{V}

plane.

Figure 16. Reduction of Nodes from the ideal to the realistic camera operational space.

Figure 17. Plots of sequential exploration with different

E_{T}

Figure 17. Plots of sequential exploration with different

E_{T}

Table 1. Estimated Energy Cost and Settling Time with Varying Delay Constant.

$τ_{d e l a y}$ [s]	$E (P_{i n i t i a l}, P_{f i n a l})$ [ws]	$t_{s} [s]$
0	10.895	0.07
0.2	0.102	0.73
0.4	0.030	1.42
0.6	0.016	2.11
0.8	0.011	2.80
1	0.009	3.49

Note:

E (P_{i n i t i a l}, P_{f i n a l})

is measured as [watts seconds]

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer

Energy Aware Camera Location Search Algorithm for Increasing Precision of Observation in Automated Manufacturing

Abstract

1. Introduction

2. Methods

2.1. Camera Location Search Process and Algorithm Flowchart

2.2. Core Alogorithm

2.2.1. Picture Number Estimation Function

2.2.2. Explorable Node Selection

2.2.3. Stochastic Modified Evaluation

2.2.4. Estimated Energy Cost Function

3. Results and Discussion

3.1. Camera Operational Space

3.1.1. Reachable and Dexterous Workspace of Two-Hybrid Systems

3.1.2. Detectable Space for the Stereo Camera

3.1.3. Camera Operational Space Development

3.1.4. Mathematic Expression for Node Coordinates Within the Camera Operational Space

3.1.5. Numerical Solution of the Ideal Camera Operation Space

3.1.6. The Camera Operational Space from the Realistic Robot Manipulator

3.2. Simulations of scenarios

4. Conclusion

Appendix A

Appendix B

References

MDPI Initiatives

Important Links

Subscribe