Global Sensitivity Analysis of Structural Reliability Using Cliff Delta

1. Introduction

Global sensitivity analysis (GSA) focuses on attributing the uncertainties of model outputs, or related performance indicators, to their inputs, thereby assessing the impact of input uncertainties on outputs or performance indicators [1,2]. Various GSA methods have been developed for this purpose [3,4]. These methods include the screening method [5,6], variance-based methods [7,8], moment-independent methods [9,10], and derivative-based methods [11,12]. Among these, variance-based sensitivity indices, also known as Sobol’ indices [7,8], are particularly notable for their mathematical elegance in measuring the individual, interaction, and total contributions of each input to the model output uncertainty, see, e.g. [13,14].

In the limit state method, probabilistic reliability analysis is based on the estimation of the failure probability [15]. Sobol' indices have been widely used in structural reliability analysis to pinpoint variables that significantly influence failure probability [16,17]. These sensitivity indices, which focus on failure probability P_f, are derived from the variance decomposition of a binary function representing failure and success [16,17]. Fort et al. [18] expanded on Sobol's sensitivity indices by introducing a contrast function in place of variance, allowing the indices to be oriented towards variance, probability, and quantile. This approach maintains the non-negative property of the indices and ensures that their sum equals one, as in traditional Sobol sensitivity analysis. Extending GSA to P_f and design quantile represented a significant advancement in civil engineering, as these quantities are crucial in structural reliability assessments [19,20].

The development of GSA methods focused on reliability demands precise estimation of P_f [21]. However, the complexity of mathematical models and the difficulty of uncertainty propagation using sampling-based methods such as Monte Carlo (MC) simulations, due to the large number of runs required, present significant challenges [22]. Nonlinear finite element models are particularly demanding on CPU time [23], which further complicates the process of structural reliability estimation, see, e.g., [24,25]. To ensure the most accurate estimate of P_f​, it is essential to develop methods that provide precise P_f ​estimates while minimizing the computational costs associated with repeated calls to the computational model [26,27].

The computational burden can be reduced by using metamodels, also known as surrogate models, which approximate the behavior of complex models with simpler ones [28,29]. Methods such as polynomial response surface [30,31], response surface based multi-fidelity model [32], polynomial chaos expansion (PCE) [33,34], Gaussian process [35,36], Kriging [37,38], and neural network [39,40,41] are used to the creation of such metamodels. While the use of metamodels significantly reduces computational burden by approximating complex models with simpler ones, there are some critical drawbacks to this approach [42,43,44]. Despite the efficiency of new meta-models in sampling [45,46], traditional (quasi-) Monte Carlo methods remain the primary choice for practical sensitivity analysis [3,4].

Existing research frequently explores the rate of advancements across various Monte Carlo based reliability applications [47,48], but there is a notable lack of studies focusing on the implications of these advancements for enhancing the efficiency of reliability-oriented global sensitivity analyses (GSAs). Ensuring the accurate estimation of failure probability P_f with the available number of simulations is critical because it directly influences reliability assessments and decision-making processes. More accurate P_f estimation is advantageous when using both the original model and the metamodel, depending on the available computational resources.

The solution proposed in this article involves adopting an alternative measure of structural reliability based on Cliff's Delta [48], which can be calculated using double-nested-loop simulations. This approach enhances the precision of P_f estimation without requiring additional computational effort, offering a robust alternative to traditional metamodel-based GSA. By maximizing the utility of existing simulations, this method ensures that reliability analyses are both more accurate and computationally efficient.

2. Cliff Delta

Cliff delta, denoted as δ_C​, was initially devised by Norman Cliff, primarily for handling ordinal data [49]. It serves as a metric to assess the frequency with which values from one distribution exceed those in another distribution. A key feature of δ_C is that it does not necessitate any specific assumptions regarding the distributions’ form or variability.

The formula for computing the sample estimate of δ_C is expressed as:

$${\delta }_C=\frac{{\displaystyle {\sum }_{i=1}^m{\displaystyle {\sum }_{j=1}^n\left[y_i>y_j\right]-\left[y_i<y_j\right]}}}{m\cdot n},\mathrm{where}{\delta }_C\in [–1,1]$$

(1)

In this equation, the two distributions are characterized by sizes n and m, with respective elements y_i and y_j​. Here, the notation [⋅][⋅] refers to the Iverson bracket notation, resulting 1 if the condition within the brackets holds true, and 0 otherwise. This statistical approach allows for an intuitive comparison of two distributions by quantifying the dominance of one distribution over the other.

Building upon the foundational description of δ_C​, this measure is specifically applied to assess the relationship between resistance R and load force F within a framework of structural reliability. In the limit state, a structure is reliable if R ≥ F, otherwise failure occurs [19]. Let the difference between R and F be denoted as

Z = R – F,

(2)

where R and F are statistically independent random variables. In the Monte Carlo simulations, these variables are represented as arrays R and F, corresponding to resistance and force, respectively. Cliff's delta, δ_C​, is utilized to quantitatively evaluate the extent to which values of resistance exceed or fall below those of force, thereby providing fundamental insights into system reliability.

Assuming arrays R and F are of equal size, with n entries each, the formula for calculating δ_C is simplified to:

$${\delta }_C=\frac{{\displaystyle {\sum }_{i=1}^n{\displaystyle {\sum }_{j=1}^n\left[R_i>F_j\right]-\left[R_i<F_j\right]}}}{n^2},\mathrm{where}{\delta }_C\in [–1,1]$$

(3)

In this expression, R_i and F_j represent the i-th and j-th entries in the R and F arrays, respectively, denoting random realizations of resistance and force. The Iverson brackets, [⋅][⋅], return 1 when the enclosed condition is true and 0 otherwise. This formulation facilitates a direct comparison between resistance and force across all sampled scenarios.

The estimation of δ_C between the measurements of resistance and force provides a metric quantifying the frequency with which resistance values surpass those of force. Employed as a statistical tool, this measure assesses how frequently resistance can withstand or exceed applied forces.

3. Sensitivity Measures of Cliff's Delta

Although Cliff's delta has been applied in numerous studies, see e.g. [50,51,52,53,54], its utilization in global sensitivity analysis of reliability is absent. This chapter demonstrates that the sensitivity measures based on the squared value of Cliff's delta, ${\delta }_C^2$​, exhibit properties similar to variance in Sobol's sensitivity analysis [7,8], oriented to reliability [16,17].

3.1. Approximation of Failure Probability with Cliff's Delta in Sensitivity Analysis

In the limit state, a structure is considered reliable if R ≥ F; otherwise, failure occurs. The probability of failure P_f can be defined as the overload probability that F > R (i.e., Z < 0). The failure probability can be expressed as the mean value of the binary reliability function of the Bernoulli distribution, where 1 occurs if Z < 0, and 0 otherwise:

$$P_f=P\left(Z<0\right)=E\left(1_{Z<0}\right),$$

(4)

where:

$$1_{Z<0}=\frac{\left|Z\right|-Z}{2\left|Z\right|}.$$

(5)

The conventional measure of reliability is 1−P_f​. The variance of the Bernoulli distribution of the random variable 1_Z < 0 can be written as:

$$V\left(1_{Z<0}\right)=P_f\left(1-P_f\right).$$

(6)

The second moment is a function of P_f, which is useful for the formulation of Sobol sensitivity indices, computable through the estimation of the conditional realizations of P_f [21]. If the variance V(1_Z<0) is used in the decomposition within Sobol sensitivity analysis, the first-order sensitivity index of the variance function 1_Z<0 can be expressed as:

$$S_i=\frac{V\left(E\left(1_{Z<0}|X_i\right)\right)}{V\left(1_{Z<0}\right)}=\frac{V\left(1_{Z<0}\right)-E\left(V\left(1_{Z<0}|X_i\right)\right)}{V\left(1_{Z<0}\right)}=\frac{P_f\left(1-P_f\right)-E\left(P_f|X_i\left(1-P_f|X_i\right)\right)}{P_f\left(1-P_f\right)}.$$

(7)

The concept of sensitivity analysis based on Cliff's Delta is predicated on the assumption that Cliff's Delta can be expressed as

$${\delta }_C=P\left(Z>0\right)-P\left(Z<0\right)=\left(1-P_f\right)-P_f=1-2P_f.$$

(8)

The failure probability P_f can be calculated using Cliff's Delta as follows:

$$P_f=\frac{1-{\delta }_C}{2}.$$

(9)

Similarly, the second moment can be approximated and written as a function of Cliff's Delta as:

$$V\left(1_{Z<0}\right)=P_f\left(1-P_f\right)=\left(\frac{1-{\delta }_C}{2}\right)\left(1-\frac{1-{\delta }_C}{2}\right)=\frac{1-{\delta }_C^2}{4}.$$

(10)

Substituting this into the Equation (7), the Sobol sensitivity index can be expressed using δ_C as:

$$S_i=\frac{\frac{1-{\delta }_C^2}{4}-E\left(\frac{1-{\delta }_C^2|X_i}{4}\right)}{\frac{1-{\delta }_C^2}{4}}=\frac{E\left({\delta }_C^2|X_i\right)-{\delta }_C^2}{1-{\delta }_C^2}.$$

(11)

This formulation provides an alternative method for calculating the sensitivity index, which carries all the advantages and disadvantages associated with the estimation of Cliff's Delta compared to failure probability.

3.2. Sensitivity Indices Based on Cliff Delta

In the development of sensitivity indices for the evaluation of structural reliability, the use of the squared measure of Cliff's delta, ${\delta }_C^2$​, has been proposed. This chapter delineates the formal definitions of these indices, categorized from first to higher orders. The sensitivity indices are computed as ratios of differences normalized by a constant, 1-C, where C₀ is defined as square of Cliff's delta, ${\delta }_C^2$​.

$$C_0={\delta }_C^2,$$

(12)

where ${\delta }_C^2$ represents the measure calculated when all input random variables, X₁, X₂,..X_M, of R and F are random.

The first-order sensitivity index, S_i​, is defined to quantify the effect of a single variable X_i on the change observed in the squared Cliff's delta, ${\delta }_C^2$​. It is calculated as follows:

$$S_i=\frac{E\left({\delta }_C^2|X_i\right)-C_0}{1-C_0}.$$

(13)

In Equation (13), having frozen one potential source of variation (X_i​), the resulting $E\left({\delta }_C^2|X_i\right)$ will be higher than the corresponding total or unconditional Cliff's delta, where C₀=${\delta }_C^2$​​. For example, if X_i were the sole source of change in the distance between R and F, fixing it to $x_i^{*}$ would result in $\left({\delta }_C^2|X_i=x_i^{*}\right)$=1.

The second-order sensitivity index, S_ij​, extends the analysis to pairs of variables, evaluating the joint effect of X_i and X_j on the change of ${\delta }_C^2$​. This index is expressed as:

$$S_{ij}=\frac{E\left({\delta }_C^2|X_i,X_j\right)-C_0}{1-C_0}-S_i-S_j.$$

(14)

Similarly, the third-order index, S_ijk​, considers the combined influence of three variables X_i​, X_j​, and X_k​. It is calculated by:

$$S_{ijk}=\frac{E\left({\delta }_C^2|X_i,X_j,X_k\right)-C_0}{1-C_0}-S_i-S_j-S_k-S_{ij}-S_{ik}-S_{jk}.$$

(15)

Other Cliff's sensitivity indices, which quantify higher-order interaction effects, are defined analogously. The sensitivity index of the last order can be expressed as follows:

$$S_{1,2,...,M}=\frac{E\left({\delta }_C^2|X_1,X_2,...,X_M\right)-C_0}{1-C_0}-{\displaystyle {\sum }_{p=1}^{M-1}{\displaystyle {\sum }_{1\le i_1<...<i_p\le M}{S}_{i_1,i_2,...,i_p}}}=1-{\displaystyle {\sum }_{p=1}^{M-1}{\displaystyle {\sum }_{1\le i_1<...<i_p\le M}{S}_{i_1,i_2,...,i_p}}},$$

(16)

where E(${\delta }_C^2$|X₁, X₂,…,X_M)=1 is ensured due to the nature of Cliff's delta, which assumes a value of either 1 or -1 when all input random variables are fixed. Consequently, each element in the array R adopts a consistent identical value denoted as v₁, and similarly, every element in the array F maintains another consistent identical value, denoted as v₂. It should be noted that these constants v₁ and v₂ are generally different.

The sum of all indices equals one. This characteristic is guaranteed by the computation of the last order sensitivity index, as shown in Equation (16), which is derived from the difference between 1 and the sum of all lower order sensitivity indices.

$${\displaystyle \sum _i{S}_i}+{\displaystyle \sum _i{\displaystyle \sum _{j>i}{S}_{ij}}}+{\displaystyle \sum _i{\displaystyle \sum _{j>i}{\displaystyle \sum _{k>j}{S}_{ijk}}}}+...+S_{\mathrm{123...}M}=1.$$

(17)

The non-negativity of sensitivity indices is proven due to their association with variance, see Equation (10), and the Sobol decomposition of variance. The fixing multiple input variables typically leads to a higher value of ${\delta }_C^2$ (with a limit of one in the last order sensitivity index) compared to a constant C₀. The properties of sensitivity indices based on Cliff's delta and their comparison with classical Sobol sensitivity analysis will be further explored in later chapters.

3.3. Sensitivity Indices Based on Failure Probability

The impact of input variables on the failure probability P_f can be analyzed using sensitivity analysis based on contrast functions [18]. Unlike Sobol's sensitivity analysis, contrast-oriented sensitivity analysis employs a contrast function, whose minimizer is of primary interest [55,56]. Fort [18] demonstrates that employing the quadratic contrast function, which calculates the mean of the squared deviations from the average, results in the well-known Sobol sensitivity indices [7,8]. In reliability-oriented sensitivity analysis, the mean value is considered as the failure probability P_f as the first moment of the binary reliability function P_f = E(1_Z<0), while the variance, upon which Sobol's sensitivity analysis is based, is considered as the second moment of the binary reliability function V(1_Z<0) = P_f (1- P_f).

The first-order sensitivity index, C_i​, quantifies the main effect of a single variable X_i on the variance of the binary reliability function:

$$C_i=\frac{V\left(E\left(1_{Z<0}|X_i\right)\right)}{V\left(1_{Z<0}\right)}=\frac{P_f\left(1-P_f\right)-E\left(P_f|X_i\left(1-P_f|X_i\right)\right)}{P_f\left(1-P_f\right)}.$$

(18)

In this equation, the freezing of the source of variation X_i affects P_f​. The sensitivity index C_i indicates by how much one could reduce, on average, the output variance of the binary function 1_Z<0 if X_i could be fixed. Hence, it is a measure of the main effect.

The second-order sensitivity index, C_ij​, measure the pair effects of X_i and X_j on the variance of binary function 1_Z<0 ​. This index can be write as:

$$C_{ij}=\frac{V\left(E\left(1_{Z<0}|X_i,X_j\right)\right)}{V\left(1_{Z<0}\right)}-S_i-S_j.$$

(19)

Similarly, the third-order index, C_ijk​, considers the combined influence of three variables X_i​, X_j​, and X_k​, calculated by:

$$C_{ij}=\frac{V\left(E\left(1_{Z<0}|X_i,X_j,X_k\right)\right)}{V\left(1_{Z<0}\right)}-S_i-S_j-S_k-S_{ij}-S_{ik}-S_{jk}.$$

(20)

Higher-order contrast sensitivity indices, which quantify higher-order interaction effects, are defined analogously. The sum of all sensitivity indices equals one.

$${\displaystyle \sum _i{C}_i}+{\displaystyle \sum _i{\displaystyle \sum _{j>i}{C}_{ij}}}+{\displaystyle \sum _i{\displaystyle \sum _{j>i}{\displaystyle \sum _{k>j}{C}_{ijk}}}}+...+C_{\mathrm{123...}M}=1.$$

(21)

The non-negativity of sensitivity indices is guaranteed by their derivation from Sobol's sensitivity analysis [6,7,8], see also [19,21].

4. The case study

In the case study, the new sensitivity indices are compared with the results of Sobol sensitivity analysis using a simple example. In Equation (2), the resistance can be considered as:

R =  X₁·X₂+X₂·X₃+K,

(22)

where K is constant. The load force is considered as

F = X₄·X₅.

(23)

All input random variables, X₁, X₂,…,X₅​, follow a Gaussian probability density function with a mean value of zero and a standard deviation of one.

Table 1 presents the estimated values of C₀ =${\delta }_C^2$​, where ${\delta }_C$ is obtained using n = 12,000 runs of the Latin Hypercube Sampling (LHS) method [57,58]. The conditional values of ${\delta }_C$ are estimated using double-nested-loop computation of LHS method. When a single input variable, X_i​, is fixed, it is sampled using n=12,000 runs, and for each realization, ${\delta }_C$ is calculated again using n=12,000 runs of the LHS method. This numerical procedure is analogous to Sobol [7,8], but the sensitivity measurement is not based on variance but on Cliff's delta. In computing of S_i, the computational complexity of E(${\delta }_C^2$|X_i) is n²=144,000,000.

In the Table 1, the data in the last column represent a set of values that are highly consistent, with low variance. Most values range between 0.924 and 0.940, with a last exceptions approaching 1.000. In Table 1, the value 1 is always present on the last row, ensuring that the sum of all indices equals one.

The column with K=0 exhibits values close to zero, indicating the absence of dominance of R over F in the observations of ${\delta }_C^2$​​. Conversely, values far from zero suggest a dominance of R over F in the observation of ${\delta }_C^2$​​.

In general, the strong influence of an input variable or variables occurs when the mean value of the fixed realizations of ${\delta }_C^2$​​ is significantly different from C₀​, see Equations (13), (14), and (15). It can be noted that the accuracy of estimation of sensitivity indices is lowest for K=4, where the conditioned realizations of ${\delta }_C^2$​ in the last column of Table 1 are very consistent, with low variance, and are minimally different from C₀.

The influences of input variables and their groups, as expressed by sensitivity indices, are displayed in Figure 1, Figure 2 and Figure 3.

In the Figure 1, all first-order sensitivity indices are zero. The second-order sensitivity indices S₁₂=0.18 and S₂₃​=0.18 have the same value, which is due to the nature of Equation 10 and the same characteristics of the input random variables. The dominant influence is the interaction effect of variables X₄ and X₅, as indicated by the value S₄₅=0.26.

Increasing the value of the constant K distances the random realizations of resistance R from load action F and enhances the value of Cliff's Delta. The interaction effects indicated by sensitivity indices S₁₂​, S₂₃​, and S₄₅ decrease with increasing K, while the proportion of third, fourth, and fifth-order sensitivity indices increases, see Figure 2 and Figure 3.

In sensitivity analysis based on Cliff's Delta, the impact of input variables on the observed changes in ${\delta }_C^2$ is quantified using sensitivity indices of the first order and higher orders. To derive meaningful conclusions and categorize input variables into influential, less influential, and non-influential groups, it is essential to assign the effects of each input variable without the complexity of interpreting numerous sensitivity indices.

To achieve this, the concept of the total effect index is employed. This index captures the comprehensive contribution of a factor, X_i​, to the changes observed in ${\delta }_C^2$​. Specifically, it encompasses both the first-order effects and all higher-order effects resulting from interactions. The total effect index provides a robust measure of the impact that each input variable has on the ${\delta }_C^2$​, accounting for all potential interactions.

For instance, in a five-factor model, the total effect of the factor X₁ is calculated by summing all terms in Equation (17) where the factor X₁ is included.

$$S_{T1}=S_1+S_{12}+S_{13}+S_{14}+S_{15}+S_{123}+S_{124}+S_{125}+S_{134}+S_{135}+S_{145}+S_{1234}+S_{1235}+S_{1245}+S_{1345}+S_{12345}.$$

(24)

This sum accounts for X₁ ​direct influence on ${\delta }_C^2$ as well as its synergistic effects with other factors. The total effect measure provides the educated answer to the question, Which factor can be fixed anywhere over its range of variability without affecting the ${\delta }_C^2$. The total effect index reflects both the main and the interaction influences of X₁ on the outcome, providing a comprehensive view of its relative importance in the system's reliability, measured by the distance from F to R. Total effects for the case study are displayed in Figure 4.

The sensitivity analysis results show the Total Sensitivity Indices for each input variable (X₁, X₂, X₃, X₄, X₅) across five distinct constant values (K=0, 1, 2, 3, 4), using Cliff's delta as the sensitivity measure.

As K increases from 0 to 4, a general trend of increasing total sensitivity indices is observed for X₁, X₂, as depicted in Figure 4. For X₃, the total sensitivity indices exhibit a slightly convex pattern. An increasing trend for X₃ is observed from K=1 to K=4. This suggests that these variables become more influential with higher values of K, as measured by Cliff's delta. Conversely, the sensitivity indices for X₄ and X₅ decrease with increasing K.

The variable X₂ consistently exhibits the highest total sensitivity index across all tested values of K, indicating its dominant influence on Cliff's delta. This effect is attributable to X₂'s involvement in both additive terms of the resistance function, as shown in Equation (22). The variables X₁ and X₃ also demonstrate an increasing influence, although they remain slightly less dominant than X₂ but are notably more influential than X₄ and X₅ as K increases. The influence of X₄ and X₅, which are involved in the load force equation, decreases especially as K surpasses 1, highlighting their reduced significance in affecting Cliff's delta.

The sensitivity analysis outcomes reflect a decreasing probability P(R<F), which diminishes as K increases. The Cliff delta-based sensitivity analysis exhibits characteristics of reliability-oriented sensitivity analysis [19], describing the change in the influence of each variable on Cliff's delta due to K. The influence of the results of the sensitivity indices by the value of the deterministic quantity K is the main difference compared to Sobol sensitivity analysis.

The results of the classical Sobol sensitivity analysis can be calculated analytically. Non-zero values of Sobol's sensitivity indices were obtained only for second-order sensitivity indices $S_{12}^{Sob}$​=1/3​, $S_{23}^{Sob}$​=1/3​, $S_{45}^{Sob}$​=1/3 , other Sobol indices are zero. The total effect Sobol sensitivity indices are $S_{T1}^{Sob}$​=1/3​, $S_{T2}^{Sob}$​=2/3​, $S_{T3}^{Sob}$​=1/3​, $S_{T4}^{Sob}$​=1/3​, $S_{T5}^{Sob}$​=1/3. The dominant influence of the input variable X₂ confirms the most important conclusions of the newly introduced sensitivity analysis based on Cliff's Delta, however, there are differences. The results of Sobol's sensitivity analysis are independent of the value of the constant K, because Sobol's indices are based on the decomposition of variance, which the deterministic variable K does not affect.

Although classic Sobol's sensitivity analysis of model output is empathetic to the results of reliability-oriented types of sensitivity analyses, it is not directly oriented towards reliability, as Sobol sensitivity indices do not reflect changes in P_f [19].

The sensitivity indices results based on Cliff's Delta closely resemble those obtained from the sensitivity analysis based on P_f ​. The pie chart on the left side of Figure 4 is practically identical to the chart in Figure 1. Moreover, the pie chart on the right side of Figure 4 is very similar to the chart on the right side of Figure 5. The results of the sensitivity analysis are the same, but the sensitivity scale is different. Overall, this demonstrates a high degree of similarity between Cliff's Delta and P_f, from which useful conclusions can be drawn.

The sensitivity indices based on Cliff's Delta are derived from the calculation of P_f using Cliff's Delta. While it may seem that estimating Cliff's Delta is more demanding compared to P_f​, using double-loop simulations to estimate Cliff's Delta leads to a more accurate numerical estimate of P_f and extracts more information from the available simulations. It can be noted that a similar double-loop simulation is used for estimating P_f based on the numerical integration of the distributions of the random variables R and F , see e.g. [59].

The advantage of estimating Cliff's Delta is that it does not require knowledge of the distributions or approximation approaches of the random variables R and F. This flexibility allows for a more robust analysis in practical scenarios where precise distribution functions may not be available or easily determined.

5. Comparative Analysis of P_f Estimations Using Cliff's Delta and Basic Definition

It can be showed that the estimation of P_f according to Equation (9) is more accurate compared to the basic definition in Equation (4) when Cliff's delta is calculated using a double nested-loop simulation according to Equation (3).

Let P_f ​denote the failure probability estimated from a binomial distribution, and let n represent the number of simulations or experiments.

$$P_f=\frac{1}{n}{\displaystyle \sum _{i=1}^n{1}_{Z<0}}.$$

(25)

The standard error SE for the failure probability estimation is provided by the binomial distribution

$$S{E}_1\left(P_f\right)=\sqrt{\frac{P_f\left(1-P_f\right)}{n}}.$$

(26)

The estimate of the failure probability P_f using Cliff’s Delta in Equation (9) is more accurate because it utilizes an increased number of simulations through a double-nested-loop process. However, the standard error SE₂(P_f) for this estimate cannot be straightforwardly determined using an analytical formula similar to SE₁(P_f). Instead, empirical observations indicate that the standard error for P_f estimated with Cliff’s Delta is smaller. This improved accuracy is attributed to the increased statistical information gained from the double-nested-loop simulations. The accuracy of both estimates can be demonstrated in the following case study using the Monte Carlo method.

The limit state of a rod made of elastic material, subjected to axial tension, is being studied. Let resistance R have a Gaussian probability density function with a mean value μ_R=9 kN and standard deviation σ_R​=0.9 kN, and let F have a Gaussian probability density function with a mean value μ_F​=4 kN and standard deviation σ_F=1.218887 kN. Under these assumptions, Z is a random variable with a Gaussian probability density function with a mean value μ_Z=9−4=5 kN and standard deviation σ_Z =(0.9²+1.218887²)^0.5=1.515 kN. ​The failure probability P_f ​=0.000483477 is evaluated by numerical integration of the Gaussian probability density function of Z from negative infinity to zero. According to EN1990 [15], a structure designed with this failure probability is classified in reliability class RC1 for reference periods of 1 and 50 years.

The aim of the case study is to compare the accuracy of P_f estimation according to the basic definition in Equation (4) and the alternative formula in Equation (9). The procedure is as follows: In the first step, the random variables R and F are simulated for 10,000 runs using the Monte Carlo (MC) method. The estimation of P_f from Equation (4) is plotted in Figure 6 as the first blue point from the left. The estimation of P_f from Equation (9) is plotted in Figure 7 as the first blue point from the left. Next, another 10,000 runs of MC are generated, and the procedure is repeated. In total, 100 estimates of P_f according to Equation (4) are plotted in the left quarter of Figure 6, and 100 estimates of P_f according to Equation (9) are plotted in the left quarter of Figure 7. This procedure is further repeated for 100,000, 1 million, and 10 million runs of the Monte Carlo (MC) method, as shown in the subsequent quarters in Figure 6 and Figure 7.

The numerical study revealed that the failure probability P_f estimation using Cliff's Delta δ_C, see Equation (9), is more accurate compared to the basic definition, see Equation (4). As illustrated in Figure 6 and Figure 7, the alternative method resulted in noticeably lower standard deviations σ_Pf (approximately three times smaller) across varying Monte Carlo run counts, indicating improved consistency and precision. In contrast, the basic definition exhibited higher variability, especially with fewer Monte Carlo runs. These findings underscore the efficacy of the alternative formula for more reliable P_f estimation in structural reliability assessments.

6. Discussion

In civil engineering, estimating the resistance R using nonlinear FEM calculations is very time-consuming, and studies are limited by the number of computationally expensive Monte Carlo or LHS method runs required to accurately simulate real structural behavior, see e.g., [60,61,62,63]. In recent years, the role of mathematical models that offer quick analytical solutions has become increasingly significant due to their efficiency in providing rapid responses [64,65]. For each model, it is useful to utilize all realized simulations to estimate failure probability as efficiently as possible.

Equation (9) offers an efficient alternative for estimating P_f with high utilization of information from the realized runs of the computational model. Although estimating Cliff's Delta is more numerically demand than direct estimation of failure probability using Equation (4), the same number of model runs (the same number of random realizations of R and F) ensures a more accurate estimation of failure probability P_f, see Figure 6 compared to Figure 7. These advantages highlight the utility of Cliff's Delta in both reliability and sensitivity analysis, making it an innovative tool for engineers and researchers to gain deeper insights into the reliability and performance of structural systems.

The computational complexity of estimating Cliff's Delta can be effectively reduced by optimizing computer algorithms. The following function formulates the algorithm for calculating Cliff's Delta using ten thousand random realizations of R and F in arrays AR and AF. In the Delphi programming language, the function to calculate Cliff's Delta can be written as follows:

`pascal

const max=10000;

var

 AR,AF: array[1..max] of Single;

 AB : array[1..max] of Boolean;

Function Cliffd:extended;

var i1,i2 :integer;

  Dominant, Dominated : Int64;

begin

 Dominant:=0;

 Dominated:=0;

 for i1:=1 to max do

 begin

 if AB[i1] then Dominant:=Dominant+max

    else

 for i2:=1 to max do

 begin

  if (AR[i2]>AF[i1]) then Inc(Dominant) else

  if (AR[i2]<AF[i1]) then Inc(Dominated);

  end;

 end;

 Cliffd:=(Dominant-Dominated)/(max*max);

end;

`

Before calling the function, the array AB is populated with random realizations for which the inner loop runs are unnecessary.

`pascal

for i1 := 1 to max do if AF[i1] < 5.49836 then AB[i1] := true else AB[i1] := false;

`

The constant 5.49836 is the smallest value of the random variable R generated by the Monte Carlo method. If a random realization of F is less than 5.49836, then all random realizations of R are less than 5.49836, and the inner loop does not need to be executed; it is sufficient to assign Dominant:=Dominant+max. In the case study, using max=10000, the estimation of Cliff's Delta was accelerated by approximately two times.

The algorithm efficiently reduces the double-loop computational burden especially for Cliff's Delta estimates close to one (very small P_f values), where the vast majority of random realizations of R are much higher than F. According to the EN1990 [15] standard, common building structures are designed with a P_f around 7.2 ·10^-5. For estimating the standard error of the P_f ​using Equation (25), one needs to perform 1,388,889 Monte Carlo simulation runs to achieve a coefficient of variation of 0.1. This results in a standard error of SE₁=7.2⋅10−6. In practice, this requires conducting Monte Carlo runs until 100 failure events are observed, i.e., 100 runs where Z < 0. Subsequently, using Equation (9) instead of Equation (4) will improve the accuracy of the P_f estimate.

Conducting sensitivity analysis based on Cliff's Delta offers functionalities that are identical to sensitivity analysis based on failure probability focused on reliability. Cliff's Delta provides a robust measure for evaluating the extent to which one distribution dominates another without making specific assumptions about the distributions' form or variability. This flexibility is particularly valuable in reliability studies where input variables may not follow normal distributions or exhibit significant variability. By focusing on the frequency with which resistance values exceed load values, Cliff's Delta directly aligns with the fundamental concepts of reliability engineering. The ability to estimate failure probabilities using Cliff's Delta enhances the robustness of reliability and sensitivity analysis, especially in cases where performing additional resistance or load simulations is numerically difficult or impossible.

7. Conclusions

In this article, an alternative measure of structural reliability based on Cliff's Delta has been adopted to ensure more accurate estimation of failure probability P_f with the available number of simulations. Using Cliff's Delta increases the accuracy of the Pf estimate by making better use of the statistical information from the data. This approach replaces the traditional failure probability calculations and provides enhanced flexibility and robustness in applications of structural reliability.

The adaptation of Cliff's Delta, initially developed for ordinal data, has been successfully applied to evaluate the sensitivity of structural reliability. This adaptation allows for the evaluation of the dominance of resistance over load without specific distributional assumptions, making it particularly suitable for structural reliability analysis. The mathematical formulations for computing Cliff's Delta between resistance R and load action F have been presented. This formulation effectively quantifies the reliability of a structure by evaluating the probability that resistance exceeds load.

Sensitivity indices based on the squared value of Cliff's Delta ${\delta }_C^2$ have been derived, demonstrating properties analogous to those used in Sobol's sensitivity analysis. This includes first-order, second-order, and higher-order sensitivity indices, which offer a comprehensive framework for evaluating the contributions of individual variables and their interactions to the overall reliability of the system. These indices provide a nuanced understanding of the factors influencing structural reliability.

The application of Cliff's Delta in reliability-oriented sensitivity analysis allows for a more computationally efficient evaluation of structural reliability. This is particularly beneficial in engineering applications where finite element method (FEM) calculations and repeated simulations of R or F are computationally expensive.

While the current study provides a foundation, future work could focus on optimizing the computational algorithms for estimating Cliff's Delta, particularly for global sensitivity analysis of reliability.