Quantifying the Cross-Sectoral Intersecting Discrepancies Within Multiple Groups Using Latent Class Analysis Towards Fairness

1. Introduction

As AI systems become increasingly prevalent, ensuring fairness in their design and implementation is crucial [18]. AI fairness research spans various sectors, including healthcare [2,3,4], finance [33], and education [8]. However, studies on cross-sectoral intersecting fairness in AI are limited. Stiglitz [27] highlights that economic inequalities extend beyond income and wealth, affecting sectors such as education and health, implying correlated inequalities. At the same time, the “Leave No One Behind” (LNOB) principle urges addressing multiple, intersecting inequalities that harm individuals’ rights [29]. Therefore, we believe that research exploring joint inequalities across sectors is necessary to comprehensively understand overall inequality and gain insights into unfairness.

Bias, Inequality, Fairness Many obstacles to accessing services, resources, and opportunities result from discriminatory laws, policies, and societal norms that marginalise specific groups [29]. Recent studies highlight concerns that AI-supported decision-making systems may also be influenced by biases [16], which can unfairly impact vulnerable groups, such as ethnic minorities, emphasising the need for research on AI system fairness [20]. A primary obstacle in advancing and implementing fair AI systems is the presence of bias [9]. In AI, bias can originate from various sources, including data collection, algorithmic design, and user interaction, as illustrated in Figure 1.

Most AI systems heavily rely on data for their operations. This close connection means that any biases in the training data can be embedded into the algorithms, leading to biased predictions. Even if the data itself is not inherently biased, algorithms can still exhibit biased behaviour due to inappropriate design choices. These biased outcomes can influence AI systems in real-world applications, creating a feedback loop where biased data from user interactions further trains and reinforces biased algorithms, resulting in a vicious cycle [18].

From our perspective, bias stemming from data plays a crucial role in achieving fairness, as it can trigger a cascade of other biases, exacerbating fairness issues. Additionally, data bias may reflect various forms of social inequality in the real world. Inequality refers to disparities in opportunities and rewards based on different social positions or statuses within a group or society [24]. Inequality is considered unfair due to its causes, such as discrimination or failures in providing equal opportunities, and its consequences, such as resulting in objectionable disparities in status or power [24]. As previously mentioned, cross-sectoral intersecting inequality is crucial. Therefore, quantifying these disparities within the data is essential for understanding inequality and bias, ultimately leading to the development of fairer AI systems.

Discrepancy According to the LNOB principle of equal opportunities, everyone should ideally have equal access to public services and resources without discrepancies. We prefer the term “discrepancy” over “disparity” or “difference” because it suggests an unexpected difference. Discrepancies in data may indicate unfairness, biases, and inequalities, given that individuals should ideally be treated equally. It is worth noting that bias can arise from various inequalities, though not all inequalities are biases [24]. However, in the context of LNOB, for this study, “discrepancy” is more appropriate when discussing equal opportunity, as it can be seen as a specific type of inequality causing unfairness.

Figure 2 illustrates the correlations between fairness, bias, inequality, and discrepancy. Essentially, fairness is indirectly linked with discrepancy, and discrepancy can contribute to unfairness. The difference between fairness and bias is that the former can be viewed as a technical issue, while the latter can be viewed as a social and ethical issue [9]. Furthermore, bias is a problem caused by historical and current social inequality [6], and inequality can manifest as discrepancies. Figure 2 starts with discrepancy and moves through inequality and bias to fairness. Therefore, our research focuses on quantifying cross-sectoral intersecting discrepancies between different groups, uncovering insights or patterns related to inequality, bias, and AI fairness.

Background and Motivation Currently, there is limited research focusing on quantifying discrepancies. Most recent research on quantifying bias and/or inequality primarily revolves around resource allocation strategies and generally relies on objective data (e.g., [32]). However, these approaches have limitations and face challenges in effectively assessing and measuring biases in datasets unrelated to resource allocation. For example, in social science, much data is collected through questionnaires, which often include binary, categorical, or ordinal data types related to subjective responses and user experiences. These questionnaires may cover various aspects, resulting in intersecting, cross-sectoral, and high-dimensional data. Analysing data based on no more than two dimensions or sectors may overlook important information or patterns. Therefore, we believe that quantifying cross-sectoral intersecting discrepancies is valuable, as it can provide comprehensive insights and joint information.

The Anonymousproject1 aims to establish safer online environments for minority ethnic (ME) populations in the UK. Its survey questionnaire covers five key aspects: demography, energy, housing, health, and online services. Notably, the data collected in this context does not directly pertain to resource allocation, making it challenging to explicitly define and detect bias within the data using current methods, despite the presence of discrepancies. These discrepancies may arise from various factors, including culture, user experience, and discrimination, potentially contributing to bias or unfairness. This research is mainly motivated by the Anonymousproject, so the datasets we used primarily cover the health, energy, and housing sectors, with our research targeting ME groups.

In our preliminary research (see Appendix A) in England, based on a Anonymousproject survey question regarding health and digital services 2, we observed a notable discrepancy in the Chinese group: 30.16% lacked English proficiency and 26.98% struggled to use the online system, while most other ethnic groups reported no concerns. These discrepancies, stemming from cultural differences, user experiences, or discrimination, contribute to inequality and may affect AI fairness. For instance, an AI system might wrongly assume Chinese individuals need more English support, leaving other ME groups with less assistance. Ideally, the percentages should be similar across ethnic groups in the LNOB context, indicating equal treatment or experience. However, investigating multiple and cross-sectoral questions simultaneously is challenging. For instance, the Anonymous project’s data contains similar questions for the energy and housing sectors, and current methods struggle to analyse these sectors jointly. Therefore, we propose an approach to quantify intersecting and cross-sectoral discrepancies for multiple ethnic groups by leveraging latent class analysis (LCA) [19].

LCA is a popular method in social science [5] because it identifies latent groups within a population based on observed characteristics or behaviours. LCA offers a flexible framework for exploring social phenomena and integrating with other analytical techniques. In this research, we use LCA to cluster intersecting and cross-sectoral data, encompassing questions across the health, energy, and housing sectors. This method enables us to derive latent classes and outcomes, with each class describing a distinct cross-sectoral user profile. This approach moves beyond defining user classes solely based on individual questions. More details of our proposed approach are presented in Section 2, with experiments and results reported in Section 4.

The main contributions of this research are as follows: (1) we propose a novel and generic approach to quantify intersecting and cross-sectoral discrepancies between user-defined groups; (2) our findings reveal that ME groups cannot be treated as a homogeneous group, as varying discrepancies exist among them; and (3) we demonstrate how the proposed approach can be used to provide insights to AI fairness.

2. Quantifying the Cross-Sectoral Intersecting Discrepancies

The overall workflow of the proposed approach is shown in Figure 3, using a binary-encoded survey data as an example. It is noted that our approach is not limited to this specific format and can be applied to a wide range of similar problems. We will present more experiments with other datasets in Section 4 to further validate and showcase the features of our approach.

In Figure 3, Stage (1) shows the binary-encoded data D, where $\{Q_1,Q_2,...\}$ represents the selected questions from the survey data, covering user experiences across sectors. Meanwhile, $\{u_1,u_2,u_3,...\}$ represents the collection of survey respondents. Here, q represents the response options, for example, $q_{2,1}$ denotes the first option for $Q_2$. A `1` indicates a selected option, while `0’ indicates it was not selected. For other datasets, the encoding method should be chosen based on the data format and type. The blue arrow in Figure 3 illustrates the LCA process 3, which includes hyperparameter selection and model fitting.

The advantage of using LCA is straightforward: it considers the joint probability distribution of all variables. This means potential inequalities or discrepancies can be analysed jointly. Once we obtain the distributions of latent classes $\{c_1,c_2,c_3,...\}$ over user-defined groups $\{e_1,e_2,e_3,...\}$ (as shown in Figure 3 Stage (2)), we can calculate the discrepancy $\Delta$.

Quantification of Discrepancy Let us denote the size of the dataset as N, where $i\in 1,2,\cdots ,N$ represents an individual sample. Concurrently, let $c\in C$ denote a latent class, with the total number of classes being $\left|C\right|$, and let $N_c$ represent the count of samples classified into latent class c. To quantify the discrepancies, it is necessary to establish a grouping variable G, which can be defined based on factors such as ethnicity, age, or income level. Here, $\left|G\right|$ denotes the total number of user-defined groups, $e\in G$ represents one specific group within this set, $N^e$ denotes the number of individuals from group e, and $N_c^e$ denotes the number of individuals from group e assigned to the class c.

To initiate the quantification process, the proportions r of samples from each user-defined group within each latent class need to be calculated, as detailed in Stage (3) in Figure 3. This calculation can be performed using $r_c^e=N_c^e/N^e,\forall e\in G,c\in C$. The reason for calculating r is that user-defined groups may have different numbers of samples; therefore, using percentages for subsequent analyses ensures fairness and consistency.

Subsequently, we can derive a matrix of results characterised by dimensions $\left|G\right|\times \left|C\right|$, as shown in Figure 3 (3). Within this matrix, each row corresponds to the proportions of samples from a specific group within each latent class. It is important to note that in this context, each latent class effectively represents an individual user profile and can be viewed as a distinctive feature. Consequently, each row within the matrix may be employed as a feature vector denoted as $R_e$, serving as a representation of a specific group within the feature space.

In the assessment of discrepancy between two vectors, various methods may be employed, including the Euclidean distance, Kullback-Leibler Divergence, Earth Mover’s Distance, and Manhattan Distance, among others. In our approach, we propose the utilisation of Cosine Similarity to calculate the discrepancy, which is defined as $\Delta =1-cos\left(\theta \right)=1-\frac{\mathbf{A}·\mathbf{B}}{\parallel \mathbf{A}\parallel \parallel \mathbf{B}\parallel }$. Finally, we can iteratively calculate $\Delta$ between any pairs of vectors R and obtain the discrepancy matrix S (as shown in Stage (4) of Figure 3). The AVG column in Stage (4) contains the mean discrepancy values for e, which can be viewed an approximation for how each e is different from others.

We suggest the use of Cosine Similarity due to its inherent characteristics, including a natural value range spanning from 0 to 1. Importantly, it does not necessitate additional normalisation procedures. This metric, possessing with a fixed value range, enhances comparability and offers support to subsequent AI fairness research. The proposed approach is summarised in Algorithm 1.

Algorithm 1 Quantifying the Intersecting Discrepancies within Multiple Groups

1: Input: D and G$▹\mathrm{G}\mathrm{denotes}\mathrm{a}\mathrm{set}\mathrm{of}\mathrm{user}-\mathrm{defined}\mathrm{groups}$ 2: Initialise M$▹\mathrm{Create}\mathrm{LCA}\mathrm{model}$ 3: Estimate M based on D 4: for  e  in  G  do 5:     for c inC do 6:         $r_c^e=N_c^e/N^e$ 7:     end for 8: end for 9: for  e  in  G  do 10:     for $e^{\prime }$ inG do 11:         ${\Delta }_{e{e}^{\prime }}=1-\frac{{\mathbf{R}}_{\mathbf{e}}·{\mathbf{R}}_{{\mathbf{e}}^{\prime }}}{\parallel {\mathbf{R}}_{\mathbf{e}}\parallel \parallel {\mathbf{R}}_{{\mathbf{e}}^{\prime }}\parallel }$$▹\mathrm{Pair}-\mathrm{wise}\mathrm{Calculation}$ 12:     end for 13: end for 14: Output: Discrepancy matrix S with size $\left|G\right|\times \left|G\right|$

3. Related Work

Quantifying and improving AI fairness As AI technologies are used more and more frequently in real life, people’s concerns about the ethics and fairness of AI have always existed, especially when AI is increasingly used in problems with sensitive data [28]. Morley et al. [20] and Garattini et al. [10] noticed that an algorithm “learns” to prioritise patients it predicts to have better outcomes for a particular disease. And they also noticed that AI models have discriminatory potential when facing ME groups on health. Therefore, people are paying more and more attention on the impact and mitigating methods of AI bias.

Wu et al. [32] proposes the allocation-deterioration framework for detecting and quantifying health inequalities induced by AI models. This framework quantifies inequalities as the area between two allocation-deterioration curves. They conducted experiments on synthetic datasets and real-world ICU datasets to assess the framework’s performance and applied the framework to the ICU dataset and quantified the unfairness of AI algorithms between White and Non-White patients. So et al. [26] explores the limitations of fairness in machine learning and proposes a reparative approach to address historical housing discrimination in the US. In that work, they used contemporary mortgage data and historical census data to conduct case studies to demonstrate the impact of historical discrimination on wealth accumulation and estimate housing compensation costs. They then proposed a remediation framework that includes analysing historical biases, intervening in algorithmic systems, and developing machine learning processes that reduce correct historical harms.

Latent Class Analysis (LCA) is a statistical method based on mixture models and often used to detect potential or unobserved heterogeneity in samples [13]. By analysing response patterns of observed variables, LCA can identify potential subgroups within a sample set [21]. The basic idea of LCA is that some parameters of a postulated statistical model differ across unobserved subgroups, forming the categories of a categorical latent variable [30]. In 1950, Lazarsfeld [14] introduced LCA as a means of constructing typologies or clusters using dichotomous observed variables. Over two decades later, Goodman [11] enhanced the model’s practical applicability by devising an algorithm for obtaining maximum likelihood estimates of its parameters. Since then, many new frameworks have been proposed, including models with continuous covariates, local dependencies, ordinal variables, multiple latent variables, and repeated measures [30].

Because LCA is a person-centered mixture model, it is widely used in sociology and statistics to interpret and identify different subgroups in a population that often share certain external characteristics from data [31]. However, in social sciences, LCA is used in cross-sectional and longitudinal studies. For example, in relevant studies in psychology [17], social sciences [1], and epidemiology [25], mixed models and LCA can be used to establish probabilistic diagnoses when no suitable gold standard is available [19].

In [17], the relationship between cyberbullying and social anxiety among Hispanic adolescents was explored. The sample consisted of 1,412 Spanish secondary school students aged 12 to 18 years. There were significant differences in cyberbullying patterns across all social anxiety subscales after applying LCA. Compared with other profiles, students with higher cyberbullying traits scored higher on social avoidance and distress in social situations, as well as lower levels of fear of negative evaluation and distress in new situations. Researchers in [1] developed a tool, using LCA, to characterise energy poverty without the need to arbitrarily define binary cutoffs. The authors highlight the need for a multidimensional approach to measuring energy poverty and discuss the challenges of identifying vulnerable consumers. The research in [25] aimed to identify subgroups in COVID-19-related acute respiratory distress syndrome (ARDS) and compare them with previously described ARDS subphenotypes by using LCA. The study found that there were two COVID-19-related ARDS subgroups with differential outcomes, similar to previously described ARDS subphenotypes.

4. Experiments

4.1. The Anonymous Project

Within the Anonymous project4, a systematic online survey was conducted to explore the experiences of individuals from ME groups in the UK, focusing on digitalised health, energy, and housing aspects. To examine cross-sectoral intersecting discrepancies among the seven ethnic groups (as shown in Table 1), we selected three questions from the Anonymoussurvey data related to these sectors. The answers from ME participants formed the dataset for this experiment. For England and Scotland, we have 594 and 284 samples, respectively. Due to varying sample sizes across ethnicities, we calculated discrepancies separately for each region.

Selecting the number of latent classes in LCA requires presetting. To address this, we conducted hyperparameter optimisation for each experiment to find the elbow point, applying this optimisation to all subsequent experiments.

The discrepancy results for England are presented in the left part of Table 1. The table shows that the Chinese group has the largest average (AVG) discrepancy value compared to other groups. Meanwhile, the Indian group exhibits the smallest discrepancy with the Bangladeshi group and also shows similarity to the Pakistani group. This is likely due to their close geographical locations and similar cultural backgrounds and lifestyles.

The right part of Table 1 presents the results for Scotland, showing similar outcomes: the Chinese group is distinct from others, while the Bangladeshi group is similar to the Pakistani group. The differences between England and Scotland may be attributed to their different policies and circumstances. We hypothesise that the primary reason for the Chinese group standing out is a lack of English proficiency. This is supported by our preliminary research mentioned in Section 1, which shows a significant number of Chinese participants expressing this concern. Additionally, BBC News [23] reports that the Chinese community experiences some of the highest rates of racism among all ethnic groups in the UK. Our discrepancy values may help explain this, as the Chinese group shows different experiences in digitalised online services.

4.2. EVENS

The Centre on the Dynamics of Ethnicity (CoDE), funded by the Economic and Social Research Council (ESRC), conducted “The COVID Race Inequalities Programme”. As part of this project, CoDE carried out the Evidence for Equality National Survey (EVENS)5, which documents the lives of ethnic and religious minorities in Britain during the coronavirus pandemic. The EVENS dataset comprises 14,215 data points and categorizes participants into 18 different ethnic minority groups. To facilitate a comparative analysis with the Anonymousproject’s experiment, we focused on the same seven ethnic groups from Anonymousproject, resulting in a filtered dataset of 4,348 participants from England and 253 participants from Scotland. We excluded entries with missing values to ensure the robustness of our analysis.

It is important to note that the EVENS dataset uses a different definition for mixed or multiple ethnic groups compared to Anonymousproject. To avoid inconsistencies that could affect the final analysis, we excluded mixed or multiple ethnic groups from the EVENS calculations. Due to the different questions in the EVENS and Anonymousproject surveys, we selected three types of cross-sectoral questions from EVENS for analysis: housing, experiences of harassment, and financial situation. These questions were chosen to provide a broad view of participants’ living conditions, social experiences, and economic status during the pandemic.

We applied LCA separately to the data from England and Scotland to uncover patterns and discrepancies within and between these regions. This approach allows us to identify distinct subgroups within the ethnic communities based on their responses to the selected questions, providing deeper insights into the intersectional and cross-sectoral experiences of these groups.

Seeing the discrepancies in the EVENS England data, shown in the left part of Table 2, it is clear that all discrepancy values are relatively small compared to the Anonymousproject results. Notably, there are large discrepancies between the Indian and Bangladeshi groups, and between the Indian and Chinese groups. This differs slightly from the conclusions of the Anonymousproject’s experiment, likely due to the different types of cross-sectoral questions selected. In the AVG column, the Bangladeshi group has the highest values, indicating they experienced COVID-19 differently. The Census 2021 reported that COVID-19 mortality rates were highest for the Bangladeshi group, for both males and females [7], supporting our findings. The right part of Table 2 describes the discrepancies for the EVENS Scotland data. There are significant discrepancies between the Caribbean and other ethnic groups, likely related to the unique background of the Caribbean group.

4.3. Census 2021 (England and Wales)

To further test our approach, we applied it to the Census 2021 dataset [22], which gathers information on individuals and households in England and Wales every decade. These data help plan and finance essential local services. We compared our results with the UK deprivation indices data from 2019 [12], which classify relative deprivation in small areas. We hypothesised that discrepancy values should correlate with the deprivation indices, reflecting discrepancies across energy, health, housing, and socioeconomic sectors. The key difference is that our discrepancy values are data-driven, while deprivation indices are based on human-centred assessments, suggesting that our approach is complementary.

For our experiments, we selected four cross-sectoral questions from the census related to energy (type of central heating), health (general health), housing (occupancy rating for bedrooms), and socioeconomic status (household deprivation). Note that the socioeconomic data in Census 2021 differ from the 2019 deprivation indices due to different definitions and coverage [12,22].

Additionally, for Census 2021, we selected Lower Layer Super Output Areas (LSOAs) [22] as samples instead of individuals, as individual data were not accessible. After cleaning the data and removing unmatched LSOAs, we had 31,810 LSOAs in total. Unmatched LSOAs, which appear only in either Census 2021 or Deprivation 2019, were removed. In the Deprivation 2019 dataset, each LSOA is labelled with a deprivation level from 1 to 10 (1 being the most deprived). As our samples are LSOAs, we quantified discrepancies between different LSOAs. Since the raw data does not include group attributes, we classified LSOAs into five groups based on the percentage of the population from ME groups: [0%, 20%), [20%, 40%), [40%, 60%), [60%, 80%), and [80%, 100%].

The proposed approach quantifies the discrepancies between the defined ME population-related groups based on the selected Census 2021 data. The results are shown in the left part of Table 3. It is evident that the discrepancies between LSOAs increase as differences in ME population percentages increase, indicating significant disparities in living conditions for ME individuals across different LSOAs, particularly in terms of energy, housing, and health aspects. Notably, the 0%-20% group shows the largest AVG discrepancy compared to other groups, suggesting that White individuals in those LSOAs experience significantly different living conditions. Since the 0%-20% group constitutes a large portion of the UK (see Appendix B), this finding suggests potential unequal treatment and possible neglect of other LSOAs. Additionally, the 40%-60% group has the smallest AVG discrepancy value, likely due to its intermediate position among the ME groups, sharing characteristics with the 0%-20% and 20%-40% groups, as well as 60%-80% and 80%-100% groups.

Correlation Analysis Furthermore, based on the deprivation indices from Deprivation 2019 [12] and the defined groups, we calculated the percentages of LSOAs in each deprivation-labelled group across various ME population groups. The results are shown in Appendix B, and we treated each row as a feature vector representing one group of LSOAs. We then iteratively calculated the deprivation discrepancies for each pair of rows, with the results presented in the right part of Table 3. We observed similar patterns (color change) to those in the left part of Table 3, which can verify the reliability of our proposed approach.

To statistically verify our proposed approach, we ran Pearson and Spearman row-wise correlation analyses for Census 2021 discrepancies (the left part of Table 3) and Deprivation 2019 discrepancies (the right part of Table 3). The detailed results are shown in Appendix B. All rows exhibit very strong correlations, implying that our approach can draw conclusions very similar to those of experts. Furthermore, we also flattened both matrices to run a one-time correlation analysis. The Pearson correlation coefficient is 0.9797 with a p-value of 1.4437e-17, and the Spearman correlation coefficient is 0.9872 with a p-value of 7.436e-20. Both p-values are far less than 0.001, indicating a strong correlation.

Discrepancy for AI fairness Now, we will show how discrepancies relate to potential AI bias. We selected logistic regression to classify deprivation indices for LSOAs using the Census 2021 data from previous experiments. To simplify the classification task, we redefined the deprivation indices as deprived (indices 1-5, labelled as 0) and not deprived (indices 6-10, labelled as 1). We split the dataset into training and validation sets in an 8:2 ratio. The prediction accuracy on the Census 2021 dataset, with an overall accuracy of 90.35% and a standard deviation (STD) of 4.20 across five group results, is shown in the Census column of Table 4. Meanwhile, the accuracy for each group is displayed on the left in Figure 4. We noticed that the accuracy varies across different groups, with the 0-20% group showing 100% accuracy. In the context of LNOB and AI fairness, we consider this biased and problematic. Additionally, in this research, the STD is considered an important indicator of fairness; smaller STD values imply that the model treats each group more equally. In the following paragraph, we will discuss how the discrepancy relates to AI fairness based on two undersampled datasets.

For the Census 2021 data, we noticed two data imbalance issues likely affecting accuracy. Firstly, the 0-20% group constitutes 82.59% of the LSOA samples and has the largest AVG discrepancy and relatively low accuracy, potentially indicating distinct features compared to other groups. Generally, in machine learning, imbalanced data can negatively impact the majority, leading to issues like overfitting. To address this, we used random undersampling [15], resampling all classes except the minority 80-100% group, which had 125 samples. After undersampling, all groups had an equal number of samples, and we split them into training and test sets with a ratio of 8:2. This method aimed to observe changes in discrepancies, accuracy, and STD to gain insights into bias. The results are shown in the middle of Figure 4 and in the Undersampled Census (ME) column of Table 4.

We found that all AVG discrepancies increased along with the STD (from 4.2 to 4.83), and the prediction accuracy decreased for three groups. This may indicate that our approach efficiently detects data discrepancies that can exacerbate bias issues. Notably, the 40-60% group showed the most significant increase in discrepancy, accompanied by a dramatic decrease in accuracy. We believe that this phenomenon is due not only to the increase in discrepancies but also to the reduced data size, which may not provide enough data for effective model training. When samples from different groups are far apart, the model may require more data to train effectively; otherwise, it may focus on some subareas of the feature space. It is worth noting that while the overall accuracy increases, the accuracy of most groups decreases due to a statistical artifact related to changes in test data size. In this research, our primary focus is on the STD and discrepancies.

Furthermore, another data imbalance issue may be related to deprivation labels. Specifically, in the original Census 2021 data, all groups except the 0-20% group have more data labelled as 0 than 1. Additionally, the ratio of data labelled as 0 to 1 increases with the percentage of the ME population. For example, in the 40-60% group, the number of samples for 0 and 1 labels are 1,014 and 215, respectively, and 537 and 54 for the 60-80% group. We believe this may contribute to bias issues. Therefore, we conducted undersampling on the original Census 2021 data, targeting the labels. After undersampling, the dataset had an equal number of samples for 0 and 1 labels. Then, we split the dataset into training and test sets following the same rule. The results are shown on the right of Figure 4 and in the Undersampled Census (deprivation) column of Table 4.

Overall, we found that all AVG discrepancies decreased, indicating that samples from different groups became closer to each other. This suggests that the model is likely to treat them more similarly in the feature space. On the right of Figure 4, we observed that accuracy across different groups became more similar compared to the other two figures (left and middle), supported by the STD dramatically decreasing to 1.82. In the context of LNOB, the model is now less biased. Meanwhile, the overall accuracy is better than the original one. In general, this set of experiments demonstrated that data discrepancies obtained from our proposed approach can potentially indicate the degree to which the model treats each group equally.

5. Conclusion and Limitations

In conclusion, the issue of AI fairness is of paramount importance and warrants attention from all stakeholders. In our research, we addressed this challenge by focusing on quantifying the discrepancies present in data, recognising that AI models heavily rely on data for their performance. Our proposed data-driven approach is aligned with the LNOB initiative, as it aids in discovering and addressing discrepancies between user-defined groups, thus contributing to efforts to mitigate inequality. Moreover, we believe that our proposed approach holds promise for applications across a broad spectrum of tasks, offering insights to develop fair AI models. Through testing on three datasets, we have demonstrated the efficacy and informativeness of our approach, yielding satisfactory results. Our proposed approach can be considered as an approximation of bias, as selecting different parameters for LCA may yield slightly varying results, to address this we have done hyperparameter optimisation.

In summary, our research represents a significant step towards promoting fairness in AI and offers an innovative avenue for social science research. By highlighting data-driven approaches and their alignment with broader societal initiatives, we aim to foster a more equitable and inclusive landscape for AI development and deployment.