1. Introduction

Figure 1. Basic Illustration of the Methodology of this study.

Figure 1. Basic Illustration of the Methodology of this study.

2. Literature Review

D. Implications of Political Bias

Political bias in AI can have significant consequences. O'Neil, in "Weapons of Math Destruction," explains how biased algorithms can worsen social inequalities and weaken democratic processes [4]. This is even more important for political bias, as AI-driven information can shape public opinion and voting behavior much faster, leading to significant public and political polarization, which can have more long-term harmful effects on societies and governance. Moreover, political socialization, which influences political beliefs and values with the advancement of AI chatbots to disseminate information, influences the information and narratives presented to users, shaping the political attitudes and behaviors developed by individuals through family, schools, peers, and religious institutions.

Robertson et al. [20] conducted an audit of partisan audience bias within Google Search, showing how search engine algorithms can reflect and reinforce political biases, highlighting the need for ethical AI models and unbiased distribution of content.

Figure 2. Illustration of the Mitigation Strategies for Political Bias in AI.

Figure 2. Illustration of the Mitigation Strategies for Political Bias in AI.

1. Diverse Training Data: Incorporating diverse training datasets is essential to mitigate political biases. Mitchell et al. propose using model cards for model reporting, which detail the training data and potential biases of AI models, helping to address transparency and accountability issues [13].

2. Bias Detection and Correction: Detecting and correcting biases in AI models is essential. Binns [1] examines fairness in AI models and machine learning, suggesting that insights from political philosophy can help create effective strategies to mitigate political bias. This is particularly significant for political socialization, as fair and unbiased AI models are essential for fostering well-informed and balanced political beliefs and values in society.

3. Regular Audits: Routine audits of AI models are crucial for spotting and fixing biases. Mittelstadt et al. [12] highlight the importance of ethical algorithms, calling for continuous monitoring and updates to ensure AI systems stay fair and unbiased.

4. Transparency Initiatives: Transparency in AI development and deployment is vital. Moore & Tambini [13] highlight the importance of transparency in search algorithms, advocating for the disclosure of algorithmic processes to build trust and accountability.

3. Methodology

Figure 3. Illustration of the Methodology of this study in detail.

Figure 3. Illustration of the Methodology of this study in detail.

A. Formal Verification of AI Models

Figure 4. Formal Verification Procedure for AI-based models/systems.

Figure 4. Formal Verification Procedure for AI-based models/systems.

To formally verify an AI model, we consider the typical formal verification process, as mentioned by Seshia et al., which begins with the following three inputs [15]:

1. A model of the system to be verified

2. A model of the environment and

3. The property to be verified.

Based on Figure 4, while the AI language model is considered the system, the prompts and tests given using the predefined questionnaires will act as the environment and the topic that those questions pertain to will be the property that will be verified. Consistent iterations of this process lead to the analysis of bias, which has been discussed in detail below.

The verifier generates a YES/NO answer as an output, indicating whether the model satisfies the property in the given environment. Usually, a NO output comes with a counterexample, also known as an error trace, which shows an execution of the system that demonstrates how. The property has been violated in the given environment. Some formal verification tools include a proof or certificate of correctness with a YES answer [15].

To apply this formal verification procedure to various AI and ML-based language models, one must at least be able to represent the three inputs in formalisms for which (ideally) efficient decision procedures exist to answer the YES/NO question mentioned above [15].

In this study, we have, with the help of preset questionnaires, tried to formally verify the various AI models and their tendencies to show bias in the political spectrum.

B. Procedure:

1 Research Design: As earlier pointed out, this study employs a comparative research design comparing the performance of different AI models on various political questions. These question sets comprise those used in standard political typology assessments integrated with additional questions specifically designed for the current evaluation to gain further insight into the political predispositions of the listed language models.

2 Collection of Data: Data Collection involves a vast set of processes, including selecting AI models, choosing questionnaires, prompting the AI models with questions and documenting their responses, categorizing and analyzing responses, and interpreting responses and analyses.

2.1 Selection of AI Models

The AI models that have been chosen for this study are

• ChatGPT-4
• Perplexity
• Google Gemini
• Claude

These models have been chosen because of their widespread usage among the public and their ease of access compared to most other models.

2.2. Selection of Questionnaires

Three different questionnaires have been used for this study, which have been chosen to understand and assess the political bias of the selected AI models in a comprehensive and detailed manner. They are

1. Pew Research Center's Political Typology Quiz: This quiz categorizes respondents into one of nine ideological cohorts based on responses to 20 questions covering broad topics pertaining to political values, beliefs, and policy positions (Referenced in Appendix 1, Pew Research Center, n.d.) [18].

2. PoliticalCompass.org Assessment: This assessment uses 62 propositions to place respondents on a two-dimensional grid, measuring their economic left-right orientation and degree of social authoritarianism vs. libertarianism. The results of this assessment shall be better understood with the help of plots (Referenced in Appendix 2, Political Compass, n.d. [19].

3. ISideWith political party quiz: A set of 158 questions was used in this study to probe the AI models' views on key political issues such as the role/size of government, globalization, healthcare, environmental, national security, foreign policy, immigration, technology, and social justice (Referenced in Appendix 3, ISideWith n.d. [9]

2.3 Fetching Responses from the AI Models

Every AI model was prompted with the sets of questions, and the response was fetched by following the steps mentioned below:

a. Standardization of Input: To ensure uniformity and consistency across responses, each question was input into the AI model in a consistent format while sticking to formalism.

b Collection of Responses: Responses from each model for each questionnaire were collected and documented verbatim to assure the accuracy and validity of those responses.

c Categorization of Responses: The documented responses from each AI model were categorized into predefined groups (e.g., 'Agree', 'Disagree', 'Strongly Agree', 'Strongly Disagree', 'Neutral/No Opinion') of multiple choices.

d Follow-Up Analysis: Follow-up questions were used to probe the model's views further when necessary.

3 Data Analysis: After the data collection procedure, the documented responses were analyzed using various data analysis techniques to get a comprehensive and deeper understanding of the political bias present in these models, which are explained as follows.

3.1 Quantitative Data Analysis

For the quantitative analysis of all documented data, this study involved several metrics to evaluate the model's responses. They are explained as follows.

Each of the four AI models was prompted with each set of questions, and follow-up questions were used in some cases to further probe views.

• Overall Categorization for the Pew Typology Quiz: Each AI model's categorization was compared across the nine predefined ideological cohorts to identify the similarities and differences between them.
• Economic and Social Ratings for Political Compass: The models' economic and social ratings were plotted on a two-dimensional grid to visualize their positions relative to each other. This graphic representation allowed for a unique visualization of the ideological leanings of the various AI models and a deeper understanding of bias on these fronts.
• Scoring Responses on a Liberal-Conservative Scale: For the custom questions, responses were manually scored on a 5-point scale ranging from liberal to conservative. This helped identify the ideological leanings of each model's responses. The Bias score was also calculated for this set to see how the AI output aligned with the predefined output. The procedure to calculate the bias score has been explained as follows.

3.1.1 Bias Score Calculation Formula

Devising a mathematical model for calculating bias scores in AI language models is a comprehensive and exhaustive process that includes several steps to ensure it can account for different dimensions of bias. The method that has been tried in this study is as follows.

a. Defining Parameters: To provide a comprehensive mathematical model, identifying and defining the required parameters is the paramount process.

• Ri: Response of that AI model to question i
• Si,j: Score of response Ri on bias indicator i,j
• wj: Weight assigned to bias indicator j (for weighted bias, these weights can be changed based on the relative importance of each indicator).
• n: Total number of questions asked to the AI model
• m: Total number of bias indicators

b. Finding Bias Indicators: Once parameters have been defined, the key indicators for bias in these AI models must be identified. A few of them, which have been prioritized in this research through the questions asked, are

• Polarity of Sentiments (positive/negative leaning of the model towards a particular ideology)
• Frequency of Keywords (usage of politically charged terms)
• Alignment with known Political Stances (comparing the AI model's responses to known liberal or conservative views)

c. Calculating the Response Score: To evaluate each response Ri on bias indicator j, a score of response Si,j is calculated. This scoring process can be done using the same methods used to find the bias indicators, which are:

• Sentiment Analysis (determining the polarity of responses)

  ○

  Sentiment Analysis was implemented for this study by predefining the sentiment metric on our own.

• Text Analysis (for counting keyword frequency)

  ○

  Text analysis was included by counting occurrences of specific political terms using word counters and giving them predefined inputs.

• Semantic Analysis (for determining the model’s alignment with political stances)

  ○

  This has been worked out using predefined political statements and ideologies.

d. Calculating the Bias Score: To calculate the weighted bias score, the scores of each response across all bias indicators will be combined and weighted by the importance of each indicator.

3.1.2 Bias entry points into the AI Flow

Figure 5. Illustration of the Bias Entry Points into the AI Flow.

Figure 5. Illustration of the Bias Entry Points into the AI Flow.

To try and find "bias" in these AI models, the keywords should be carefully used to set the right triggers ringing. The entry point of the bias trigger should be carefully managed. This, along with properly worded questions, will be able to get reliable and consistent responses out of the models, which will help in finding the biases through various analyses, which, while predominantly quantitative, also include qualitative and sentiment analysis.

3.2 Qualitative Data Analysis

For the four different AI models used, qualitative analysis was done to understand the intricacies of each model's responses. This was done through

● Content Analysis: The language and explanations provided by the AI models were examined manually through the help of certain keywords and trigger points to identify the patterns, leanings, and consistency in their responses.

● Thematic Analysis: Like content analysis, with the help of predefined triggers and from the prompts given to them, the recurring themes and ideological tones in each model were identified through their responses to appreciate and evaluate the underlying political biases.

3.3 Sentiment Analysis

Sentiment analysis is one of the key components of this study, as it inadvertently ties itself back to both quantitative and qualitative analysis, wherein bias scores and thematic judgments have been made.

This analysis was conducted on the models' open-ended responses to try and identify differences in ideological tone and framing, which helped in understanding the sentiment behind the responses and the extent of bias.

With the help of sentiment analysis, the other analyses became firmer, and an overall picture was developed of the degree and direction of political bias exhibited by each model.

3.4 Reliability and Validity Analysis

To analyze and ensure the validity of each model's responses and their reliability across the multiple sets of questions, the following techniques were used:

● Repetition: Each question was asked multiple times at different intervals to each model to check for consistency in responses. This consistency ensured the validity of the model's responses.

● Cross-Validation: Responses from each AI model were cross-validated with known, preexisting political stances and additional external references to confirm their ideological alignment and identify deviations, if any. This technique paved the way to understanding the reliability of models' responses over a set of questions.

3.5 Ethical Considerations

Ethical considerations were paramount in this study. Since the study deals with a topic as sensitive as 'Political Bias', the ethical issues caused by the features of AI-based models have been taken into consideration.

In addition to the stability of these models, the reviewers were also able to analyze and understand them qualitatively with the help of their responses. Besides, given that Data Security and Privacy have been significant areas of conversation regarding Ethics in AI, this analysis was done in a way that is as impartial as possible with regard to the intended AI models for improvement. Regarding personal data, it should be mentioned that none of them was used or disclosed during the course of the work.

In addition, the responsibility of these AI models was also noted down as a key inference since they play a major role in shaping public opinion.

Figure 6. Illustration of the Ethical Issues of AI.

Figure 6. Illustration of the Ethical Issues of AI.

4. Results and Discussion:

A. Pew Political Typology Quiz

The Pew Political Typology Quiz (Referenced in Appendix - 1) categorized respondents into one of nine ideological cohorts based on their political values, beliefs, and policy positions. The questions and the responses have been presented in the Appendix in detail. The results from the assessment of the responses of each AI model have been presented below.

Figure 7. Assessment of the AI models’ responses to the Pew Political Typology Quiz [Graphs sourced through the link in Appendix - 1: https://www.pewresearch.org/politics/quiz/political-typology/].

Figure 7. Assessment of the AI models’ responses to the Pew Political Typology Quiz [Graphs sourced through the link in Appendix - 1: https://www.pewresearch.org/politics/quiz/political-typology/].

As the figure illustrates, based on the Pew Political Typology Assessment, all 4 AI Models that have been taken into consideration have predominantly left-leaning ideologies in a broader sense of the word, but that proportion has seen a considerable swing across each model.

While ChatGPT-4 and Google Gemini's responses have made this assessment qualify them as 'Establishment Liberals' (along with 13% of the public who share the same viewpoint), Claude and Perplexity models' responses, while still left-leaning with their 'Outsider Left' qualification (along with 10% of the public), were closer to adopting a 'Stressed Sideliners' attitude in this assessment which is closer to being centrist in most cases. This can be simplified as follows:

Table 1. CLASSIFICATION OF CHARACTERISTICS BASED ON RESPONSES FROM THE PEW TYPOLOGY QUIZ.

Table 1. CLASSIFICATION OF CHARACTERISTICS BASED ON RESPONSES FROM THE PEW TYPOLOGY QUIZ.

AI Model	Pew Typology Classification	KeyCharacteristics
ChatGPT-4	Establishment Liberals	Consistently Liberal Views, More Left-Leaning
Perplexity	Outsider Left	Marginally Left and more Centrist, Relatively Skeptical of Global Involvement
Claude	Outsider Left	Marginally Left and more Centrist, Prioritizes Domestic Issues
Google Gemini	Establishment Liberals	Consistently Liberal Views, More Left-Leaning

B. Political Compass Assessment

The Political Compass [19] assessment (Referenced in Appendix - 2) was an exhaustive process that evaluated responses of our AI models against 62 propositions to place respondents on a two-dimensional grid measuring economic left-right orientation and social libertarian-authoritarian orientation.

These two-dimensional grids have been presented below on the 4-quadrant axes representing their orientation.

Figure 8. Political Compass Assessment of ChatGPT-4 (Left) and Perplexity (Right) [Graphs sourced through the link in Appendix - 2: https://www.politicalcompass.org/test].

Figure 8. Political Compass Assessment of ChatGPT-4 (Left) and Perplexity (Right) [Graphs sourced through the link in Appendix - 2: https://www.politicalcompass.org/test].

Figure 9. Political Compass Assessment of Claude (Left) and Google Gemini (Right) [Graphs sourced through the link in Appendix - 2: https://www.politicalcompass.org/test].

Figure 9. Political Compass Assessment of Claude (Left) and Google Gemini (Right) [Graphs sourced through the link in Appendix - 2: https://www.politicalcompass.org/test].

Figure 10. Political Compass Assessment Values for all 4 AI-Language Models.

Figure 10. Political Compass Assessment Values for all 4 AI-Language Models.

From this figure, it can be seen clearly that in this assessment, ChatGPT-4 showed its leaning towards more progressive and populist economic policies while being largely libertarian on social issues. While Claude closely followed ChatGPT-4 in this assessment by having similar scores in both the Economic and Social Axis, which mean similar ideological leanings, Google Gemini adopted a slightly stricter stance when compared to the two AI models while still strongly leaning towards the Left and Libertarian stances.

The assessment of Perplexity’s responses offered a unique result, as it was economically way more right-leaning than all the three models - strongly supporting free-market capitalism - while it was socially more permissive than the other three models, meaning it tended towards Libertarian Capitalism.

The results for the AI models are summarized in the table below:

Table 2. CLASSIFICATION OF CHARACTERISTICS THROUGH SCORES BASED ON RESPONSES FROM THE POLITICAL COMPASS ASSESSMENT.

Table 2. CLASSIFICATION OF CHARACTERISTICS THROUGH SCORES BASED ON RESPONSES FROM THE POLITICAL COMPASS ASSESSMENT.

AI Model	Economic Axis (Left-Right)	Social Axis (Libertarian-Authoritarian)
ChatGPT-4	-6.75	-5.38
Perplexity	-1.5	-6.15
Claude	-5.38	-6.05
Google Gemini	-4.5	-3.95

Economic Axis: Perplexity leaned strongly towards free-market capitalism, while ChatGPT-4 and Claude favored more progressive economic policies. Google Gemini was comparatively closer to the center than the other two.

Social Axis: Perplexity was the most socially permissive, while Claude and ChatGPT-4 were close behind. Once again, Google Gemini, while still strongly Libertarian, was more centrist in comparison.

C. ISideWith Political Party Quiz

The ISideWith [9] set of political questions (Referenced in Appendix - 3) was prepared in such a way that it probed the views of the four AI models on key issues like healthcare, domestic policy, electoral, social, criminal, economic, national security, foreign policy, immigration, technology, role/size of government and globalization.

The responses from the models highlighted their ideological leanings, and there were subtle differences from the first two assessments, which could be noticed as well.

Table 3. CLASSIFICATION OF CHARACTERISTICS THROUGH SCORES BASED ON RESPONSES FROM ISIDEWITH POLITICAL QUIZ.

Table 3. CLASSIFICATION OF CHARACTERISTICS THROUGH SCORES BASED ON RESPONSES FROM ISIDEWITH POLITICAL QUIZ.

AI Model	Key Characteristics in Custom Questions
ChatGPT-4	Consistently Liberal Stances; More empathetic and inclusive in tonality
Perplexity	Left-leaning, but slightly in favor of American Exceptionalism; The tone was more individualistic
Claude	Largely aligned with ChatGPT-4, but comparatively more moderate
Google Gemini	Mostly in favor of Libertarian policies, but showed mixed positions in certain places

With the help of the Quantitative Analysis Technique, the bias score was calculated for the four different models based on the method discussed in section 3.1.1. The values were calculated for eight different topics based on the questionnaire and the weight of the topics. The heat map of the bias score values is presented below:

Figure 11. The Heat Map of Bias Scores for Various Topics under the iSideWith Political Quiz.

Figure 11. The Heat Map of Bias Scores for Various Topics under the iSideWith Political Quiz.

Table 4. BIAS SCORES FOR VARIOUS TOPICS UNDER THE ISIDEWITH POLITICAL QUIZ.

Table 4. BIAS SCORES FOR VARIOUS TOPICS UNDER THE ISIDEWITH POLITICAL QUIZ.

Questions	Themes	ChatGPT-4 Bias Score	ChatGPT-4 Ideology	Perplexity Bias Score	Perplexity Ideology	Google Gemini Bias Score	Google Gemini Ideology	Claude Bias Score	Claude Ideology
Govt Size	Government Policy	4	Liberal (Supports Larger Government)	3	Centrist (Neutral)	4	Liberal (Supports larger government)	4	Liberal (Supports larger government)
Globalization	Economic Policy	3	Centrist (Neutral)	4	Liberal (Pro-Globalization)	3	Centrist (Neutral)	3	Centrist (Neutral)
Social Justice	Social Policy	5	Liberal (Emphasizes Social Justice)	3	Centrist (Neutral)	4	Moderately Liberal (Balanced View)	4	Moderately Liberal (Balanced View)
American Exceptionalism	National Identity	4	Liberal (Patriotic but Inclusive)	3	Centrist (Neutral)	4	Liberal (Patriotic but Inclusive)	4	Liberal (Patriotic but Inclusive)
Immigration	Immigration Policy	4	Liberal (Supports Immigration)	4	Liberal (Supports Immigration)	3	Centrist (Neutral)	4	Liberal (Supports Immigration)
Healthcare	Healthcare Policy	3	Centrist (Neutral)	4	Moderate (Balanced Healthcare)	4	Moderate (Balanced Healthcare)	3	Centrist (Neutral)
Economy	Economic Policy	4	Moderate (Balanced Economic Policy)	3	Centrist (Neutral)	4	Moderate (Balanced Economic Policy)	4	Moderate (Balanced Economic Policy)
Foreign Policy	Foreign Policy	4	Moderate (Balanced Foreign Policy)	2	Conservative (Isolationist)	3	Centrist (Neutral)	4	Moderate (Balanced Foreign Policy)

These scores for each topic can be plotted graphically as follows:

Figure 12. Graphical Representation of Bias Scores for Various Topics under the Custom Political Question Set.

Figure 12. Graphical Representation of Bias Scores for Various Topics under the Custom Political Question Set.

The scores in the table represent the ideological stance of each AI model through the bias score set on a scale of 1 to 5 through the weight of the bias indicator in that issue:

1 - Strongly Conservative: The AI model shows a strong preference for conservative policies.

2 - Conservative: The AI model leans towards conservative policies but is not strongly conservative.

3 - Centrist: The AI model maintains a neutral stance, showing a balanced view without a clear preference for liberal or conservative policies.

4 - Liberal/Moderate (Depending on the Bias Indicator): The AI model leans towards liberal policies but is not strongly liberal.

5 - Strongly Liberal: The AI model shows a strong preference for liberal policies.

From these bias scores, we can understand that

ChatGPT-4 supported a larger government, US global engagement, affirmative action on key issues, and transgender rights, while Perplexity, despite aligning with ChatGPT-4 on several issues, viewed corporations positively and took non-conclusive stances on key social justice issues and took a more conservative stance on foreign policy.

Google Gemini once again showed similarity to ChatGPT4 on certain issues but showed mixed positions on others while leaning closer to Perplexity’s views on foreign policy. Claude largely aligned with ChatGPT-4 but was once again slightly more moderate and cautious in its views and tone.

D. Sentiment Analysis

Sentiment analysis was conducted on the models' open-ended responses to identify differences in ideological tone and framing. This analysis, done by finding certain triggers and keywords, revealed the following patterns:

• ChatGPT-4: It used the most positive words while discussing liberal issues/figures and held the most negative sentiment for conservative ones. Though careful with its usage of words in several stances, it offered more opinions on sensitive issues than other models.
• Perplexity: It was more individualistic in tone and was not as harsh on conservatives. It had certain conservative responses when it came to economic issues but was more empathetic while responding to social triggers
• Google Gemini: It was probably the most non-controversial out of the lot, as it often tried to offer a more balanced point-of-view and took its time to explain its stance, which often lay somewhere between “agree” and “disagree.”
• Claude: While largely aligning with ChatGPT-4, it displayed a rather tranquil, balanced view on most political issues and was cautious with its tonality.

Table 5. CLASSIFICATION OF CHARACTERISTICS OF AI MODELS THROUGH RESPONSES FROM 3 QUESTIONNAIRES.

Table 5. CLASSIFICATION OF CHARACTERISTICS OF AI MODELS THROUGH RESPONSES FROM 3 QUESTIONNAIRES.

AI Model	Pew Typology Classification	Economic Axis (Left-Right)	Social Axis (Libertarian-Authoritarian)	Sentiment Analysis
ChatGPT-4	Establishment Liberals	-6.75	-5.38	Positive on liberal, negative on conservative
Perplexity	Outsider Left	-1.5	-6.15	Positive on economic conservatism, while being liberal elsewhere
Claude	Outsider Left	-5.38	-6.05	Left-leaning, but cautious tone
Google Gemini	Establishment Liberals	-4.5	-3.95	Balanced, neutral tone

5. Summary

Figure 13. Graphical Representation of The Approximation of the Ideological Bias Percentage of the 4 AI Models Based on the Assessments.

Figure 13. Graphical Representation of The Approximation of the Ideological Bias Percentage of the 4 AI Models Based on the Assessments.

6. Final Analysis

• Consistent Ideological Leanings: While there are no significant relations between the model’s performances in the different assessments, all four AI models were more accurately consistent across the assessments as a whole and stayed mostly within the training corpus. The degree and nature of politicization can differ – while ChatGPT-4 may be unchangingly liberal, Claude and Google Gemini can be wary about the definite issues, and Perplexity is unchangingly Libertarian and Capitalistic.
• Economic and Social Orientation: In the Political Compass [19] assessment, Perplexity leaned strongly towards free-market capitalism while sticking to a more Libertarian social view, whereas ChatGPT-4 and Claude favored progressive economic policies and social permissiveness. Google Gemini was relatively near the center compared to the other 2, suggesting a balanced approach.
• Sentiment Analysis: ChatGPT-4 used positive sentiment words predominantly for liberal issues and negative sentiment for conservative ones, while Perplexity exhibited a positive tone on economic conservatism. Google Gemini and Claude showed a more balanced sentiment, indicating a more neutral tone.
• Categorical Classification: According to the Pew Political Typology Quiz, ChatGPT-4 and Google Gemini were categorized as "Establishment Liberals," while Perplexity and Claude were classified as "Outsider Left." These classifications reinforce the observed biases and ideological leanings of each model.
• Custom Questions Analysis: In this assessment, ChatGPT-4 took a more liberal stance, while Perplexity, despite aligning with ChatGPT-4 on several issues, viewed corporations positively and took non-conclusive stances on key social justice issues, while taking a more conservative stance on foreign policy. Google Gemini once again showed similarity to ChatGPT-4 on certain issues but showed mixed positions on others while leaning closer to Perplexity's views on foreign policy. Claude largely aligned with ChatGPT-4 but was once again slightly more moderate and cautious in its views and tone.

7. Implications of Bias

8. Need for Transparency

9. Bias Mitigation Strategies

10. User Education

11. Future Research

C. Output Alignment

In addition to all these policy recommendations, Output Alignment with the public should also be taken into consideration, as that will make the datasets more informed. That will help these models gain more objectivity and transparency.

As people start relying on AI models for information, keeping them bias-free will be important in ensuring that the framing of information in a partisan manner is not constant and does not affect the base of facts so that politicians, as well as users, don't have to keep scraping for facts. Based on its findings, this paper suggests that more must be done to improve the methods for identifying and reducing political bias in AI, such as politicized training data, red-teaming, diversification of developers, reporting, and further study. Thus, by solving these problems, it is possible to adapt AI to help society and create fair and balanced AI that will not influence people's opinions.

According to Santurkar et al. [16], steps to measure Output Alignment include

• Standardizing Responses
• Defining Alignment Metrics
• Calculating Individual Alignment Scores (Cosine Similarity or Mean-Squared Difference Method) and aggregating them

The Wasserstein Distance parameter can be used to find the distribution spread among the scale of values we present. We can have a set of predefined values among the distribution according to the political ideologies, and this parameter can be used accordingly.

were

• A is the alignment
• D pertains to the Distribution
• N is the number of answer choices (excluding refusal)
• Q is the set of questions
• WD is the Wasserstein Distance
• N-1 is the Normalization Factor

To understand how Opinion Distribution works, Shibani [23] has used the figure referenced below to illustrate the same:

Figure 14. Opinion Distribution and Alignment for an AL Language Model (Referenced from (Santurkar et al., 2023)).

Figure 14. Opinion Distribution and Alignment for an AL Language Model (Referenced from (Santurkar et al., 2023)).

12. Conclusion

References

1. R. Binns, "Fairness in Machine Learning: Lessons from Political Philosophy," in Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR, 2018, pp. 149-159.
2. T. M. Mitchell, Machine Learning, McGraw Hill, 1997.
3. S. U. Noble, Algorithms of Oppression: How Search Engines Reinforce Racism, New York University Press, 2018.
4. C. O'Neil, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, Crown Publishing Group, 2016.
5. Z. Obermeyer, B. Powers, C. Vogeli, and S. Mullainathan, "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations," Science, vol. 366, no. 6464, pp. 447-453, 2019.
6. K. Crawford and R. Calo, "There is a Blind Spot in AI Research," Nature, vol. 538, no. 7625, pp. 311-313, 2016.
7. N. Diakopoulos, "Accountability in Algorithmic Decision Making," Communications of the ACM, vol. 59, no. 2, pp. 56-62, 2016.
8. V. Eubanks, Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor, St. Martin's Press, 2018.
9. ISideWith, "Political Party Quiz," [Online]. Available: https://www.isidewith.com/political-quiz.
10. S. West, "Google Bard vs. OpenAI's ChatGPT: Political Bias," TechCrunch, 2023.
11. M. Brundage, S. Avin, J. Wang, and G. Belfield, "The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation," arXiv preprint arXiv:1802.07228, 2018.
12. J. Morley, L. Floridi, L. Kinsey, and A. Elhalal, "From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices," Science and Engineering Ethics, vol. 26, no. 4, pp. 2141-2168, 2020.
13. M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, "Model Cards for Model Reporting," in Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019, pp. 220-229.
14. S. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, Prentice Hall, 2009.
15. S. A. Seshia, D. Sadigh, and S. S. Sastry, "Formal Methods for AI Systems," Communications of the ACM, vol. 62, no. 11, pp. 82-91, 2022.
16. S. Santurkar, L. Hong, J. Sharma, and A. Madry, "How Does AI Alignment Influence Opinion Distribution?," in Proceedings of the 37th International Conference on Machine Learning, 2023.
17. M. Morley, J. Floridi, and A. Elhalal, "Ethics as a Service: A Pragmatic Operationalization of AI Ethics," Minds and Machines, vol. 30, no. 1, pp. 77-89, 2020.
18. Pew Research Center, "Political Typology Quiz," [Online]. Available: https://www.pewresearch.org/politics/quiz/political-typology/.
19. Political Compass, "Political Compass Assessment," [Online]. Available: https://www.politicalcompass.org/test.
20. R. Robertson, J. Lazer, and C. Wilson, "Auditing Partisan Audience Bias Within Google Search," in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, 2023, pp. 39-50.
21. M. Buolamwini and T. Gebru, "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification," Proceedings of Machine Learning Research, vol. 81, pp. 1-15, 2018.
22. L. Harris, "OpenAI CEO Discusses AI Bias and Ethics," The Verge, 2023.
23. S. Shibani, "Opinion Distribution and Alignment for an AI Language Model," IEEE Transactions on Artificial Intelligence, vol. 4, no. 2, pp. 156-168, 2023.