Investigating Gender-Specific Discourse about Online Learning during COVID-19 on Twitter using Sentiment Analysis, Subjectivity Analysis, and Toxicity Analysis

1. Introduction

In December 2019, the world was challenged by the outbreak of COVID-19. COVID-19 is a coronavirus (CoV), specifically or otherwise known as SARS-CoV-2, that causes severe respiratory illness [1]. For some individuals, this respiratory illness may even cause acute respiratory distress syndrome or extra-pulmonary organ failure due to the extreme inflammatory response of the virus [2]. The dangers of COVID-19 resulted in a global search for understanding as to how this virus works and is contracted. SARS-CoV-2, named such because of its similarities with SARS-CoV, is believed to initially be contracted by humans through animal-human contact and thereafter spread through human-human contact [3]. This is not unusual, as other CoVs, like MERS-CoV, were transmitted through animal-human contact [4]. While COVID-19 is believed to have originated in a seafood market in Wuhan, China, the exact animal that may have infected the first identified patients remains unclear [3]. However, genome sequencing between SARS-CoV-2 and SARS-CoV in a bat from Yunnan Province, China suggests that bats may have been the originators of SARS-CoV-2 due to the CoVs having a 93.1% identity with the RaTG12 virus found in the bat [1].

After the initial outbreak, COVID-19 soon spread to different parts of the world and on March 11, 2020, the World Health Organization (WHO) declared COVID-19 an emergency [5]. As no treatments or vaccines for COVID-19 were available at that time, the virus rampaged unopposed across different countries, infecting and leading to the demise of people the likes of which the world had not witnessed in centuries. As of September 21, 2023, there have been a total of 770,778,396 cases and 6,958,499 deaths due to COVID-19 [6]. As an attempt to mitigate the spread of the virus, several countries across the world went on partial to complete lockdowns [7]. Such lockdowns affected the educational sector immensely. Universities, colleges, and schools across the world were left searching for solutions to best deliver course content online, engage learners, and conduct assessments during the lockdowns. During this time, online learning was considered a feasible solution. Online learning platforms are applications (web-based or software) that are used for designing, delivering, managing, monitoring, and accessing courses online [8]. This switch to online learning took place in more than 100 countries [9] and led to an incredible increase in the need to familiarize, utilize, and adopt online learning platforms by educators, students, administrators, and staff at universities, colleges, and schools across the world [10].

In today’s Internet of Everything era [11], the usage of social media platforms has skyrocketed as such platforms serve as virtual communities [12] for people to seamlessly connect with each other. Currently, around 4.9 billion individuals worldwide actively participate in social media, and it is projected that this number will reach 5.85 billion by 2027. On average, a social media user maintains approximately 8.4 social media profiles and allocates roughly 145 minutes each day to engage with various social media platforms. Among the various social media platforms available, Twitter has gained substantial popularity across diverse age groups [13,14]. This rapid transition to online learning resulted in a tremendous increase in the usage of social media platforms, such as Twitter, where individuals communicated their views, perspectives, and concerns towards online learning, leading to the generation of Big Data of social media conversations. This Big Data of conversations holds the potential to provide insights about these paradigms of information and seeking behavior about online learning during COVID-19.

1.1. COVID-19: A Brief Overview

COVID-19 is a type of coronavirus (CoVs). CoVs are a type of RNA virus consisting of four proteins: spike (S) protein, membrane (M) protein, envelope (E) protein, and nucleocapsid (N) protein. The S protein is involved with the attachment and recognition of the host cell (infection); the M protein is involved with shaping virions; the E protein is responsible for packaging and reproduction, and the N protein packages RNA into a nucleocapsid. The virions also have polyproteins that are translated after entry into the host or target cell. These polyproteins include 1a and b (pp1a, pp1b) [15,16]. The SARS-CoV-2 virus particle measures between 60 to 140 nanometers in diameter and boasts a positive-sense, single-stranded RNA genome spanning a length of 29891 base pairs [15].

Infection by SARS-CoV-2 occurs when the S protein binds to the surface receptor, angiotensin-converting enzyme 2 (ACE2), and enters type II pneumocytes, which are found in human lungs. The S protein is critical to transmission and infection by SARS-CoV-2, as the S protein has two domains, S1 and S2, where S1 involves binding of ACE2 and S2 involves fusion to the host cell at its membrane. Similarly important is the cleavage of the S protein. Having two cleavage sites, the S protein must be cleaved by nuclear proteases so that viral entry, and subsequent infection, of the host cell can happen. Previous research suggests that the S protein of SARS-CoV-2 has a higher binding efficiency and may explain its high rate of transmissibility. The high transmissibility is also explained by four amino acids found during insertion, P681, R682, R683, and A684, that have not been found in other CoVs before, nor was it found in the RaTG12 virus observed in the bat thought to have infected the first human patients of COVID-19 [17,18].

While infections involving various organs have been documented in different cases, the typical effect of the SARS-CoV-2 virus on patients is centered around their respiratory systems. Investigation of the infections caused in Wuhan in December 2019 has shown that patients suffer from a range of symptoms during the initial days of contracting this virus. These symptoms encompass fever, a dry cough, breathing difficulties, headaches, dizziness, fatigue, nausea, and diarrhea. It’s important to note that the symptoms of COVID-19 can vary from person to person both in terms of the nature of the symptoms as well as the intensity of one or more symptoms [19,20].

1.2. Twitter: A Globally Popular Social Media Platform

Twitter ranks as the sixth most popular social platform in the United States and the seventh globally [21,22]. Notably, 42% of Americans between the ages of 12 and 34 are active Twitter users, marking a substantial 36.6% surge over the span of two years. The frequency of posting on the platform appears to correlate with the number of accounts followed, as users who post more than five Tweets per month tend to follow an average of 405 accounts, in contrast to those who post less frequently and follow an average of 105 accounts [22]. Furthermore, users spend an average of 1.1 hours per week on Twitter, which equates to 4.4 hours per month on the platform [23].

In 2023, Twitter boasts 353.9 million monthly active users, constituting 9.4% of the global social media user base [24]. The majority of Twitter users, accounting for 52.9%, fall within the age range of 25 to 49 years. Notably, 17.1% of users belong to the 14-18 age group, 6.6% to the 13-17 age group, and the remaining 17% are aged 50 and above [25]. On average, U.S. adults spend approximately 34.1 minutes per day on Twitter [26]. Impressively, a staggering 500 million tweets are published each day, equivalent to 5,787 tweets per second. An encouraging statistic reveals that 42.3% of U.S. users utilize Twitter at least once a month and it is currently the ninth most visited website globally. The countries with the highest number of Twitter users include the United States with 95.4 million users, Japan with 67.45 million, India with 27.25 million, Brazil with 24.3 million, Indonesia with 24 million, the UK with 23.15 million, Turkey with 18.55 million, and Mexico with 17.2 million [28,29]. On average, a Twitter user spends 5.1 hours per month on the platform, translating to approximately 10 minutes daily. A fifth of users under 30 visit frequently, and 25% use the platform every week, with 71% visiting at least weekly. Twitter is a significant source of news, with 55% of users accessing it regularly for this purpose. Ninety-six percent of U.S. Twitter users report monthly usage. Additionally, 82% engage with Twitter for entertainment. In terms of activity, 6,000 tweets are sent per second. Mobile usage is dominant, with 80% of active users accessing Twitter via smartphones [30,31].

Due to this ubiquitousness of Twitter, studying the multimodal components of information-seeking and sharing behavior has been of keen interest to scientists from different disciplines as can be seen from recent works in this field that focused on the analysis of tweets about various emerging technologies [32,33,34,35], global affairs [36,37,38], humanitarian issues [39,40,41], and societal problems [42,43,44]. Since the outbreak of COVID-19, there have been several research works conducted in this field (Section 2) where researchers analyzed different components and characteristics of the tweets to interpret the varying degrees of public perceptions, attitudes, views, and responses towards this pandemic. However, the tweeting patterns about online learning during COVID-19, with respect to the gender of twitter users, have not been investigated in any prior work in this field.

1.3. Gender Diversity on Social Media Platforms

Gender differences in content creation online have been comprehensively studied by researchers from different disciplines [45] as such differences have been considered important in the investigation of digital divides that produce inequalities of experience and opportunity [46,47]. Analysis of gender diversity and the underlying patterns of content creation on social media platforms has also been widely investigated [48]. However, the findings are mixed. Some studies have concluded that males are more likely to express themselves on social media as compared to females [49,50,51], while others found no such difference between genders [52,53,54]. The gender diversity related to the usage of social media platforms has varied over the years in different geographic regions [55]. For instance, Figure 1 shows the variation in social media use by gender from the findings of a survey conducted by the Pew Research Center from 2005 to 2021 [56].

In general, most social media platforms tend to exhibit a notable preponderance of male users over their female counterparts, for example – WhatsApp [57], Sina Weibo [58], QQ [59], Telegram [60], Quora [61], Tumblr [62], Facebook, LinkedIn, Instagram [63], and WeChat [64]. Nevertheless, there do exist exceptions to this prevailing trend. Snapchat has male and female users accounting for 48.2% and 51%, respectively [65]. These statistics about the percentage of male and female users in different social media platforms are summarized in Table 1. As can be seen from Table 1, Twitter has the highest gender gap as compared to several social media platforms such as Instagram, Tumblr, WhatsApp, WeChat, Quora, Facebook, LinkedIn, Telegram, Sina Weibo, QQ, and SnapChat. Therefore, the work presented in this paper focuses on the analysis of user diversity-based (with a specific focus on gender) patterns of public discourse on Twitter in the context of online learning during COVID-19.

The rest of this paper is organized as follows. In Section 2, a comprehensive review of recent works in this field is presented. Section 3 discusses the methodology that was followed for this work. The results and scientific contributions of this study are presented and discussed in Section 4. It is followed by Section 5, which summarizes the contributions of this study and outlines the scope of future research in this area.

2. Literature Review

This section is divided into two parts. Section 2.1 presents an overview of the recent works related to sentiment analysis of tweets about COVID-19. In Section 2.2, a review of emerging works in this field is presented where the primary focus was the analysis of tweets about online learning during COVID-19.

2.1. A Brief Review of Recent Works related to Sentiment Analysis of Tweets about COVID-19

Villavicencio et al. [66] analyzed tweets to determine the sentiment of people towards the Philippines government, regarding their response to COVID-19. They used the Naïve Bayes model to classify the tweets as positive, negative, and neutral. Their model achieved an accuracy of 81.77%. Boon-Itt et al. [67] conducted a study using Twitter data to gain insights into public awareness and concerns related to the COVID-19 pandemic. They conducted sentiment analysis and topic modeling on a dataset of over 100,000 tweets related to COVID-19. Taking a slightly different angle, Marcec et al. [68] analyzed 701,891 tweets mentioning the COVID-19 vaccines, specifically AstraZeneca/Oxford, Pfizer/BioNTech, and Moderna. They used the AFINN lexicon to calculate the daily average sentiment. The findings of this work showed that Pfizer and Moderna remained consistently positive as opposed to AstraZeneca which showed a declining trend. Machuca et al. [69] focused on evaluating the general public sentiment towards COVID-19. They used a Logistic Regression-based approach to classify the tweets as positive or negative. The methodology achieved 78.5% accuracy. Kruspe et al. [70] performed sentiment analysis of Tweets about COVID-19 from Europe and their approach used a neural network for performing sentiment analysis. Similarly, the works of Vijay et al. [71], Shofiya et al. [72], and Sontayasara et al. [73] focused on sentiment analysis of tweets about COVID-19 from India, Canada, and Thailand, respectively. Nemes et al. [74] used a Recurrent Neural Network for sentiment classification of the tweets about COVID-19.

Okango et al. [75] employed a dictionary-based method for detecting sentiments in tweets about COVID-19. Their work indicated that mental health issues and lack of supplies were a direct result of the pandemic. The work of Singh et al. [76] focused on a deep-learning approach for sentiment analysis of tweets about COVID-19. Their algorithm was based on an LSTM-RNN-based network and enhanced featured weighting by attention layers. Kaur et al. [77] developed an algorithm, the Hybrid Heterogeneous Support Vector Machine (H-SVM), for sentiment classification. The algorithm was able to categorize tweets as positive, negative, and neutral as well as detect the intensity of sentiments. In [78], Vernikou et al. implemented sentiment analysis through seven different deep-learning models based on LSTM neural networks. Sharma et al. [79] studied the sentiments of people towards COVID-19 from the USA and India using text mining-based approaches. The authors also discussed how their findings could provide guidance to authorities in healthcare to tailor their support policies in response to the emotional state of their people. Sanders et al. [80] took a slightly different approach to also aid policy makers, by analyzing over one million tweets to illustrate public attitudes towards mask-wearing during the pandemic. They observed both the volume and polarity of tweets relating to mask-wearing increased over time. Alabid et al. [81] used two machine learning classification models - SVM and Naïve Bayes classifier to perform sentiment analysis of tweets related to COVID-19 vaccines. Mansoor et al. [82] used Long Short-Term Memory (LSTM) and Artificial Neural Networks (ANN) to perform sentiment analysis of the public discourse on Twitter about COVID-19. Singh et al. [83] studied two datasets, one of tweets from people all over the world and the second restricted to tweets only by Indians. They conducted sentiment analysis using the BERT model and achieved a classification accuracy of 94%. Imamah et al. [84] conducted a sentiment classification of 355384 tweets using Logistic Regression. The objective of their work was to study the negative effects of ‘stay at home’ on people’s mental health. Their model achieved a sentiment classification accuracy of 94.71%. As can be seen from this review, a considerable number of works in this field have focused on the sentiment analysis of tweets about COVID-19. In the context of online learning during COVID-19, understanding the underlying patterns of public emotions becomes crucial and this has been investigated in multiple prior works in this field. A review of the same is presented in Section 2.2.

2.2. Review of Recent Works in Twitter Data Mining and Analysis related to Online Learning during COVID-19

Sahir et al. [85] used the Naïve Bayes classifier to perform sentiment analysis of tweets about online learning posted in October 2020 from individuals in Indonesia. The results showed that the percentage of negative, positive, and neutral tweets were 74%, 25%, and 1%, respectively. Althagafi et al. [86] analyzed tweets about online learning during COVID-19 posted by individuals from Saudi Arabia. They used the Random Forest approach and the K-Nearest Neighbor (KNN) classifier alongside Naïve Bayes and found that most tweets were neutral about online learning. Ali [87] used Naïve Bayes, Multinomial Naïve Bayes, KNN, Logistic Regression, and SVM to analyze the public opinion towards online learning during COVID-19. The results showed that the SVM classifier achieved the highest accuracy of 89.6%. Alcober et al. [88] reported the results of multiple machine learning approaches such as Naïve Bayes, Logistic Regression, and Random Forest for performing sentiment analysis of tweets about online learning.

While Remali et al. [89] also used Naïve Bayes and Random Forest, their research additionally utilized the Support Vector Machine (SVM) approach and a Decision Tree-based modeling. The classifiers evaluated tweets posted between July 2020 and August 2020. The results showed that the SVM classifier using the VADER lexicon achieved the highest accuracy of 90.41% [89]. The work of Senadhira et al. [90] showed that an Artificial Neural Network (ANN)-based approach outperformed an SVM-based approach for sentiment analysis of tweets about online learning. Lubis et al. [91] used a KNN-based method for sentiment analysis of tweets about online learning. The model achieved a performance accuracy of 88.5% and showed that a higher number of tweets were positive. These findings are consistent with another study [92] which reported that for tweets posted between July 2020 and August 2020, 54% were positive tweets. The findings of the work by Isnain et al. [93] indicated that the public opinion towards online learning between February 2020 and September 2020 was positive. These results were computed with a KNN-based approach that reported an accuracy of 84.65%.

Aljabri et al. [94] analyzed results at different education stages. Using Term Frequency-Inverse Document Frequency (TF-IDF) as a feature extraction to a Logistic Regression classifier, the model developed by the authors achieved an accuracy of 89.9%. The results indicated positive sentiment from elementary through high school, but negative sentiment for universities. The work by Asare et al. [95] aimed to cluster the most commonly used words into general topics or themes. The analysis of different topics found 48.9% of positive tweets, with “learning,” “COVID,” “online,” and “distance” being the most used words. Mujahid et al. [96] used TF-IDF alongside Bag of Words (BoW) for analyzing tweets about online learning. They also used SMOTE to balance the data. The results demonstrated that the Random Forest and SVM classifier achieved an accuracy of 95% when used with the BoW features. Al-Obeidat [97] also used TF-IDF to classify sentiments related to online education during the pandemic. The study reported that students had generally negative feelings towards online learning. In view of the propagation of misinformation on Twitter during the pandemic, Waheeb et al. [98] proposed eliminating noise using AutoEncoder in their work. The results found that their approach yielded a higher accuracy for sentiment analysis, with an F1-score value of 0.945. Rijal et al. [99] aimed to remove bias from sentiment analysis using concepts of feature selection. Their methodology involved the usage of the AdaBoost approach on the C4.5 method. The results found that the accuracy of C4.5 and Random Forest went up from 48.21% and 50.35% to 94.47% for detecting sentiments in tweets about online learning. Martinez [101] investigated negative sentiments about “teaching and schools” and “teaching and online” using multiple concepts of Natural Language Processing. Their study reported negativity towards both topics. At the same time, a higher negative sentiment along with expressions of anger, distrust, or stress towards “teaching and school,” was observed.

As can be seen from this review of works related to the analysis of public discourse on Twitter about online learning during COVID-19, such works have multiple limitations centered around lack of reporting from multiple sentiment analysis approaches to explain the trends of sentiments, lack of focus on subjectivity analysis, lack of focus on toxicity analysis, and lack of focus on gender-specific tweeting patterns. Addressing these research gaps serves as the main motivation for this work.

3. Methodology

This section presents the methodology that was followed for this research work. This section is divided into 2 parts. In Section 3.1 a description of the dataset that was used for this research work is presented. Section 3.2 discusses the procedure and the methods that were followed for this research work.

3.1. Data Description

The dataset used for this research was proposed in [102]. The dataset consists of about 50,000 unique Tweet IDs of Tweets about online learning during COVID-19. The dataset covers tweets posted on Twitter between November 9, 2021, and July 13, 2022. The dataset includes tweets in 34 different languages, with English being the most common. The dataset spans 237 different days, with the highest tweet count recorded on January 5, 2022. These tweets were posted by 17,950 distinct Twitter users, with a combined follower count of 4,345,192,697. The dataset includes 3,273,263 favorites and 556,980 retweets. Furthermore, 5,722 tweets in the dataset were posted by verified Twitter accounts, while the rest came from unverified accounts. There are a total of 7,869 distinct URLs embedded in these tweets. The Tweets IDs present in this dataset are organized into nine .txt files based on the date range of the tweets. The dataset was developed by mining tweets that referred to COVID-19 and online learning at the same time. To perform the same, a collection of synonyms of COVID-19 such as COVID, COVID19, coronavirus, Omicron, etc., and a collection of synonyms of online learning such as online education, remote education, remote learning, e-learning etc. were used. Thereafter, duplicate tweets were removed to obtain a collection of about 50,000 Tweet IDs. The standard procedure for working with such a dataset is the hydration of the Tweet IDs. However, this dataset was developed by the first author of this paper. So, the tweets were already available, and hydration was not necessary. In addition to the Tweet IDs, the dataset file that was used comprised several characteristic properties of Tweets and Twitter users who posted these Tweets, such as the Tweet Source, Tweet Text, Retweet count, user location, username, user favorites count, user follower count, user friends, count, user screen name, and user status count.

The dataset complies with the FAIR principles (Findability, Accessibility, Interoperability, and Reusability) of scientific data management. It is designed to be findable through a unique and permanent DOI. It is accessible online for users to locate and download. The dataset is interoperable as it uses .txt files, enabling compatibility across various computer systems and applications. Finally, it is reusable because researchers can obtain tweet-related information, such as user ID, username, and retweet count, for all Tweet IDs through a hydration process, facilitating data analysis and interpretation while adhering to Twitter policies.

3.2. System Design and Development

At first, the data preprocessing of these Tweets was performed by writing a program in Python 3.11.5. The data preprocessing involved the following steps. The pseudocode of this program is shown in Algorithm 1.

a)

Removal of characters that are not alphabets.

b)

Removal of URLs

c)

Removal of hashtags

d)

Removal of user mentions

e)

Detection of English words using tokenization.

f)

Stemming and Lemmatization.

g)

Removal of stop words.

h)

Removal of numbers

Algorithm 1: Data Preprocessing

After performing data preprocessing, the GenderPerformr package in Python developed by Wang et al. [104,105] was applied to the usernames to detect their gender. GenderPerformr uses an LSTM model built in PyTorch to analyze usernames and detect genders in terms of male or female. The working of this algorithm was extended to classify usernames into 4 categories – male, female, none, and maybe. The algorithm classified a username as 'male' if that username matched a male name from the list of male names accessible to this Python package. Similarly, the algorithm classified a username as 'female' if that username matched a female name from the list of female names accessible to this Python package. The algorithm classified a username as 'none' if that username was a word in the English dictionary that cannot be a person's name. Finally, the algorithm classified a username as 'maybe' if the username was a word absent in the list of male and female names accessible to this Python package and the username was also not an English word. The classification performed by this algorithm was manually verified. Furthermore, all the usernames that were classified as ‘maybe’ were manually classified as male, female, or none. The pseudocode that was written in Python 3.11.5 to detect genders from Twitter usernames is presented as Algorithm 2.

Algorithm 2: Detect Gender from Twitter Usernames

Thereafter, three different models for sentiment analysis – VADER, Afinn, and TextBlob were applied to the Tweets. VADER (Valence Aware Dictionary and sEntiment Reasoner), developed by Hutto et al. [106] is a lexicon and rule-based sentiment analysis tool that is specifically attuned to sentiments expressed in social media. The VADER approach can analyze a text and classify it as positive, negative, or neutral. Furthermore, it can also detect the compound sentiment score and the intensity of the sentiment (0 to +4 for positive sentiment and 0 to -4 for negative sentiment) expressed in a given text. The AFINN lexicon developed by Nielsen is also used to analyze Twitter sentiment [107]. The AFINN lexicon is a list of English terms manually rated for valence with an integer between -5 (negative) and +5 (positive). Finally, TextBlob, developed by Lauria [108] is a lexicon-based sentiment analyzer that also uses a set of predefined rules to perform sentiment analysis and subjectivity analysis. The sentiment score lies between (-1 to 1) where -1 identifies the most negative words such as ‘disgusting’, ‘awful’, and ‘pathetic’, and 1 identifies the most positive words like ‘excellent’, and ‘best’. The subjectivity score lies between (0 and 1), It shows the amount of personal opinion, if a sentence has high subjectivity i.e., close to 1, it resembles that the text contains more personal opinion than factual information. These three approaches for performing sentiment analysis of tweets have been very popular as can be seen from several recent works in this field which used VADER [109,110,111,112], Afinn [113,114,115,116], and TextBlob [117,118,119,120]. The pseudocodes of the programs that were written in Python 3.11.5 to apply VADER, Afinn, and TextBlob to these Tweets are shown in Algorithms 3, 4, and 5, respectively.

Algorithm 3: Detect Sentiment of Tweets Using VADER

Algorithm 4: Detect Sentiment of Tweets Using Afinn

Algorithm 5: Detect Polarity and Subjectivity of Tweets Using TextBlob

Thereafter, toxicity analysis of these Tweets was performed using the Detoxify package [121]. It includes three different trained models and outputs different toxicity categories. These models are trained on data from the three Kaggle jigsaw toxic comment classification challenges [122,123,124]. Using this package, each Tweet received a score in terms of the degree of toxicity, obscene content, identity attack, insult, threat, and sexually explicit content. The pseudocode of the program that was written in Python 3.11.5 to apply the Detoxify package to these Tweets is shown in Algorithm 6.

Algorithm 6: Perform Toxicity Analysis of the Tweets Using Detoxify

Figure 2 represents a flowchart summarizing the working of Algorithms 1 to 6. In addition to the above, average activity analysis of different genders (male, female, and none) was also performed. The pseudocode of the program that was written in Python 3.11.5 to compute and analyze the average activity of different genders is shown in Algorithm 7. This program uses the formula for the total activity calculation of a twitter user which was proposed in an earlier work in this field [125]. This formula is shown in Equation (1):

Activity of a Twitter User = Author Tweets count + Author favorites count

(1)

Algorithm 7: Compute Average Activity of different Genders on a monthly basis

Finally, the trends in tweeting patterns related to online learning from different geographic regions were also analyzed to understand the gender-specific tweeting patterns from different geographic regions. To perform this analysis, the PyCountry [127] package was used. Specifically, the program that was written in Python applied the fuzzy search function available in this package to detect the country of a twitter user based on the publicly listed city, county, state, or region on their Twitter profile. Algorithm 8 shows the pseudocode of the Python program that was written to perform this task. The results of applying all these algorithms on the dataset are discussed in Section 4.

Algorithm 8: Detect Locations of Twitter Users, Visualize Gender-Specific Tweeting Patterns

4. Results and Discussion

This section presents and discusses the results of this study. As stated in Section 3, Algorithm 2 was run on the dataset to detect the gender of each Twitter user. After obtaining the output from this algorithm, the classifications were manually verified as well and the ‘maybe’ labels were manually classified as either male, female, or none. Thereafter, the dataset contained only three labels for the “Gender” attribute – male, female, and none. Figure 3 shows a pie chart-based representation of the same. As can be seen from Figure 3, out of the tweets posted by males and females, males posted a higher percentage of the tweets.

The results obtained from Algorithm 3 are presented next. Figure 4 presents a pie chart to show the percentage of tweets in each of the sentiment classes (positive, negative, and neutral) by taking all the genders together. As can be seen from this Figure, the percentages of positive, negative, and neutral tweets as per VADER were 41.704%, 29.932%, and 28.364%, respectively.

Next, for each sentiment class (positive, negative, and neutral) the distribution in terms of tweets posted by males, females, and twitter accounts assigned a none gender was calculated. The results of the same are shown in Figure 5, Figure 6 and Figure 7.

As can be seen from Figure 5, Figure 6 and Figure 7, for each sentiment label (positive, negative, and neutral) between males and females, males posted a higher percentage of tweets. A similar analysis was performed by applying Algorithm 4 and 5 on the dataset. The results of applying Algorithm 4 are presented in Figure 8, Figure 9, Figure 10 and Figure 11.

As can be seen from Figure 8, the percentage of positive tweets (as per the Afinn approach for sentiment analysis) was higher than the percentage of negative and neutral tweets. This is consistent with the findings from VADER (presented in 4). Furthermore, Figure 9, Figure 10 and Figure 11 show that for each sentiment label (positive, negative, and neutral) between males and females, males posted a higher percentage of the tweets. This finding is also consistent with the results obtained from VADER as shown in Figure 5, Figure 6 and Figure 7. The results of applying TextBlob for performing sentiment analysis are shown in Figure 12, Figure 13, Figure 14 and Figure 15.

From Figure 12, it can be inferred that, as per Afinn, the percentage of positive tweets was higher as compared to the percentage of negative and neutral tweets. This is consistent with the results of VADER (Figure 4) and Afinn (Figure 8). Furthermore, Figure 13, Figure 14 and Figure 15 show that for each sentiment class (positive, negative, and neutral), between males and females, males posted a higher percentage of the tweets. Once again, the results are consistent with the observations from VADER (Figure 5, Figure 6 and Figure 7) and Afinn (Figure 9, Figure 10 and Figure 11). In addition to sentiment analysis, TextBlob also computed the subjectivity of each tweet and categorized each tweet as highly opinionated, least opinionated, or neutral. The results of the same are shown in Figure 16, Figure 17, Figure 18 and Figure 19.

As can be seen from Figure 16, more than a majority of the tweets were least opinionated. To add to this, Figure 17, Figure 18 and Figure 19 show that for each subjectivity class (i.e. highly opinionated, least opinionated, and neutral), between males and females, males posted a higher percentage of the tweets. The results obtained from Algorithm 6 are discussed next. This algorithm analyzed all the tweets and categorized them into one of toxicity classes - toxicity, obscene, identity attack, insult, threat, and sexually explicit. The number of tweets that were classified into each of these classes was 36081, 8729, 3411, 1165, 18, and 4, respectively. This is shown in Figure 20. Thereafter, the percentage of tweets posted by each gender for each of these categories of toxic content was analyzed and the results are presented in Figure 21, Figure 22, Figure 23, Figure 24, Figure 25 and Figure 26.

From Figure 21, Figure 22, Figure 23 and Figure 24, it can be seen that for the classes – toxicity, obscene, identity attack, and insult, between males and females, males posted a higher percentage of the tweets. Furthermore, Figure 25 shows that there wasn’t any tweet from females that was assigned a threat label. Figure 26 shows that for those tweets that were categorized as sexually explicit, between males and females, females posted a higher percentage of those tweets. It is worth mentioning here that the results of Figure 25 and Figure 26 are based on data that constitutes less than 1% of the tweets present in the dataset. So, in a real-world scenario, these percentages could vary when a greater number of tweets are posted for each of the two categories – threat and sexually explicit.

In addition to analyzing the varying trends in sentiments and toxicity, the content of the underlying tweets was also analyzed using word clouds. For generation of these word clouds the top 100 words (in terms of frequency were considered). To perform the same, a consensus of sentiment labels from the three different sentiment analysis approaches was considered. For instance, to prepare a word cloud of positive tweets, all those tweets that were labeled as positive by VADER, Afinn, and TextBlob were considered. A word cloud was developed to represent the same. Thereafter, for all the positive tweets, gender-specific tweeting patterns were also analyzed to compute the top 100 words used by males for positive tweets, the top 100 words used by females for positive tweets, and the top 100 words used by twitter accounts associated with a none gender label. A high degree of overlap in terms of the 100 words for all these scenarios was observed. More specifically, a total of 79 words were common amongst the lists of the top 100 words for positive tweets, the top 100 words used by males for positive tweets, the top 100 words used by females for positive tweets, and the top 100 words used by Twitter accounts associated with a none gender label. So, to avoid redundancy, Figure 27 shows a word cloud-based representation of the top 100 words used in positive tweets. Similarly, a high degree of overlap in terms of the 100 words was also observed for the analysis of different lists for negative tweets and neutral tweets. So, to avoid redundancy, Figure 28 and Figure 29 show word cloud-based representations of the top 100 words used in negative tweets and neutral tweets, respectively. In a similar manner, the top 100 frequently used words for the different subjectivity classes were also computed and word cloud-based representations of the same are shown in Figure 30, Figure 31 and Figure 32.

After performing this analysis, a similar word frequency-based analysis was performed for the different categories of toxic content that were detected in the tweets using Algorithm 6. These classes were toxicity, obscene, identity attack, insult, threat, and sexually explicit. As explained in Algorithm 6, each tweet was assigned a score for each of these classes and whichever class received the highest score, the label of the tweet was decided accordingly. For instance, if the toxicity score for a tweet was higher than the scores that the tweet received for the classes - obscene, identity attack, insult, threat, and sexually explicit, then the label of that tweet was assigned as toxicity. Similarly, if the obscene score for a tweet was higher than the scores that the tweet received for the classes - toxicity, identity attack, insult, threat, and sexually explicit, then the label of that tweet was assigned as obscene. The results of this word cloud-based analysis for the top 100 words (in terms of frequency) for each of these classes are shown in Figure 33, Figure 34, Figure 35, Figure 36, Figure 37 and Figure 38.

As can be seen from Figure 33, Figure 34, Figure 35, Figure 36, Figure 37 and Figure 38 the patterns of communication were diverse for each of the categories of toxic content designated by the classes - toxicity, identity attack, insult, threat, and sexually explicit. At the same time, Figure 37 and Figure 38 appear significantly different in terms of the top 100 words used. This also shows that for tweets that were categorized as threat (Figure 37) and as containing sexually explicit content (Figure 38) the paradigms of communication and information exchange in those tweets were very different as compared to tweets categorized into any of the remaining classes representing toxic content. In addition to performing this word cloud-based analysis, the scores each of these classes received were analyzed to infer the trends of their intensities over time. To perform this analysis, the mean value of each of these classes was computed per month and the results were plotted in a graphical manner as shown in Figure 39.

From Figure 39, several insights related to the tweeting patterns of the general public can be inferred. For instance, the intensity of toxicity was higher than the intensity of obscene, identity attack, insult, threat, and sexually explicit content. Similarly, the intensity of insult was higher than the intensity of obscene, identity attack, threat, and sexually explicit content. Next, gender-specific tweeting patterns for each of these categories of toxic content were analyzed to understand the trends of the same. These results are shown in Figure 40, Figure 41, Figure 42, Figure 43, Figure 44 and Figure 45. This analysis also helped to unravel multiple paradigms of tweeting behavior of different genders in the context of online learning during COVID-19. For instance, Figure 40 and Figure 44 show that the intensity of toxicity and threat in tweets by males and females has increased since July 2022. The analysis shown in Figure 41, shows that the intensity of obscene content in tweets by males and females has decreased since May 2022.

The result of Algorithm 7 is shown in Figure 46. As can be seen from this Figure, between males and females, the average activity of females has been higher in all months other than March 2022. The results from Algorithm 7 are presented in Figure 47 and Figure 48, respectively. Figure 47 shows the trends in tweets about online learning during COVID-19 posted by males from different countries of the world. Similarly, Figure 48 shows the trends in tweets about online learning during COVID-19 posted by females from different countries of the world.

Figure 47 and Figure 48 reveal the patterns of posting tweets by males and females about online learning during COVID-19. These patterns include similarities as well as differences. For instance, from these two figures, it can be inferred that in India a higher percentage of the tweets were posted by males as compared to females. However, in Australia, a higher percentage of the tweets were posted by females as compared to males. Finally, a comparative study is presented in Table 2 where the focus area of this work is compared with the focus areas of prior areas in this field to highlight its novelty and relevance. As can be seen from this Table, the work presented in this paper is the first work in this area of research where the focus area has included text analysis, sentiment analysis, analysis of toxic content, and subjectivity analysis of tweets about online learning during COVID-19. It is worth mentioning here that the work by Martinez et al. [101] considered only two types of toxic content – insults and threats whereas the work presented in this paper performs the detection of six types of toxic content - toxicity, obscene, identity attack, insult, threat, and sexually explicit. Furthermore, no prior work in this field has performed a gender-specific analysis of tweets about online learning during COVID-19. As this paper analyzes the tweeting patterns in terms of gender, the authors would like to clarify three aspects. First, the results presented and discussed in this paper aim to address the research gaps in this field (as discussed in Section 2). These results are not presented with the intention to comment on any gender directly or indirectly. Second, the authors respect the gender identity of every individual and do not intend to comment on the same in any manner by presenting these results. Third, the authors respect every gender identity and associated pronouns [126]. The results presented in this paper take into account only three gender categories – male, female, and none as the GenderPerformr package (the current state-of-the-art method that predicts gender from usernames) has limitations.

5. Conclusions

To reduce the rapid spread of the SARS-CoV-2 virus, several universities, colleges, and schools across the world transitioned to online learning. This was associated with a range of emotions in students, educators, and the general public who used social media platforms such as Twitter during this time to share and exchange information, views, and perspectives related to online learning leading to the generation of Big Data. Twitter has been popular amongst researchers from different domains for the investigation of patterns of public discourse related to different topics. Furthermore, out of several social media platforms, Twitter has the highest gender gap as of 2023. There have been a few works published in the last few months where sentiment analysis of tweets about online learning during COVID-19 was performed. However, those works have multiple limitations centered around a lack of reporting from multiple sentiment analysis approaches, a lack of focus on subjectivity analysis, a lack of focus on toxicity analysis, and a lack of focus on gender-specific tweeting patterns. The work presented in this paper aims to address these research gaps as well as aims to contribute towards advancing research and development in this field. A dataset comprising about 50,000 Tweets about online learning during COVID-19, posted on Twitter between November 9, 2021, and July 13, 2022, was analyzed for this study. This work reports multiple novel findings. First, the results of sentiment analysis from VADER, Afinn, and TextBlob show that a higher percentage of the tweets were positive. The results of gender-specific sentiment analysis indicate that for positive tweets, negative tweets, and neutral tweets, between males and females, males posted a higher percentage of the tweets. Second, the results from subjectivity analysis show that the percentage of least opinionated, neutral opinionated, and highly opinionated tweets were 56.568%, 30.898%, and 12.534%, respectively. The gender-specific results for subjectivity analysis show that for each subjectivity class (least opinionated, neutral opinionated, and highly opinionated) males posted a higher percentage of tweets as compared to females. Third, toxicity detection was applied to the tweets to detect different categories of toxic content - toxicity, obscene, identity attack, insult, threat, and sexually explicit. The gender-specific analysis of the percentage of tweets posted by each gender in each of these categories revealed several novel insights. For instance, males posted a higher percentage of tweets that were categorized as toxicity, obscene, identity attack, insult, and threat, as compared to females. However, for the sexually explicit category, females posted a higher percentage of tweets as compared to males. Fourth, gender-specific tweeting patterns for each of these categories of toxic content were analyzed to understand the trends of the same. These results unraveled multiple paradigms of tweeting behavior of different genders in the context of online learning during COVID-19. For instance, the results show that the intensity of toxicity and threat in tweets by males and females has increased since July 2022. To add to this, the intensity of obscene content in tweets by males and females has decreased since May 2022. Fifth, the average activity of males and females per month in this time range was also investigated. The findings indicate that the average activity of females has been higher in all months as compared to males other than March 2022. Finally, country-specific tweeting patterns of males and females were also investigated which presented multiple novel insights. For instance, in India, a higher percentage of tweets about online learning during COVID-19 were posted by males as compared to females. However, in Australia, a higher percentage of such tweets were posted by females as compared to males. As per the best knowledge of the authors, no similar work has been done in this field thus far. Future work in this area would involve performing gender-specific topic modeling to investigate the similarities and differences in terms of the topics that have been represented in the tweets posted by males and females.