Introduction
As of 2022, more than 5.3 billion people, accounting for over 67.9% of the global population, were using the internet. In South Korea, as of 2022, 13,284,218 people were utilizing social networking platforms. [
1,
2] Among these, YouTube had approximately 785.39 million users as of 2022, focusing on video content, and users were consuming necessary information in video format in their daily lives. [
3,
4] These characteristics indicated that YouTube had emerged as an important means of communication and information sharing among users, extending beyond being a mere video repository. Especially in the medical field, it played a key role in the dissemination of health information. Consequently, the advertising costs on online platforms, including YouTube, had been on the rise. [
5] However, healthcare providers and the government had expressed concerns about the quality of information provided on these platforms, given the lack of regulation and guidelines in the face of the vast amount of uploaded information. These concerns were regarding the reliability of information and the potential for misinformation dissemination.
Hence, the primary objective of this study was to apply a systematic evaluation framework for health-related information to videos discovered on YouTube, utilizing a variety of keywords associated with 'eyelid ptosis' and 'droopy eyelid.' This study specifically targeted users who rely on YouTube as a source of medical information. Our intention was to conduct a comparative analysis of the outcomes derived from this assessment.
Methods
The study was conducted in September 2023, with data collection taking place on YouTube (
http://Youtube.com) through searches. Prior search history and cookies were deleted, and the search was conducted using the Google Chrome browser with algorithmic elements excluded. The top 20 videos were selected for evaluation after entering search queries, and the following types of videos were excluded when selecting videos:
The video evaluation in this study, based on the search results in Korean, was carried out using the "Internet Health Information Certification Standards" developed by the Korean Medical Association (
Table 1). This consisted of reliability assessment, format evaluation, and content evaluation, with each consisting of several components. Additionally, the evaluation was conducted according to widely used evaluation criteria for health information using the Global Quality Scale (GQS) (
Table 2) [
7,
8]. One plastic surgeon assessed information quality, applying scientific reliability evaluation criteria to six specific components out of a total of 20 items. The remaining 19 items were evaluated based on a "yes" or "no" rating criterion, resulting in a maximum score of 24 points. The study also included an evaluation of the producers based on whether the video was produced by a certified medical service provider in South Korea or by medical professionals. Furthermore, evaluations were conducted to identify the presence of non-medical information. The search terms included "eyelid ptosis" and "droopy eyelid" as the main keywords in Korean and "surgery," "hospital," and "diagnosis" as additional keywords, resulting in the creation of a total of six search queries. They were all searched in Korean, as detailed in
Table 3.
Statistical analysis
Data analysis in this study was conducted by dividing the data into multiple groups and using ANOVA analysis, chi-square tests, and t-tests. Descriptive methods such as means, standard deviations, minimum values, and maximum values were used. Results were evaluated at a 95% confidence interval, with significance set at P < 0.05.
Results
By combining the two main keywords and three additional keywords, a total of six Korean search results were obtained. A total of 76 videos were analyzed, excluding duplicate videos from the top 20 videos for each keyword. The data in
Table 4 summarizes the views, video lengths, GQS, and KMA’s scale for these videos.
YouTube provided a marker for South Korean certified medical service providers, and 50% of the "ptosis surgery" search results were produced by South Korean certified medical service providers, whereas "droopy eyelid diagnosis" search results showed the lowest ratio at 5% (
Table 5). The ratio of South Korean certified medical service providers in search results related to "ptosis" was 40%, and in those related to "droopy eyelid," it was 10%. This indicated a tendency for videos produced by certified medical service providers to be more easily searchable when medical terminology was used. However, medical professionals or hospitals produced 86.7% of the videos, and there were also non-medical videos produced by various creators, including 2 vlogs or reviews, 1 news video, 3 produced by physical therapists, and 1 produced by a Pilates instructor. According to the Internet Health Information Certification Standards of the Korean Medical Association, the search results for "ptosis surgery" had the highest average score of 16.3, while the search results for "droopy eyelid diagnosis" had the lowest average score of 13.8. The GQS scored the highest at an average of 3.25 points for the keyword combination "ptosis surgery" and the lowest at 2.25 points for the keyword combination "droopy eyelid diagnosis."
Out of the total 76 videos, 39 were produced or involved plastic surgeons, 20 involved ophthalmologists, 5 involved dermatologists, and 11 were produced by professionals from other fields or general practitioners. There were 4 videos produced by non-medical creators (
Table 6).
A comparison was made between the GQS and KMA’s scale of South Korean certified medical service providers on YouTube (
Table 7). The GQS for South Korean certified medical service providers showed an average of 2.93, and the KMA’s scale was 15.27. For uncertified creators, the GQS score was 2.66, and the KMA’s scale was 14.67, with no statistically significant difference. This was likely because even uncertified creators had videos involving medical professionals, as seen in the results.
Additionally, a comparison was made between the GQS and KMA’s scale scores of specialists (ophthalmologists and plastic surgeons) and non-specialists (general practitioners, dermatologists, and family medicine practitioners) (Figure 1). There was a statistically significant difference in GQS scores (1.53 for non-specialists vs. 3.12 for specialists) and KMA’s scale scores (13.33 for non-specialists vs. 15.47 for specialists) (P-value: GQS 0.036, KMA’s scales 0.01).
Figure 1.
Comparison of GQS and KMA scale.
Figure 1.
Comparison of GQS and KMA scale.
Discussion
The demand for information through social media is rapidly increasing, resulting in a surge in the quantity of information. [
3,
4] Therefore, verifying the accuracy of information is becoming increasingly important. Platforms like YouTube have greatly improved accessibility to medical information, but the increasing advertising costs have led to a proliferation of inaccurate information and one-sided opinions. [
5,
6] When videos are produced by South Korean certified medical service providers, there is a higher likelihood of the information having high reliability. However, there is also content produced without the involvement of medical professionals or without verification, and this necessitates concern about the reliability of such content. This problem has also been confirmed in other research results, such as a study that found YouTube lacks in providing sufficient information on topics related to dental pain. [
9,
10,
11,
12,
13] In this study, 5% of the videos contained inaccurate information. This indicates that non-medical individuals may disseminate inaccurate medical information or post non-medical content for commercial purposes. This is also evident in videos produced by physical therapists and Pilates instructors, not to mention in videos produced by creators outside the medical field. However, some studies have shown that high-quality content does exist on YouTube, such as research on wrinkles using botulinum toxin A. [
14] That's why it's important to have a standardized way to access quality content.
As observed in our study, the choice of keywords, such as 'ptosis' and 'droopy eyelid,' significantly influenced the ratio of certified medical service providers in the search results (80% for 'eyelid ptosis' and 20% for 'droopy eyelid'). Therefore, users are encouraged to make diligent efforts in selecting precise keywords. Patients may find it crucial to use accurate medical terminology when searching for information related to specific medical conditions or symptoms. Content creators, in turn, should prioritize the inclusion of commonly used medical terms when registering keywords for their content.
Two evaluation tools, the KMA’s scale and the GQS, were used in this study. The Korean Medical Association's Internet Health Information Certification Standards have a total of 20 items, including 7 criteria for website reliability, 5 for format evaluation, and 8 for content evaluation. Scientific reliability evaluation criteria are applied to 6 specific sub-items, while the remaining 19 items are evaluated using a "yes" or "no" rating system, for a total maximum score of 24 points. This tool provides a detailed breakdown of the criteria, accompanied by appropriate annotations, to facilitate accurate evaluations. It should be noted, however, that the scoring process is time-consuming.
The Global Quality Score is primarily a tool used to assess the educational quality of content. It is rated on a scale from 1 to 5, with a maximum score of 5 indicating high-quality videos. This tool offers an intuitive and efficient evaluation process, making it a popular choice for assessing YouTube videos in many studies. [
15,
16]
In this study, the GQS and KMA’s scale scores of uncertified creators were lower than those of certified medical service providers in Korea, although no statistically significant differences were observed. This suggests that even uncertified creators can produce videos of high credibility when medical professionals are involved. However, we analyzed the differences between the GQS and KMA’s scales in content produced by ophthalmologists, plastic surgeons, and non-specialists. Significant differences were found. Currently, blepharoplasty is one of the procedures performed by many non-specialists in South Korea. In particular, related plastic surgeries such as double eyelid surgery are easily accessible to general practitioners. However, most of the information and resources related to these non-specialist procedures are predominantly commercial and advertisement-focused content. As a result, users receive one-sided information. To address this issue, it is essential to establish methods that provide access to quality information and resources.
The GQS and KMA’s scales used in this study did not show statistical significance, but they generally showed a tendency to agree. (Fig 1.) The KMA’s scales, which was developed primarily for evaluating internet sites, evaluates 20 items related to reliability, format, and content. It requires more time to complete than the intuitive GQS, and opinions may vary among raters. However, from the perspective of evaluating quality content, it is worth noting that content with a high score on one side tends to have higher scores on the other side.
The government and medical organizations should make additional efforts to manage and improve the quality and reliability of medical information provided on YouTube and other online platforms. It is essential to establish criteria and guidelines for the involvement of medical professionals and for verifying the accuracy of information. In addition, providing self-regulatory tools for reporting can encourage content creators to act responsibly. In particular, this study identified one piece of harmful content that had accumulated 166,563 views, suggesting the need for swift sanctions. The role of YouTube as a channel to share medical information and provide educational content will continue to expand. Therefore, it is important to focus on verifying and increasing the reliability of medical information. To address this, not only doctors but also other healthcare professionals should participate, and efforts should be made to encourage hospitals to participate more actively in sharing information. Furthermore, the quality and reliability of the content shared on YouTube need to be improved.
Limitations
While this study provides results for specific keywords and platforms, further research is needed to cover a wider range of health topics and different platforms. Discussion and improvement of the evaluation criteria and scoring scales used in this study should also be considered. The scores were assigned based on the Korean Medical Association's criteria for certifying Internet health information, which primarily emphasize the source and scientific usefulness. Evaluating the accuracy of medical information would require a significant amount of time and expert involvement. In addition, the use of GQS for evaluation provides a simple 1-5 points scale and may benefit from more detailed evaluation criteria. The vast amount of information available online makes it challenging to secure sufficient resources for verification. Therefore, the development of simplified rating tools or the provision of certification marks or services by YouTube and similar platforms to content creators could generally improve reliability.
In addition, this study is based on search results in Korean, which may have limitations compared to videos produced abroad or in English. Information may be relatively scarce, and there may be country-specific limitations. Therefore, there is a need for standardized evaluation of videos in widely used English content. Although this study used GQS and KMA’s scales, research using other tools such as DISCERN, Video Poser Index (VPI) score, and JAMA score is also feasible. [
16,
17,
18,
19,
20,
21,
22]
Conclusions
Video platforms such as YouTube have improved access to medical information, but there is a need for caution regarding the reliability and quality of this information. Government agencies, medical organizations, and users need to work together to improve the quality of online health information. In particular, there is a need for mechanisms that allow for self-regulation to provide high-quality information. Users should also develop critical thinking skills and make an effort to seek out trustworthy information.
Funding Disclosure
The authors have no disclosure.
Data Availability
This study did not involve the use of medical data or patients’ information, and all data analyzed in this study were obtained from publicly available sources. The dataset used for this study can be directly accessed via YouTube (
https://www.youtube.com/).
References
- Stats, I.W. WORLD INTERNET USAGE AND POPULATION STATISTICS Available online:. Available online: https://internetworldstats.com/stats.htm (accessed on 23 September 2023).
- Trend, I. Internet trend, SNS. Available online: http://www.internettrend.co.kr/trendForward.tsp (accessed on 23 September 2023).
- Statista. Number of YouTube users worldwide from 2019 to 2028. Available online: https://www.statista.com/forecasts/1144088/youtube-users-in-the-world (accessed on 23 September 2023).
- Viralblog.com. YouTube Statistics. Available online: http://www.viralblog.com/research-cases/youtube-statistics/ (accessed on 23 September 2023).
- WARC. Ad spend on YouTube forecast to reach $30.4bn in 2023, growth to accelerate next year. Available online: https://www.warc.com/content/paywall/article/warc-curated-datapoints/ad-spend-on-youtube-forecast-to-reach-304bn-in-2023-growth-to-accelerate-next-year/en-gb/151404? (accessed on 23 September 2023).
- Madathil, K.C.; Rivera-Rodriguez, A.J.; Greenstein, J.S.; Gramopadhye, A.K. Healthcare information on YouTube: a systematic review. Health informatics journal 2015, 21, 173–194. [Google Scholar] [CrossRef] [PubMed]
- Bernard, A.; Langille, M.; Hughes, S.; Rose, C.; Leddin, D.; Veldhuyzen van Zanten, S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol 2007, 102, 2070–2077. [Google Scholar] [CrossRef] [PubMed]
- Singh, A.G.; Singh, S.; Singh, P.P. YouTube for information on rheumatoid arthritis--a wakeup call? J Rheumatol 2012, 39, 899–903. [Google Scholar] [CrossRef]
- Ayranci, F.; Buyuk, S.K.; Kahveci, K. Are YouTube™ videos a reliable source of information about genioplasty? J Stomatol Oral Maxillofac Surg 2021, 122, 39–42. [Google Scholar] [CrossRef]
- Hegarty, E.; Campbell, C.; Grammatopoulos, E.; DiBiase, A.T.; Sherriff, M.; Cobourne, M.T. YouTube™ as an information resource for orthognathic surgery. J Orthod 2017, 44, 90–96. [Google Scholar] [CrossRef] [PubMed]
- Özdal Zincir, Ö.; Bozkurt, A.P.; Gaş, S. Potential Patient Education of YouTube Videos Related to Wisdom Tooth Surgical Removal. J Craniofac Surg 2019, 30, e481–e484. [Google Scholar] [CrossRef] [PubMed]
- Sezici, Y.L.; Gediz, M.; Dindaroğlu, F. Is YouTube an adequate patient resource about orthodontic retention? A cross-sectional analysis of content and quality. Am J Orthod Dentofacial Orthop 2022, 161, e72–e79. [Google Scholar] [CrossRef]
- Topsakal, K.G.; Duran, G.S.; Görgülü, S.; Eser Misir, S. Is YouTube(TM) an adequate source of oral hygiene education for orthodontic patients? Int J Dent Hyg 2022, 20, 504–511. [Google Scholar] [CrossRef]
- Wong, K.; Doong, J.; Trang, T.; Joo, S.; Chien, A.L. YouTube Videos on Botulinum Toxin A for Wrinkles: A Useful Resource for Patient Education. Dermatol Surg 2017, 43, 1466–1473. [Google Scholar] [CrossRef]
- Erdem, M.N.; Karaca, S. Evaluating the accuracy and quality of the information in kyphosis videos shared on YouTube. Spine 2018, 43, E1334–E1339. [Google Scholar] [CrossRef] [PubMed]
- Esen, E.; Aslan, M.; Sonbahar, B.Ç.; Kerimoğlu, R.S. YouTube English videos as a source of information on breast self-examination. Breast cancer research and treatment 2019, 173, 629–635. [Google Scholar] [CrossRef] [PubMed]
- Atci, A.G. Quality and reliability of the information on YouTube Videos about Botox injection on spasticity. Romanian Neurosurgery 2019, 473–477. [Google Scholar] [CrossRef]
- Aydin, M.A.; Akyol, H. Quality of information available on YouTube videos pertaining to thyroid cancer. Journal of Cancer Education 2020, 35, 599–605. [Google Scholar] [CrossRef] [PubMed]
- Charnock, D. The Dıscern Handbook. University of Oxford and The British Library. Radcliffe Medical Press Ltd Abingdon, Oxon Erişim adresi: http://www discern org uk/discern pdf, Erişim tarihi 1998, 27, 2022.
- Kunze, K.N.; Cohn, M.R.; Wakefield, C.; Hamati, F.; LaPrade, R.F.; Forsythe, B.; Yanke, A.B.; Chahla, J. YouTube as a source of information about the posterior cruciate ligament: a content-quality and reliability analysis. Arthroscopy, sports medicine, and rehabilitation 2019, 1, e109–e114. [Google Scholar] [CrossRef] [PubMed]
- Kuru, T.; Erken, H.Y. Evaluation of the quality and reliability of YouTube videos on rotator cuff tears. Cureus 2020, 12. [Google Scholar] [CrossRef] [PubMed]
- Silberg, W.M.; Lundberg, G.D.; Musacchio, R.A. Assessing, controlling, and assuring the quality of medical information on the Internet: Caveant lector et viewor--Let the reader and viewer beware. Jama 1997, 277, 1244–1245. [Google Scholar] [CrossRef] [PubMed]
|
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).