Preprint
Article
Could You Evaluate Sounds in a Virtual Environment?: Evaluation Components of Auditory Experience in a Metaverse Environment
Altmetrics
Downloads
99
Views
56
Comments
0
A peer-reviewed article of this preprint also exists.
This version is not peer-reviewed
Submitted:
14 September 2023
Posted:
15 September 2023
You are already at the latest version
Alerts
Abstract
This study aims to develop an auditory experience evaluation questionnaire to improve the presence of metaverse environments, and to derive evaluation components considering auditory presence and auditory user experience (AUX) through a survey. After conducting a survey with a total of 232 participants, five evaluation components were extracted from auditory presence and AUX evaluation factors through principal component analysis (PCA) and reliability analysis (RA): 'realistic auditory background', 'acoustic aesthetics', 'consideration of acoustic control and accessibility', 'auditory utility and minimalist design', and 'auditory consistency'. In particular, although AUX evaluation factors such as 'ease of access to sound control' have limitations in improving the sense of presence, negative factors of presence such as 'distraction due to sound' can be improved by utilizing AUX evaluation factors, so it is judged that the sense of presence in the metaverse environments can be improved by enhancing the auditory sense of presence and AUX evaluation factors according to the composition of the five evaluation components derived in this study. This study can be used as a basis for developing an auditory experience evaluation questionnaire for the metaverse platform, creating sound design guidelines, and identifying sound development priorities.
Keywords:
Subject: Arts and Humanities - Other
1. Introduction
Metaverse technology is growing rapidly as it is used in various industries such as manufacturing, construction, education, healthcare, and social networking to enhance the participation of society members by recreating sensory experiences [1,2]. Especially in metaverse fields such as education, gaming, and healthcare, research on presence and immersion has been actively conducted to provide more realistic experiences, and such presence is an essential element of a metaverse environment [3,4,5,6,7,8]. There is an ongoing discussion on how to integrate sensory information such as visual and auditory information of users in realizing presence, and up to now, auditory factors have been used mainly as a supplement to visual information, but recently, realization of enhanced presence has become an important task, and realization of a lifelike auditory environment is becoming important [9]. In general, questionnaire evaluation methods such as presence questionnaire (PQ) and slater-usoh-steed presence questionnaire (SUS-PQ) are used to identify presence, and recently, auditory factors have been treated as a major factor to improve presence, and the need to evaluate them has emerged [10,11]. However, the existing questionnaires for evaluating presence have the limitation that it is difficult to evaluate auditory presence because there are fewer auditory examples compared to visual studies [12,13], and presence is evaluated by various factors such as emotion, ease, and familiarity that users experience in interacting with avatars and systems [14], but the existing questionnaires for evaluating auditory presence do not fully reflect these aspects. In particular, considering that the sense of presence is evaluated by factors such as the illusion created by the user’s sensory information, the cumulative user experience (UX) of using the system, and the experience formed after using the system, it seems that it can be used if evaluation factors related to the sense of presence are added to the existing UX evaluation factors [8,15,16], but the existing UX evaluation contents have a limitation that it is difficult to evaluate the auditory part due to its general characteristics [17].
Against this background, since UX evaluation factors affect the understanding of presence, and presence evaluation factors act to understand UX [15,16], extracting common components of AUX evaluation factors and auditory presence evaluation factors may be able to identify auditory experience evaluation factors considering presence enhancement in metaverse environments. Therefore, this study aims to develop an auditory experience evaluation questionnaire considering presence by conducting a literature review and primary and secondary expert interviews to improve the presence of the metaverse environment, and to derive the evaluation components considering auditory presence and AUX through questionnaires, PCA, and RA.
2. Literature Review
2.1. Presence and Auditory Characteristics of a Metaverse Environment
An aspect that should be considered to provide a more realistic experience in the field of metaverse is the sense of presence, which is considered an essential prerequisite for visual, tactile, as well as auditory factors [10]. The sense of presence considered in metaverse environments is related to sensations such as perceiving oneself as existing in a virtual space [18] or being transported to a virtual space [19], and the feeling of ‘being there,’ i.e., the subjective experience of the receiver focusing on and immersing in the virtual world [20]. In metaverse environments, presence is also related to the perception of the user and others with whom they interact, including the experience of perceiving virtual objects as if they were real [21], the degree to which the user identifies with the avatar's body [22], the degree to which they feel they are interacting with others [23,24], and the degree to which the recipient attributes personality to virtual humans, making them perceive them as social actors [25,26]. Therefore, the sense of presence can be categorized as the feeling of being transferred and immersed in a virtual world [18,19,20], the feeling of similarity to a real person [21,22], and the feeling of similarity in social behavior [23,24,25,26]. Table 1 shows the definitions and classification of presence by researchers.
In a metaverse environment, presence is interrelated to spatial, temporal, and acoustic properties related to hearing. Spatial characteristics describe where the sound is coming from, temporal characteristics describe when the sound is heard, and acoustic characteristics describe the pitch or intensity of the sound. These are the auditory characteristics of a metaverse environment because they help users recognize their location or orientation in the virtual space, determine their distance or relationship to objects or people, and feel emotions or moods [27]. The auditory evaluation factors of a metaverse environment can be derived by identifying the auditory factors that make the metaverse environment feel real [19]. In addition, since the equipment elements of the metaverse environment, subjective factors considering the user's personality characteristics and immersion tendencies, social factors arising from interactions with users, emotional factors, movement in the virtual environment, and familiarity can affect the sense of presence of the metaverse environment [14,28], the UX factors of existing accumulated digital devices, including the newly emerging metaverse environment, can be considered together [16]. Therefore, it is desirable to consider the auditory factors of the metaverse environment together with the AUX evaluation factors of digital devices that are familiar to the user and the auditory presence evaluation factors of the newly emerging metaverse environment.
2.2. Auditory Evaluation Factors in the Presence Evaluation Questionnaire for a Metaverse Environment
Since most of the evaluations of auditory presence in metaverse environments are technical evaluations of engineering sensors, and it is difficult to find data in the form of questionnaires for auditory presence evaluation, we first conducted a literature review of auditory evaluation factors in general presence evaluation questionnaires. In order of development date, the reviewed questionnaires include presence questionnaire (PQ), slater-usoh-steed presence questionnaire (SUS-PQ), reality judgment presence questionnaire (RJPQ), igroup presence questionnaires (IPQ), independent television commission-sense of presence inventory (ITC-SOPI), engagement, enjoyment and immersion questionnaire (E2IQ), and measurement, effects, conditions spatial presence questionnaire (MEC-SPQ).
PQ categorizes presence into three characteristics: involvement, immersion, and presence, and emphasizes sensory stimulation, participation, immersion, and interaction. The auditory presence evaluation factors include QN 6 (How much did the auditory aspects of the environment involve you?), QN 15 (How well could you identify sounds?), and QN 16 (How well could you localize sounds?), which are all about presence [20]. SUS-PQ assesses the sense of presence, which is the degree to which a person feels that he or she exists in a virtual environment rather than in an actual physical space. In particular, the degree to which a person is convinced that fictional information is real is viewed as presence, and it measures the degree of presence caused by the relevance of the user and the avatar. SUS-PQ consists of five evaluation factors: the experience and degree of 'being there', comparison of virtual and real images, association of the visual structure of the virtual environment, and the frequency of detecting virtual reality [28,29].
RJPQ categorizes presence into nine dimensions: reality judgment, presence, involvement, interaction, control, attention, realism, perceptual congruence, and expectations in virtual environment, and emphasizes natural interaction. The auditory presence dimension consists of three questions: QN 4 (How clear were the sounds in the virtual world?) on realism, QN 73 (To what extent did what you heard and the quality of the sound in the virtual world influence how real the experience seemed to you?) on reality judgment, and QN 76 (To what extent did the sounds influence how deep into the virtual world you went?) on presence [30]. IPQ categorizes presence into three factors: spatial presence, involvement, and realism, and covers a wide range of subjective experiences, immersion, and interaction, with no questions related to hearing [23]. ITC-SOPI categorized the sense of presence into four factors: sense of physical space, engagement, ecological validity, and negative effects, and found that not only the implementation of the physical environment, but also the overall usability, such as the attractiveness, naturalness, and trustworthiness of the content, could affect the sense of presence, and there were no questions related to hearing [31].
E2IQ categorizes the sense of presence into six factors: sensory factor, distraction factor, realism factor, control factor, pleasure factor, and satisfaction factor, and considers various UX aspects such as the attractiveness of the visual scene, the degree of distraction caused by noise outside the device, the realism and consistency of the sensation of virtual movement, and the immersion of the task. The auditory dimension is covered by QN 2 (To what extent did events such as noise occurring outside Crayolaland distract your attention from Crayolaland?), which is about distraction [32]. MEC-SPQ approaches the evaluation of presence from a UX-centered perspective rather than a technology-centered perspective, and its contents are important for redefining spatial presence, which has not been systematically addressed in existing studies. In the MEC-SPQ, there are questions that consider the user's state, behavior, and sense of presence, and in particular, the questionnaire on the sense of presence consists of questions about the sense of location of the user and the objects surrounding the user, and questions about whether the impression (feeling) received in the virtual environment allows the user to control the user's behavior [33]. As shown above, the evaluation factors of auditory presence in the questionnaire for evaluating presence in metaverse environments were examined and are shown in Table 2.
As shown in Table 2, the characteristics of the sense of presence questionnaire include sensory stimulation, engagement, immersion, interaction, attractiveness, distraction, and coherence, confirming that the multifaceted experience of a virtual world device can affect the sense of presence. In addition, because of examining the auditory presence evaluation factors in existing presence evaluations, most of the evaluation items related to hearing are very small, so it is very difficult to identify the auditory evaluation factors only from the evaluation questionnaire. Therefore, the auditory evaluation factors of the metaverse environment can be derived by identifying the auditory factors that make the metaverse environment feel real [19]. Based on the literature study, we reviewed the auditory presence factors of the metaverse environment as follows.
2.3. Prior Research on Auditory Presence in a Metaverse Environment
In addition to the implementation of acoustic technologies such as soundscapes that embody real-world sounds [34,35,36,37,38,39], studies on the quality of sound sources [34,39], studies on the stereophonics of sounds [34,39], studies on the spatial characteristics of sounds [38,39,39,40], studies on auditory cues that help with navigation tasks [39], and studies on auditory factors that match visuals [35,37], factors such as coherence between sounds [12] have been identified as influencing the sense of presence. Table 3 shows research examples of auditory presence evaluation factors in metaverse environments identified by researchers.
As shown in Table 3, we analyzed the research cases of auditory presence evaluation factors in metaverse environments and found that even the same kind of auditory presence evaluation factors are classified as different presence factors by individual researchers. In the case of 'spatial acoustics', some studies classified presence as presence [34,38,39], while others classified presence as immersion [37], presence [39], and social presence [40]. Studies on 'soundscape' classify presence as presence [35] and social presence [36], and studies on 'auditory factors related to visual experience' classify presence [35] and presence and immersion [37]. In other words, we found that auditory presence factors are not yet systematically organized as they are classified differently by different researchers. This result is consistent with a literature review that found it difficult to generalize about auditory presence [10]. Against this background, this study focuses on auditory presence, which is the most basic and essential element of virtual worlds, i.e., the feeling of 'being there', and the auditory sensation such as the transfer of sounds from the physical world to the virtual space, among various concepts of presence.
2.4. Reflections on the Evaluation Factors of AUX for Products and Services
UX is a branch of interaction research in human computer interaction (HCI) that refers to the total experience, including usability and emotions, experienced in interactions with products, systems, and services [41,42]. AUX factors are specifically about the auditory aspects of UX and include musical elements (melody, rhythm, harmony, timbre, etc.), auditory user interface (AUI) such as notifications that occur while using products and services; AUI such as notifications that occur while using products and services, auditory backgrounds such as ambient sounds and noises, and voice user interfaces (VUI) that assist in using the product [43]. AUX is also considered very important in the sense that it provides differentiated emotions and usability through hearing and imprints a company's identity and is often described as the counterpart of graphic user interface (GUI) [44].
Since the evaluation of UX is characterized by avoiding formalized formats and reconfiguring them to fit the unique features of the target audience [45], AUX factors can be described as evaluating the usability and emotions that users experience in auditory interactions with products, systems, and services [46]. However, research on auditory UX factors does not have the same standard as the existing visual UX evaluation, 'heuristic evaluation of user interfaces' [46], so we conducted a literature review to identify auditory UX factors. The literature review includes a study on deriving a meaningful auditory attractiveness measure for UX [48], a study on proposing an auditory interface UX scale [17], a study on evaluating acoustic measurements and noise abatement in general [49], a study on delivering consistent and differentiated experience and value in products and services [50], a study on AUX in UX evaluation of glasses-type hearable products [51], and an expert interview on designing AUX in a web or app environment [52]. From these results, we found that the existing studies on AUX evaluation factors are mainly distributed in products and services, and we derived AUX evaluation factors according to researchers as shown in Table 4.
As shown in Table 4, a literature review of AUX evaluation factors shows that they deal with the accessibility and intuitiveness of the UX, such as 'ease of access to sound control' [51] and ‘how easy it is to understand what the sound represents' [17], while the characteristics of auditory presence evaluation factors differ in that they aim to realize the sound of reality, such as ' spatial properties of sound' [12,38,39,40] and ‘similarity to real-world sounds’ [30], ‘implementing real-world sounds' [34].
3. Methods
In this study, we conducted three steps in addition to the literature review to derive auditory experience evaluation components in a metaverse environment. In Step 1, we grouped auditory factors through primary expert interviews; in Step 2, we created an auditory experience evaluation questionnaire that considers presence so that it is easy for the general public to understand through secondary expert interviews; and in Step 3, we conducted surveys and statistical analysis. The overall research process is shown in Figure 1.
3.1. Step 1: Primary Expert Interviews for Grouping Auditory Eveluation Factors
The primary expert interviews were conducted to group the auditory evaluation factors derived from the literature review. Participants in the primary expert interviews were selected as experts with a level of understanding of the existing literature review, and with at least three years of experience in the field of sound design (sound design, planning, composition, etc.) in a metaverse environment. The primary expert interview method was conducted in the form of listening to the opinions of experts using an Excel sheet, and the participant information is shown in Table 5.
3.2. Step 2: Secondary Expert Interviews to Create a Questionnaire to evaluate auditory experiences for Presence
The purpose of the secondary expert interviews was to define auditory presence in a way that the public can understand, and to create an evaluation questionnaire that the public can easily understand. The criteria for recruiting participants for the secondary expert interviews and the interview method were the same as the primary expert interviews, and additional experts in the fields of music education and music therapy were selected to help the public understand musical terms. We defined auditory presence based on the contents of the literature review [10] that it is difficult to generalize about auditory presence, and the participant information is shown in Table 6.
3.3. Step 3: Survey and Statistical Analysis
To derive an evaluation component of auditory experience that takes into account the presence of the metaverse environment, a survey was conducted using a 5-point Likert scale on the auditory factors that enhance the presence of the metaverse environment. We recruited participants who met the following criteria: they had used a VR device at least once to understand their familiarity with the virtual world, played games and metaverse platforms at least once or twice a week, and listened to music or media content at least 2-3 times a week to be able to discriminate auditory stimuli. The total number of participants was 232 (139 males and 93 females), with 67 participants aged 10-19, 60 participants aged 20-29, 60 participants aged 30-39, 25 participants aged 40-49, and 20 participants aged 50-59, with 80.6% of participants aged 10-39. The survey was also subjected to PCA to reduce the data with as little loss of information as possible. The PCA was performed in IBM SPSS® Statistics, with a varimax factor rotation, and a RA was performed on the derived values to confirm the appropriateness of the newly derived evaluation components of auditory experience
4. Results and Discussion
4.1. Results of Grouping for Auditory Evaluation Factors
From the primary expert interviews, the auditory evaluation factors identified in the literature review were grouped and presented in Table 7.
As shown in Table 7, the factors of ‘the concentration of contents due to hearing’ [20], and ‘presence with or without auditory backgrounds’ [30] were grouped as 'focusing the metaverse environment with sound’ because they are classified as the same concept of ‘sense of presence’ in the reviewed literature [28,30,39]. The factors of ‘the spatial characteristics of sound’ [12], ‘spatialized sound’ [38], ‘spatialized sound source’ [39], and binary audio (spatial acoustics) [40] were grouped into ‘implementing the spatial properties of sound' because they use the common concept of ‘space’ according to P1’s opinion. ‘Similarity to real-world sounds [30], ‘realization of real sounds’ [34], ‘reproduction of soundscapes’ [35], and ‘everyday soundscapes’ [36] were grouped ‘implementing realistic soundscapes in a metaverse environment’ because they use the common auditory concept of ‘real soundscapes ‘according to P1’s opinion. ‘Perceived location factor of sound’ [20], three-dimensional spatial representation of sound [34], and ‘three-dimensional representation of sound’ [37] were grouped into ‘three-dimensional perception of sound’ because they use the common concept of ‘location’ according to P2’s opinion.
The factors of ‘good sound’ [17], interesting sound [17], boring sound [17], unpleasant sound [48], wanting to stay again [49], and unpleasant sound [49] were grouped under the common term ‘willingness to return due to sound’ because of their common meaning of measuring the impression of sound, according to all expert opinions. The factors of ‘quality of sound’ [12], ‘sound source quality’ [34], ‘degree of sound echoing’ [48] ‘degree of sound softness’ [48], ‘loudness of sound’ [48,49], and ‘keeping sounds short and smooth elements’ [52] were grouped into ’the quality of the sound source’ because they use the common auditory concept of ‘quality of sound’ according to P2's opinion. It seems that the factors of ‘how confusing it is to hear the sound’ [17], ‘distractions from noise outside the metaverse environment’ [32], and the ‘attracting attention without being distracting’ [52] can be grouped into 'how distracting the sound is' as they use the common auditory concept of 'distraction' according to P1 and P3's opinions. The factors of ‘auditory clarity’ [30], ‘the right amount of sound resolution’ [51], and ‘how clear the sound is’ [51] can be grouped into ‘clarity of sound’ because they use the common auditory concept of ‘resolution of sound’ in P2 and P3's opinions. The factor of ‘delivering the sound you expect visually’ [35], ‘ambience sounds to match the visual experience’ [37], and ‘degree of harmony between surroundings and sound’ [49] could be grouped together as ' sound experiences that live up to visual expectations’ as they use a common auditory concept in the sense that 'sight and sound should match' according to P3's opinion.
The factors of ‘consistency between sounds’ [12], ‘how difficult it is to understand the sound being varied’ [17], and ‘consistency between sounds within products and services’ [50] are believed to use a common auditory concept of ' how consistent the overall sound is in a sound variation situation’, and thus can be grouped into 'the degree to which the overall sound is consistent in the context of sound variation', by P3's opinion. The factors of ‘how much the sound helps’ [17], ‘how easy it is to understand the sound’ [17], ‘how easy it is to understand what the sound represents’ [17], ‘matching sound and meaning’ [17], ‘identification of the sound’ [20], and ‘auditory cues to help you navigate’ [39] can be grouped under ‘auditory cues to help you navigate’ as they share a common auditory concept of the role of sound in metaverse environments as an alternative to data navigation and visual interfaces based on a literature review [17]. As mentioned above, we conducted primary expert interviews and found that the grouping within the auditory presence evaluation factor was dominated by the tendency to realize a realistic auditory experience, while the grouping within the AUX evaluation factor was dominated by acoustic characteristics, such as the sufficiency of volume size control, the distribution of the sound range, and the ease of access to sound control. In the case of grouping between the auditory presence and AUX evaluation factors, we found that the factors covered by the existing AUX evaluation factors, such as 'auditory utility to help navigation', are also covered by auditory presence, confirming that the scope of auditory presence can be expanded.
4.2. Creatinge an Auditory Experience Evaluation Questionnaire that Considers the Public Understanding of Presence
Secondary expert interviews were conducted to explain the auditory presence and AUX evaluation factors in the public’s terms, and to create evaluation questionnaires that make it easier for the public to understand presence. As a result of the secondary expert interviews, the definition of auditory presence in a metaverse environment is shown in Table 8, the considerations for creating an evaluation questionnaire for auditory experiences with presence are shown in Table 9, and the terminology for the evaluation questionnaire is shown in Table 10.
Based on the secondary expert interviews shown in Table 8, we found that the experts' opinions on the question ‘How can we define auditory presence in a metaverse environment?’ were generally described as related to the auditory sensation of ‘being there,’ i.e., the transfer of sounds from the physical world to a virtual space, which is consistent with the theories advocated in the literature review{18, 19}, and we considered them when creating the presence evaluation questionnaire.
As shown in Table 9, we found that the auditory presence and AUX factors are subject to subjective interpretation and should be accompanied by examples, and that the examples should be described in as intuitive and everyday terms as possible, rather than in technical terms.
As shown in Table 10, we found that existing terms such as ‘three-dimensional’ can be concretized into ‘location and distance identification,’ and terms such as ‘approach to sound control’ can be simplified to be understood by the general public, such as ‘sound you want to reduce. After completing the expert interviews in Step 1 and Step 2, the questionnaire for evaluating the auditory experience with a final sense of presence is shown in Table A1 in the Appendix.
4.3. Results of the Survey and Statistical Analysis
The Kaiser-Meyer-Olkin (KMO) and Bartlett's test of sphericity indicators indicate the degree to which the correlation between variables is well explained by other variables, and low values suggest that the variables are not appropriate for PCA. The KMO value for the survey in this study is 0.830, and the probability of significance (p) is less than the significance level (0.001), so the model fit is high. RA shows that a Cronbach's α value of 0.6 or higher is generally accepted as reliable, and the reliability of this study is very high with a total component of Cronbach's α=0.896, and the Cronbach's α of component 1, component 2, component 3, component 4, and component 5 was 0.867, 0.832, 0.829, 0.903, and 0.744 respectively. In addition, the eigenvalue of the extracted components indicates the amount of variance in the component, with a value greater than 1 being significant, and all five components were found to be the main auditory components considered in the metaverse environment. Commonality values below 0.4 are subject to removal, but all of the components were above 0.5, so no components were removed, and the results are shown in Table 12.
From the results in Table 12, the cumulative variance of the five components, including 'realistic auditory background' and 'acoustic aesthetics', is 70.458%, which can explain more than 70% of the auditory experience of the metaverse environment, so it is believed that improving these components can improve the sense of presence. Component 1, which is explained by 35.320% (variance) of the total factors, consists of QNs 1, 2, 10, 5, and 16, and has the characteristics of ‘realistic auditory background implementation’ in that it considers the positional and spatial characteristics of sounds in the metaverse environment and the implementation of everyday soundscapes and auditory backgrounds (such as background music and everyday noises) that can evoke a place. In particular, considering that 'Component 1' is derived from the AUX factors 'harmonization between digital and natural sounds' (QN 16) and 'auditory presence factors related to constructing a realistic sound environment' (QNs 1, 2, 10, 5), it can be interpreted that the auditory environment of the metaverse should not be implemented as it is in reality, but should seek harmonization with existing digital sounds, which can enhance the sense of presence.
Component 2, described as 12.330% of the total factor, consists of QNs 3, 4, 18, and 20. Component 2 is characterized as ‘acoustic aesthetics’ in terms of fewer bad sounds, clearer sounds, auditory discomfort, and user revisits due to sounds in the metaverse environment. The reason why factors such as (QNs 3, 4, 18, and 20) were derived as ‘Component 2’ is that the auditory presence factor (QNs 3, 4), which is related to acoustic quality, considers ‘willingness to return due to sound’ (QN 20), which tends to be a strong AUX evaluation factor These results suggest that users' "willingness to return due to sound" (QN 20) does not directly increase presence, but can function as a checklist for presence by ensuring that "Component 2" has no issues with sound quality if metaverse use is low (QNs 3, 4).Comprising QNs 6, 19, 15, and 14, 'Component 3', which explains 8.590% of the total factor, can be described as 'consideration of acoustic control and accessibility' in terms of the granularity of sound volume and sound effect control in the metaverse environment and the ability to turn different kinds of sounds on and off. In particular, since 'Component 3' is the component with the most AUX evaluation factors, including 'Sufficiency of volume control' (QN 19), 'ease of access to sound control' (QN 14), and the auditory presence evaluation factor 'distraction from sound' (QN 6), it is judged that the AUX evaluation factor itself does not improve the sense of presence, but the sense of presence can be improved by controlling 'distraction from sound' (QN 6) through acoustic control and accessibility considerations. Comprised of QNs 11, 12, and 9, 'Component 4', explained as 7.848% of the total factor, is characterized by 'auditory utility and minimalist design' in the sense that AUI such as notifications, signals, etc. that help navigate in the metaverse environment should be designed in such a way that users do not notice the sound looping. In particular, ‘Component 4’ states that ‘signals that help with navigation’ (QN 11, 12) are not just AUI, but auditory signals that are experienced in the real world (such as car horns), and if the auditory signals are mechanically repeated, the sense of presence is diminished. Therefore, the auditory design of the metaverse environment should be minimalist, with sounds that can be useful to the user, rather than importing all the sounds of the real world.
Comprising QNs 13, 8, 17, and 7, 'Component 5' explains 6.371% of the total factor and can be described as 'auditory consistency', considering the coherence of sound and visual motion and atmosphere in the metaverse environment, as well as the coherence of sound changes in the content. 'Component 5' is grouped into 'sound matching visual motion timing' (QN 7), 'sound experience meeting visual expectations' (QN 8), 'relevance of sound to ideas' (QN 17), and 'degree of consistency of overall sound across sound variations' (QN 13), which is a strong tendency of AUX evaluation factors, suggesting that the unified experience of sight and sound covered by traditional auditory presence should be harmonized with the auditory experience across the metaverse environments. Based on these results, it is difficult to conclude that AUX factors improve presence on their own because they tend to be more functional, such as acoustic control and accessibility, but they can improve presence if they improve factors that reduce auditory presence factors, such as the degree of distraction from sound. In addition, since auditory presence factors tend to embody the auditory experience of the real environment in the metaverse, such as realizing spatial characteristics and matching the visual atmosphere, we believe that it is possible to improve presence by expanding its scope through AUX evaluation factors such as the degree of consistency of the overall sound in sound variation situations. Therefore, it is believed that improving the auditory experience of the metaverse environment based on the five components derived in this study can improve the user's sense of reality, and furthermore, it can be used as a basis for creating sound design guidelines and identifying sound development priorities.
5. Conclusion
As the realistic experience of users in a metaverse environment becomes important, we developed an evaluation questionnaire for auditory experience considering presence to improve the presence of the metaverse environment, and derived evaluation components considering auditory presence and AUX through a survey, PCA, and RA. As a result of grouping the auditory evaluation factors through primary expert interviews (Step 1), we found that the tendency to realize a realistic auditory experience was strong in the grouping within an auditory presence evaluation factor, and that the grouping within the AUX evaluation factor had strong acoustic characteristics such as ‘sufficiency of volume size control’, ‘distribution of the sound range ‘ and 'ease of access to sound control'. In the case of grouping between the auditory presence and AUX evaluation factors, we found that the factors covered by the existing AUX evaluation factors, such as ‘auditory cues to help you navigate’, are also covered by auditory presence, confirming that the evaluation area of auditory presence can be expanded.
Through secondary expert interviews (Step 2), we created an evaluation questionnaire that can be easily understood by the public and confirmed that the following should be included in the auditory experience evaluation questionnaire: consideration of auditory sensation such as the transfer of sounds from the physical world to the virtual space, attachment of auditory examples to the questionnaire, and use of intuitive and everyday terms. We also identified that existing broad terms such as three dimensions could be refined to ‘location and distance identification’ and that difficult terms such as 'ease of access to sound control' could be simplified to make them more understandable to the public, such as ‘sounds you want to reduce’. Among the five components extracted from the results of survey, PCA and RA (Step 3), the eigenvalue for 'realistic auditory background' was the highest, so it is a prioritized evaluation component of auditory experience in the metaverse environments. In particular, although AUX evaluation factors such as 'ease of access to sound control' have limitations in improving the sense of presence, negative factors of presence such as 'distraction due to sound' can be improved by utilizing AUX evaluation factors, so it is judged that the sense of presence in the metaverse environment can be improved by improving the auditory sense of presence and AUX evaluation factors according to the composition of the five evaluation components derived in this study.
Acknowledgments
This paper is based on research supported by the Korea Institute for Advancement of Technology (KIAT), funded by the Korean government (Ministry of Trade, Industry and Energy, MOTIE) in 2023 (P0012725).
Appendix A
Table A1.
Questionnaire to evaluate auditory experiences for presence according to primary and secondary expert interviews
Table A1.
Questionnaire to evaluate auditory experiences for presence according to primary and secondary expert interviews
| Auditory factors | QN | Questionnaire contents for evaluating auditory experiences |
| Implementing the spatial properties of sound | 1 | Being able to distinguish places in a virtual world based on sound alone. (e.g., you can tell if you're in a library or a cave by the echoes of the sounds.) |
| Three-dimensional perception of sound | 2 | You can orient yourself in a virtual world by listening to sounds. (e.g., I can tell forward/backward/left/right/up/down by the sounds I hear) |
| The quality of the sound source | 3 | Less ‘bad sound’ in the virtual world (e.g. poor sound quality, cracking, tearing, reverberant, loud, harsh, etc.) |
| How clear the sound is | 4 | Sound is clear in the virtual world. (e.g., crisp sound, clean sound) |
| Focusing the metaverse Environment with Sound | 5 | Auditory backgrounds are immersive in virtual worlds. (e.g., background music, everyday noises, etc.) |
| How distracting is the sound? | 6 | Different types of sounds can be turned on and off in the virtual world (on/off). (e.g., turn on and off notifications, button clicks, the other person's voice, background music, and more) |
| Sound to match visual motion timing | 7 | The timing of motion and sound in the virtual world is correct. (e.g., in a hypothetical running situation, the sound of footsteps matches the motion of running) |
| Sound experiences that live up to visual expectations | 8 | The virtual world plays music that matches the background. (e.g., grandiose music on a grandiose background, cutesy music on a cutesy background, etc.) |
| Non-looping auditory experiences | 9 | Notice that the sound is looping in the virtual world. (e.g., when you notice the sound of waves repeating in an imaginary ocean.) |
| Implementing Realistic Soundscapes in a metaverse Environment | 10 | The noise of everyday life is heard in the virtual world. (e.g., keyboard sounds, air conditioning blowing, copier sounds, etc. from a fictional office space.) |
| Auditory cues to help you navigate | 11 | There are sounds that can help you navigate in virtual worlds. (e.g., if you're wandering around in a virtual world trying to find your school, look for the school in the direction of the bell.) |
| 12 | Hear notifications in the virtual world and know what to do. (e.g., message notifications, alert notifications, quest notifications, etc. and know what they are by just hearing them) |
|
| How consistent the overall sound is in a sound variation situation | 13 | Just by listening to music or certain sounds in the virtual world, you can tell which company made it. |
| ease of access to sound control' | 14 | When you need to adjust the volume in the virtual world, you can easily find the volume icon. |
| Accessibility for those with sensory sensitivities | 15 | Adjust the effect of sounds in the virtual world (high, low, soft, ringing, etc.). |
| Harmonize between digital and natural sounds | 16 | Artificial and natural sounds blend together in the virtual world (e.g., artificial music blends with the natural sound of the wind as you run through a virtual forest). |
| Relating sounds to ideas | 17 | You can hear sounds and see images in the virtual world. (e.g., when you hear thunder, you think of rain.) |
| Distribution of the sound range | 18 | Sound is offensive in virtual worlds. (e.g., during a relaxing playthrough, the sound suddenly gets louder) |
| Sufficiency of Volume Size Control | 19 | When you adjust the volume (sound) in the virtual world, you can adjust it as much as you want. |
| Willingness to return due to sound | 20 | The music and sounds of the virtual world are memorable and I want to play them again. |
References
- Cheng, X.; Zhang, S.; Fu, S.; Liu, W.; Guan, C.; Mou, J.; Ye, Q.; Huang, C. Exploring the metaverse in the digital economy: an overview and research framework. J. Electron. Bus. Digit. Econ. 2022, vol.1, pp. 206–224. [CrossRef]
- Tocci, L. What is the metaverse? An explanation and in-depth guide. Available online: https://www.techtarget.com/whatis/feature/The-metaverse-explained-Everything-you-need-to-know (accessed on 26 August 2023).
- Berkman, M.I.; Akan, E. Presence and Immersion in Virtual Reality. In: Lee, N. (Ed.), Encyclopedia of Comput. Graph. Games. Springer: Cham, 2019. [CrossRef]
- Hwang, I. The Effect of Metaverse Telepresence on Intention to Continuous Use Through Interaction Value: The Moderation of Platform Trust. J. Digit. Content Soc. 2022, vol. 23(12), pp. 2469–2480.
- Kang, M.; Kim, G.; Shin, Y.; Kim, D. Effects of Learning Presence and Flow on the Learning Satisfaction of Learner's in a Graduate School of Education Class Utilizing the Metaverse Platform Gather Town. Educational Research 2022, vol. 35(1), pp. 83–116.
- Blackman, T. Virtual Reality and Videogames: Immersion, Presence, and the Performative Spatiality of ‘Being There’ in Virtual Worlds. Soc. Cult. Geogr. 2022. [CrossRef]
- Lee, J.H. VR System Environment Technologies and User Input Elements. J. Korean Soc. Des. Cult. 2018, vol. 24(2), pp. 585–596.
- Sheridan, T.B. Further Musings on the Psychophysics of Presence. Proc. IEEE Int. Conf. Syst. Man Cybern., San Antonio, TX, USA, 1994, vol. 2, pp. 1073–1077. [CrossRef]
- Song, E. J. Research Trends on Virtual Reality User Experience (VRUX). Korea information processing society review vol.29(2) , 2022, pp.4 – 13.
- Kern, A.C.; Ellermeier, W. Audio in VR: Effects of a Soundscape and Movement-Triggered Step Sounds on Presence. Front. Robot. AI, 2020, vol. 7(20). [CrossRef]
- Zhang, S.; Feng, X.; Shen, Y. Quantifying Auditory Presence Using Electroencephalography. Appl. Sci. 2021, vol. 11(10461). [CrossRef]
- Larsson, P.; Väljamäe, A.; Västfjäll, D.; Tajadura Jimenez, A.; Kleiner, M. Auditory-Induced Presence in Mixed Reality Environments and Related Technology. In: The Eng. Mix. Real. Syst.; Springer, 2010, vol. 1, pp. 143–163. [CrossRef]
- Roth, Z. T.; Thompson, D. R. Usability of Sound-Driven UIs. Undergrad. Hon. Theses Comput. Sci. Comput. Eng.; Univ. of Arkansas: Fayetteville, AR, USA, 2018. https://scholarworks.uark.edu/csceuht/58.
- Moinnereau, M.A.; de Oliveira, A.A.; Falk, T.H. Immersive Media Exp.: Survey of Methods & Tools for Human Influential Factors Assess. Qual User Exp, 2022, vol.7, pp.5. [CrossRef]
- Won, M.J.; Park, S.; Kim, C.; Lee, E.; Whang, M.C. Study on Eval. of Visual Factor for Meas. Subjective Virtual Realization. Sensibility Sci., 2012, vol.15, pp.389–398.
- Kim, G. J. Study on the Recon. & Charac. of VR Media Presence. Dept. of Commun. Grad. Sch. of Sogang Univ., 2023.
- Tomlinson, B.J.; Noah, B.E.; Walker, B.N. BUZZ: Auditory Interface UX Scale. In Proc. of the ACM SIGCHI Conf. on Human Factors in Comput. Syst. (CHI 2018), Montreal, Canada, 2018, pp. 21–26. [CrossRef]
- Kim, T.; Biocca, F. Telepresence via TV: Two Dims. of Telepresence & Conn. to Memory & Persuasion. J. Comput.-Med. Commun., 1997, vol.3. [CrossRef]
- Heeter, C. Being There: Subjective Exp. of Presence. Presence: Teleop. and Virt. Environ., 1992, vol.1, pp.262–271. [CrossRef]
- Witmer, B.G.; Singer, M.J. Meas. Presence in Virtual Environ.: A Presence Ques. Presence: Teleop. and Virt. Environ, 1998, vol.7, pp.225–240. [CrossRef]
- Rizzo, A.A.; Wiederhold, M.D.; Buckwalter, J.G. Basic Issues in Use of Virtual Environ. for Mental Health Apps. In Virtual Environ. in Clin. Psych. and Neurosci.; Riva, G., Widerhold, B.K., Molinari, E., Eds.; IOS Press: Amsterdam, The Netherlands, 1998, pp. 22–42.
- Slater, M.; Wilbur, S. Framework for Immersive Virtual Environ. (FIVE): Role of Presence in Virtual Environ. Presence: Teleop. and Virt. Environ, 1997, vol.6, pp.603–616.
- Schubert, T.; Friedmann, F.; Regenbrecht, H. Experience of Presence: Factor Analytic Insights. Presence, 2001, vol.10, pp.266–281. [CrossRef]
- Short, J.; Williams, E.; Christie, B. Soc. Psych. of Telecom. John Wiley & Son: London, UK, 1976.
- Lemish, D. Rules of Viewing TV in Public Places. J. of Broad., 1982, vol.26(4), pp.757–781. [CrossRef]
- Takiff, J. Reality? Well … Virtually. Philadelphia Daily News, 1993.
- Park, S.; Kim, S.; Lee, J. H. Dev. a Service of the Audio Metaverse Concept. In Proc. of the Korea HCI Soc. Conf., 2023, pp. 928–933.
- Usoh, M.; Catena, E.; Arman, S.; Slater, M. Using Presence Ques. in Reality. Presence: Teleop. and Virt. Environ, 2000, vol.9. [CrossRef]
- Usoh, M.; Arthur, K.; Whitton, M.C.; Bastos, R.; Steed, A.; Slater, M.; Brooks, F.P. Walking > Walking-in-Place > Flying in Virtual Environ. In Proc. of the 26th Annu. Conf. on Comp. Graphics and Inter. Tech., 1999.
- Baños, R.M.; Botella, C.; Garcia-Palacios, A.; Villa, H.; Perpiña, C.; Alcañiz, M. Presence & Reality Judgment in Virtual Environ.: A Unitary Construct? CyberPsychology & Behav, 2000, vol.3(3), pp.327–335. [CrossRef]
- Lessiter, J.; Freeman, J.; Keogh, E.; Davidoff, J. A Cross-Media Presence Ques.: The ITC-Sense of Presence Inventory. Presence: Teleop. and Virt. Environ., 2001, vol.10(3), pp.282–297. [CrossRef]
- Lin, J.J.W.; Duh, H.B.L.; Parker, D.E.; Abi-Rached, H.; Furness, T.A. Effects of FOV on Presence, Enjoyment, Memory, & Simul. Sickness. In Proc. IEEE Virtual Reality, Orlando, FL, USA, 2002, pp.164–171. [CrossRef]
- Vorderer, P.; Wirth, W.; Gouveia, F.; Biocca, F.; Saari, T.; Jäncke, L.; Böcking, S.; Schramm, H.; Gysbers, A.; Hartmann, T.; Klimmt, C.; Laarni, J.; Ravaja, N.; Sacau, A.; Baumgartner, T.; Jäncke, P. MEC-SPQ: Short Doc. and Instr. for Appl. Report to the EC, Project Presence: MEC (IST-2001-37661), 2004.
- Michitaka Hirose. Virtual Reality. Industrial Books, 1993, p.137.
- Serafin, S.; Serafin, G. Sound Design to Enhance Presence in Photorealistic Virtual Reality., 2004.
- Baharin, H.; Mühlberger, R. Hearing but not Listening: Auditory Icons & Presence. In Proc. of the 2010 Intl. Conf. on User Sci. and Eng. (i-USEr), Shah Alam, Malaysia, 2010, pp.179–184. [CrossRef]
- Lee, S.; Lee, J.-H. Does Spatial Attribute between 2D & 3D Virt. Spaces Make Different User Immersion? In Proc. of the 9th Intl. Conf. on Comp. & Auto. Eng. (ICCAE '17), New York, NY, USA, 2017, pp.56–59. [CrossRef]
- Hendrix, C.; Barfield, W. Presence in Virt. Environ. as Function of Visual & Auditory Cues. In Proc. Virt. Reality Annu. Intl. Symp. '95, Research Triangle Park, NC, USA, 1995, pp.74–82. [CrossRef]
- Hendrix, C.; Barfield, W. Sense of Presence within Auditory Virt. Environ. Presence: Teleop. and Virt. Environ., 1996, vol.5(3), pp.290–301. [CrossRef]
- Kiridoshi, A.; Otani, M.; Teramoto, W. Spatial Auditory Presentation Induces Social Simon Effect. Sci Rep, 2022, vol.12, pp.5637. [CrossRef]
- Moon, J.; Lim, S.; Park, C.; Lee, I.; Kim, J. Conceptual Study on UX in HCI: Definition of UX & Intro. of CX (Co-Experience). J. of the HCI Soc. of Korea, 2018, vol.24(3), pp.441–457.
- Law, E.; Schaik, P. Modelling UX–Research & Practice Agenda. Interact. Comput., 2010, vol.22(5), pp.313–322.
- Namkung, K.; Pan, Y.H. Study on Structural Def. of Sound Identity in Corporate Auditory Comm. J. of Brand Design Assoc. of Korea, 2018, vol.16(3), pp.246–254.
- Namkung, K.; Heo, J. Case Study on Auditory UX as Component of Brand Identity. In Proc. of HCI KOREA, 2017, pp.629–632.
- Yoo, H.S.; Ju, D.Y. Auditory UI Guideline for Emotional UX. In: Marcus, A., Wang, W. (eds) DUXU 2017. LNCS, Springer, Cham, 2017, vol 10289. [CrossRef]
- Kim, J. Empirical Study on UX Evaluations: Focusing on Website Improvement. Master’s thesis, Grad. Sch. of Policy Industry, Sookmyung Women's Univ., Republic of Korea, 2016. [Online]. Avail.: https://dl.nanet.go.kr/.
- Nielsen, J.; Molich, R. Heuristic Evaluation of UI. In Proc. of the SIGCHI Conf. on Human Factors in Computing Systems (CHI '90), New York, NY, USA, 1990, pp.249–256. [CrossRef]
- Boos, B.; Brau, H. Erweiterung des UEQ um die Dim. Akustik und Haptik. Mensch und Comp. 2017 - Usab. Prof. 2017. [CrossRef]
- Aletta, F.; Guattari, C.; Evangelisti, L.; Asdrubali, F.; Oberman, T.; Kang, J. Compatibility of “Method A” & “Method B” in ISO/TS 12913-2:2018 for Urban Soundscape. Appl. Acoust., 2019, vol.155, pp.190–203. [CrossRef]
- Namkung, K. Sound Identity Structure in Comm. Env. Between Company & User. Dept. of Exp. Des. Grad. Sch. of Techno Des., Kookmin Univ. Seoul, Korea, 2019.
- Seok, A.; Choi, Y. Study on UX Evaluation of Glass-Type Hearables: Based on Zungle Panther. J. of Integ. Des. Res., 2020, vol.19(4), pp.75–92. [CrossRef]
- Blackburn, R.; Beercroft, A. Testing Usability of Sound. Avail. online: usertesting.com, [Online]. Avail.: https://www.usertesting.com/resources/topics/testing-sound-usability (accessed on Feb. 28, 2023).
Figure 1.
Overall Research Process.
Table 1.
Definition and Classification of Presence by Researchers
| Researchers | Definition of presence | Classification of presence |
| Kim et al. (1997) [18] | Subjective sensations, such as being transported to a virtual space | A sense of transference and immersion in the virtual world |
| Heeter (1992) [19] | ||
| Witmer et al. (1998) [20] | The cognitive process of focus and engagement and the subjective experience of the receiver | |
| Rizzo et al. (1998) [21] | How much the avatar's body identifies with the user | Similarity to real people |
| Slater et al. (1997) [22] | The experience of feeling that a person exists in a virtual environment | |
| Schubert et al. (2001) [23] | How much you feel connected to others | Similarity to social behavior |
| Short et al. (1976) [24] | ||
| Lemish et al. (1982) [25] | Recognizing virtual humans as social actors | |
| Takiff (1993) [26] |
Table 2.
Characteristics of Presence Evaluation Questions and Auditory Presence Evaluation Factors in Metaverse Environments by Evaluation Questionnaire
Table 2.
Characteristics of Presence Evaluation Questions and Auditory Presence Evaluation Factors in Metaverse Environments by Evaluation Questionnaire
| Type of questionnaire | Characteristics of presence evaluation questions | Auditory presence evaluation factors | Classification of presence |
| PQ [20] | Sensory stimulation, engagement, immersion and interaction | Whether you focus on content due to hearing (QN 6) | Presence |
| Identification of sounds (QN 15) | |||
| location of the sound recognized (QN 16) | |||
| SUS-PQ [28] | Comfort, naturalness, and immersion | No auditory presence factor | |
| RJPQ [30] | Natural interactions | Auditory clarity (QN 4) | Realism |
| Similarity to real-world sounds (QN 73) | Reality judgment | ||
| Focus on content due to hearing (QN 76) | Presence | ||
| IPQ [23] | Subjective experience, immersion, and interaction | No auditory presence factor | |
| ITC-SOPI [31] | Appeal, naturalness, and credibility | No auditory presence factor | |
| E2IQ [32] | Attractiveness, distraction, realism, consistency and immersion | Distractions from noise outside the metaverse environments (QN 2) | Distraction |
| MEC-SPQ [33] | User state, behavior, and presence | No auditory presence factor | |
Table 3.
Auditory Presence Evaluation Factors for Metaverse Environments Identified by Researchers
| Researchers | Contents of the study | Auditory presence evaluation factors | Classification of presence |
| Hirose (1993) [34] | Theory and technology on the five senses that enhance the sense of reality in the metaverse environment | Implementing real-world sounds | Presence |
| Creating a three-dimensional space for sound | |||
| Sound source quality | |||
| Serafin et al. (2004) [35] | The function and role of Soundscapes in a metaverse environment | Delivering the sound that you expect visually | Presence |
| Auditory experiences that match motion but don't loop | |||
| Recreating soundscapes | |||
| Baharin et al. (2010) [36] | How everyday sounds affect the sense of presence in a metaverse home environment | Everyday soundscapes | Social presence |
| Larsson et al. (2010) [12] | A conceptual framework for the relationship between sound and immersion, illusions of place, illusions of plausibility, and virtual body ownership in a metaverse environment | Spatial properties of sound | Presence |
| Presence with and without auditory backgrounds | |||
| Consistency between sounds | |||
| Sound quality | |||
| Lee et al. (2017) [37] | How sound matches the visual experience in a metaverse environment to create a sense of presence | Three-dimensional representation of sound | Immersion |
| Ambience sounds to match the visual experience | Presence, immersion | ||
| Hendrix et al. (1995) [38] | Elements of presence related to sight and sound in a metaverse environment | Spatialized sounds | Presence |
| Hendrix et al. (1996) [39] | The role of visual and auditory cues in perceived presence in a metaverse environment | Spatialized sound sources | Realism |
| Auditory cues to help you navigate | Presence | ||
| Kiridoshi et al. (2022) [40] | How auditory spatial information affects users in a metaverse environment | Binary audio (spatial acoustics) | Social presence |
Table 4.
AUX Factors Identified by Researchers
| Researchers | Research summary | Evaluation factors for AUX | Research categories |
| Boos et al. (2017) [48] | Deriving meaningful auditory attractiveness measures from UX | Loudness of sound | AUX Scale |
| Unpleasantness of hearing | |||
| Degree of sound echoing | |||
| Degree of sound softness | |||
| Tomlinson et al. (2018) [17] | Auditory interface proposal of UX scale (interpretation, meaning and enjoyment) | How much the sound helps | AUI Scale |
| How interesting the sound is | |||
| How good the sound is | |||
| How easy it is to understand the sound | |||
| Relevance of sound and ideas | |||
| Matching sound and meaning | |||
| How difficult it is to understand the sound being varied | |||
| How pleasant the sound is to listen to | |||
| How boring it is to hear the sound | |||
| How confusing it is to hear the sound | |||
| How easy it is to understand what the sound represents | |||
| Aletta et al. (2019) [49] | Acoustic measurements and noise abatement in general using ISO standard (Method B of the ISO/TS 12913- 2:2018) | Loudness of sound | Soundscape evaluation |
| Unpleasantness of sound | |||
| Degree of harmony between surroundings and sound | |||
| How likely you are to stay again | |||
| Namkung (2019) [50] | Delivering consistent and differentiated experiences and value across products and services | Consistency between sounds within products and services | Sound identity structural diagram |
| Seok et al. (2020) [51] | AUX evaluation factors for hearable type of glasses | Ease of access to sound control | AUX evaluation |
| The right amount of sound resolution | |||
| How clear the sound is | |||
| How much is enough to resize the volume | |||
| How rich the range of the sound is | |||
| Blackburn et al. (2023) [52] | Interviewing an expert for designing AUX design in web or app environments | Accessibility for those with sensory sensitivities | AUX design guideline |
| Blending the organic and digital | |||
| Keeping sounds short and smooth | |||
| Attracting attention without being distracting |
Table 5.
Participants Information of Step 1
| Participants | Gender | Age | Work experience | Activity industry | Profession/Occupation |
| P1 | Male | 45 | 15 years | About AUX university labs | Professor and sound designer |
| P2 | Male | 26 | 5 years | Game company | Sound designer |
| P3 | Male | 30 | 10 years | Entertainment company | Service planning and sound designer |
Table 6.
Participants Information of Step 2
| Participants | Gender | Age | Work history | Activity industry | Profession/Occupation |
| P4 | Male | 31 | 6 years | Doctoral graduate students | Music educator and sound designer |
| P5 | Female | 33 | 3 years | Music therapy practice in the university hospital | Music therapist and composer |
| P6 | Male | 32 | 7 years | Game company | Sound designer |
| P7 | Female | 31 | 10 years | Healthcare company | Composer and sound designer |
| P8 | Male | 26 | 3 years | Consumer electronics company | Composer and sound designer |
Table 7.
Grouping of Auditory Evaluation Factors from Primary Expert Interviews.
|
Grouped/ Non-grouped |
Grouping within an auditory presence factor | Grouping within AUX factors | Grouping between auditory presence and AUX |
| Grouped factors | Focusing the metaverse environment with sound | - | - |
| Implementing the spatial properties of sound | |||
| Implementing realistic soundscapes in a metaverse environment | |||
| Three-dimensional perception of sound | |||
| - | Willingness to return due to sound | ||
| - | The quality of the sound source | ||
| How distracting the sound is | |||
| Clarity of sound | |||
| Sound experiences that live up to visual expectations | |||
| How consistent the overall sound is in a sound variation situation | |||
| Auditory cues to help you navigate | |||
| Non-grouped factors | Sound to match visual motion timing* | - | - |
| Non-looping auditory experiences*. | |||
| - | Ease of access to sound control | ||
| Relating sounds to ideas | |||
| Harmonize between digital and natural sounds | |||
| Accessibility for those with sensory sensitivities | |||
| Distribution of the sound range | |||
| Sufficiency of volume size control |
* ‘Auditory experience matched to motion but not looped’ [30] was split into two factors by experts P1 and P2.
Table 8.
Questions and expert responses to the definition of auditory presence
| Interview question | Participants | Expert answers |
| How do we define auditory presence in a metaverse environment? | P4 | Auditory stimuli that make you feel ‘real in space, time, and emotion in a virtual space. |
| P5 | Make it feel like a real-world auditory experience. | |
| P6 | Auditory stimuli in the virtual world that reflect the sounds or emotionally evocative elements that listeners expect in the real world. | |
| P7 | To make a virtual environment sound and feel like it is real. | |
| P8 | The emotional and socially relevant auditory experience of everyday life is embodied in a virtual world. |
Table 9.
Questions and expert responses for creating an auditory experience evaluation questionnaire that considers the public understanding of presence
Table 9.
Questions and expert responses for creating an auditory experience evaluation questionnaire that considers the public understanding of presence
| Interview question | Participants | Expert answers |
| What should we consider when asking the public about auditory presence and AUX factors in a metaverse environment? | P4 | Strongly open to subjective interpretation and should be explained with examples. |
| P5 | Avoid technical terminology. | |
| P6 | Use simple, intuitive terminology. | |
| P7 | Reflects the words used to express sounds verbally. | |
| P8 | Organize your survey questions so that they're clearly separated and don't look like the same thing. |
Table 10.
Questions and expert responses for deriving questionnaire terms from auditory evaluating factors that considers the public understanding of presence
Table 10.
Questions and expert responses for deriving questionnaire terms from auditory evaluating factors that considers the public understanding of presence
| Interview question | Auditory evaluation factors | Expert answers | ||||
| P4 | P5 | P6 | P7 | P8 | ||
| Please create a terminology for the public to understand auditory presence and AUX metrics in a metaverse environment. | Implementing the spatial properties of sound | Location | Sound environment | Sense of space | Spatial awareness | Realism |
| Three-dimensional perception of sound | Distance | Object location | Distance | Location | Source | |
| The quality of the sound source | Broken | Bad | Torn | Sound quality | Reverberation | |
| Clarity of sound | Mushy | Clean | Cloudy | No response | Frustrating | |
| Focusing the metaverse environment with sound | Background music | Immersion | Background sounds | No response | Music | |
| How distracting the sound is | Varies | Complexity | Select a sound | Recklessness | Sound control | |
| Sounds that match visual motion timing | Matching | Gestures | Gaze | Movement | As it changes | |
| Sound experiences that live up to visual expectations | Matching | Guess | Blending with the background | Fits the mood | Expectations | |
| Non-looping auditory experiences | Naturalness | Regular | Constant | Repetitive | No response | |
| Implementing realistic soundscapes in a metaverse environment | Natural sounds | Routine sounds | Daily sounds | Realistic sounds | Household Noise | |
| Auditory cues to help you navigate | Environmental awareness | Situational judgment | Navigation help | When you need it | Intuitive | |
| How consistent the overall sound is in a sound variation situation | Consistent | Content identity | Change of Scene | Change of mood | Sound identity | |
| Ease of access to sound control | Volume sounds | Easy access | Volume icon | menu | Volume control | |
| Accessibility for those with sensory sensitivities | Adjusting sound effects | Acoustic control | Specific sounds manipulation | Desired sound | Sound texture | |
| Harmonize between digital and natural sounds | Artificial | Natural | Realistic | The actual | Created | |
| Relating sounds to ideas | Imagined | No response | Thought-related | Image | Semantics | |
| Distribution of the sound range | Contextualized | Suddenly getting bigger or smaller | No volume control | Negative pitch | All notes | |
| Sufficiency of volume size control | Adjust at will | Max volume | The desired as many | Volume control | Volume size | |
| Willingness to return due to sound | Memorable | No response | An impressive | Music, sound effects | Good memory | |
Table 12.
Results of Principal Component Analysis.
 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated