Optimization of Real-Time Spatial Mapping in Architectural Visualization: A Comparison Among Mixed Reality Devices

Recent advancements in communication technology have catalyzed the widespread adoption of realistic content, with augmented reality (AR) emerging as a pivotal tool for seamlessly inte-grating virtual elements into real-world environments. In construction, architecture, and urban design, the integration of mixed reality (MR) technology enables rapid interior spatial mapping, providing clients with immersive experiences to envision their desires. The rapid advancement of MR devices, or devices that integrate MR capabilities, offers users numerous opportunities for enhanced entertainment experiences. However, to support designers at a high level of expertise, it is crucial to ensure the accuracy and reliability of the data provided by these devices. This study explores the potential of utilizing spatial mapping within various methodologies for surveying architectural interiors. The objective is to identify optimized spatial mapping procedures and determine the most effective applications for their use. Experiments were conducted to evaluate the interior survey performance, using HoloLens 2, iPhone 13 Pro for spatial mapping, and photogrammetry. The findings indicate that HoloLens 2 is most well-suited for the tasks exam-ined in the scope of experiments. Nonetheless, based on the acquired parameters, the author also proposes approaches to apply the other technologies in specific real-world scenarios.

Keywords:

Subject: Engineering - Architecture, Building and Construction

1. Introduction

In an era dominated by advanced ICT technology, which serves as a cornerstone for myriad applications across diverse domains, emerging products aim to enhance productivity in conventional tasks. The evolution of data collection and transmission, particularly of spatial data, has undergone significant transformations. Various technologies have been developed to streamline data collection and reconstruct real-world environments, thereby augmenting user experience and bridging the gap between virtual and physical realms [1,2,3,4]. With recent advancements in mixed reality (MR) technology, a surge in the utilization of realistic content has been noted. Augmented reality (AR), which seamlessly integrates virtual information with the real world, is recognized as an effective method for achieving realism. This technology overlays virtual objects onto actual surroundings, ensuring alignment with real-world coordinates [5,6]. Consequently, the successful implementation of AR depends on the real-time integration of virtual data with the physical environment. To achieve this, MR devices must accurately acquire spatial mapping information of real spaces. Spatial mapping is an operation that allows users to scan or capture spatial data and reconstruct it into mesh data, which can be used for further modification, spatial analysis, and visualization [7,8,9].

In construction, architecture, and urban design, practitioners are increasingly incorporating contemporary data visualization tools to depict their designs visually [1,10,11]. These technologies aid in data collection, current situation assessment, and the rapid, accurate creation of 3D content. Within the context of smart cities, enhancing human quality of life takes precedence [12,13,14]. Addressing human needs through problem-solving and human-centered design has become a shared objective among urban and architectural professionals. MR technology can serve as a versatile platform for various development applications, aiding designers in addressing the above context. Rapid mapping of existing interior spaces within a building enables users or clients to freely engage with their surroundings, envisioning their ideal home environment. This process serves as an effective initial survey step for projects, allowing architects to both capture 3D spatial data and grasp the client's preferences. AR technology further facilitates the presentation of design proposals by consultants to clients. Despite the need for higher precision in professional projects, advanced scanning technologies such as terrestrial laser scanning (TLS), mobile laser scanning (MLS), and simultaneous localization and mapping (SLAM) are preferred. MR devices offer distinct advantages in certain applications to replace the professional devices. Thanks to the advancements in computing, sensor technology, and 3D modeling tools, MR devices enable cost-effective exploration while allowing practitioners to achieve satisfactory results. When coupled with AR content, these devices prove invaluable for educational, research, and design endeavors [8,15,16,17]. Several MR devices can be used for this operation, including head-mounted display (HMD) devices (HoloLens 2, Meta Quest 3, Apple Vision Pro, etc.) and mobile devices (iPhone Pro, ProMax 12 and later versions, iPad Pro 2020 and later versions). These devices are capable of capturing spatial information due to their built-in depth sensors [8,18].

Although MR technology has been researched and developed for a long time, its applications have not yet proven highly effective in professional fields requiring high precision, such as architecture, construction, and urban design [19,20,21]. However, the limitations of the spatial survey capabilities of commonly used MR devices have been noted, due to factors such as environmental changes, spatial configurations, material variations, lighting conditions, and operators’ skill. Additionally, the rapid pace of spatial exploration results in relatively low detail meshes generated by MR devices, thereby impacting the accuracy of spatial data [7,17]. A few studies have been conducted to test the accuracy of these devices across various contexts, aiming to identify the situations in which device performance meets the necessary standards. The research team noticed some gaps in previous studies, some common factor in practical applications such as the complexity of the survey environment, has been overlooked. The optimal level spatial mapping was only be assessed through consideration of data variation [7,8,9,16,22].

Within this study, the research team selected HoloLens 2 and iPhone 13 Pro for the spatial mapping experiment. To enhance comparability, photogrammetry was introduced as an additional surveying and mesh construction approach. Photogrammetry is a popular method frequently used for collecting real-world spatial data. With advancements in supporting software and computing power, photogrammetry enables users to reconstruct highly accurate spatial data in an ideal environment [23,24,25]. There are two objectives that the research team aims for in this article: O1 - Evaluate the accuracy of spatial data collected by the solutions proposed in the article, based on data variation; O2 - Calculate the performance of each method, based on the following criteria: complexity of the environment, data file size, and data variation. From these evaluations, discuss the suitability of each method for different application contexts.

To carry out this research and collect the data, three methodologies, including spatial mapping with HoloLens 2, iPhone 13 Pro, and photogrammetry with a camera, were evaluated across three interior environments featuring distinct complexities. The outputs of these methods were compared against a ground truth model constructed using 3D modeling with precise dimensions, allowing for the assessment of each approach’s performance in different scenarios. The collected data is used to calculate performance for each method, corresponding to the requirements set out in the scope of this study.

2. Materials and Methods

2.1. Spatial mapping operation with MR devices

The HoloLens 2 is a state-of-the-art mixed reality (MR) headset developed by Microsoft, offering an immersive computing experience as an HMD device. Its enhanced optics provide a wider field of view and higher resolution, which significantly improve realism. Advanced hand tracking allows for intuitive interaction with holographic objects. Furthermore, HoloLens 2 is equipped with a range of sensors, including depth sensors, cameras, and inertial measurement units, which enable precise spatial mapping and tracking. This functionality allows 3D content to accurately anchor to real-world surfaces and environments, facilitating the seamless integration of digital content into the physical world. HoloLens 2 is utilized in various industries such as manufacturing, healthcare, architecture, design, and education, revolutionizing our visualization and interaction with digital content [9,18]. In this study, the HoloLens 2 was employed for spatial mapping, enabling the collection and reconstruction of spatial data into a 3D model that can be exported for user analysis and utilization. The device's primary function of collecting spatial data is facilitated by its long throw mode in conjunction with the built-in depth camera (Figure 1). By capturing and replicating the geometric features surrounding the device's location, HoloLens 2 enables users to engage in interactive AR functionalities, including navigation, physics simulation, occlusion, and object placement [7,9,18]. Although Microsoft has announced the discontinuation of HoloLens 2 development, it remains a valuable HMD device for various research purposes. Launched and operated within the past five years, HoloLens 2 uniquely allows users to access and extract spatial data to other storage devices. This capability sets it apart from other popular HMD devices on the market, such as Meta Quest and Apple Vision Pro.

The iPhone features a light detection and ranging (LiDAR) sensor, a cutting-edge technology that significantly enhances depth perception and spatial awareness across various applications. The LiDAR sensor emits laser pulses to measure distances to objects, enabling more accurate AR experiences, faster autofocus under low-light conditions, and enhanced portrait photography with improved depth mapping. This advancement in sensor technology revolutionizes how users interact with their surroundings, providing new opportunities for immersive experiences and precise spatial mapping on mobile devices. Built-in LiDAR sensors are found in the following devices: iPhone Pro, ProMax 12 and later versions, and iPad Pro 2020 and later versions. For this research, the iPhone 13 Pro with a built-in LiDAR sensor was selected (Figure 1). However, an application from the iOS store must be installed to operate the spatial mapping experiment [8,26]. With the integrated technology on the iPhone, application developers continuously launch AR applications that support users in collecting and interacting with spatial data. This creates a seamless process, enabling users to collect, edit, and utilize realistic 3D objects within their environment.

2.2 Photogrammetry for architectural space

Owing to the advanced revolution in photogrammetry technology, including cameras and computer vision, rapidly reconstructing spatial data from images containing geometric data is now feasible, allowing even non-professionals to regenerate digital 3D models from real-life objects. The 3D digital content generated through photogrammetry can be easily modified, manipulated, and visualized directly inside 3D software, or used in render images, virtual environments, or various other types of 3D post-production. In the fields of architecture, construction, and urbanism, utilizing photogrammetry for surveying spatial data presents a cost-effective approach that yields dependable results suitable for professional projects, particularly when compared with other methods such as LiDAR sensors or laser scanning [23,24,25,27]. Given its proven effectiveness in collecting spatial data within these fields, photogrammetry emerges as a viable solution for comparison alongside selected MR devices in this study.

2.3 Experiment procedure

The process of experimenting and evaluating the output results comprises four stages: input, process, output, and post-process (Figure 2). This sequence ranges from collecting input data in building environments using various solutions to the final assessment for comparison.

Input stage: In this phase, the research team selects a mutual space for all three different experimental environments. The overall structure and lighting conditions of the entire space remain consistent. Progressively, from Environment 1 to 3, the complexity level escalates with the addition of more furniture within the room. Environment 1 contains one furniture object, while Environments 2 and 3 house seven and fourteen different furniture objects, respectively.

Process: Firstly, all the experimental environments and interior objects are measured carefully to construct 3D ground truth models with modeling software. Subsequently, each spatial survey method is operated to collect spatial data from every environment. HoloLens 2 and iPhone 13 Pro are used for real-time spatial mapping, while a mobile camera captures images of all details as input data for photogrammetry.

Output: In addition to the three ground truth models of the environments, nine different scenarios of spatial models are created using three methods across the three experimental environments.

Post-Process: The objective of this research is to optimize spatial mapping operations for different contexts of architectural spaces. To achieve this, the nine different spatial models are compared with the ground truth model of their respective environment references. For instance, the ground truth model of Environment 1 serves as a reference for the comparisons of spatial models 1.1, 2.1, and 3.1. Finally, using the comparison values, the research team assesses the performance based on three different factors: the complexity level of the environment, the total number of polygons in the spatial model, and the standard deviation between the spatial model and its reference.

2.4 Experiment environment preparation

The selection of this particular space for experimental practice is based on its versatility for alterations and its manageable size, spanning approximately 20 m². This size is ideal for practicing various spatial survey techniques such as spatial mapping and photogrammetry, striking a balance between being neither overly spacious nor exceedingly confined. Corresponding to a typical room size in common household designs, this space offers an ideal scale for experimentation. Furthermore, the room presents certain challenges for surveying due to the presence of specific materials such as metal, glass, and white-painted walls (Figure 3). These conditions serve as appropriate obstacles for devices utilizing depth sensors and for photogrammetry methods, testing their ability to navigate and map space effectively.

Regarding the experimental environment, the presence of windows could directly impact the quality of light in the room. Therefore, we chose to conduct the study on a cloudy day and utilized artificial lighting as the primary light source to ensure consistent lighting conditions. To enhance the intricacy of the experimental settings, a selection of furniture items was strategically positioned within the communal area of the room. These items were arranged in a haphazard manner, with a few items clustered together and others placed individually, thereby altering the overall dynamics of the environment. The chosen objects exhibit diversity in both size and shape, ranging from simple forms such as paper boxes to more distinct shapes such as plant pots and fire extinguishers. The list of items for each environment is detailed according to Table 1.

3. Results

3.1. Ground truth model with measurement and 3D modeling

In alternative studies, ground truth models are constructed using specialized spatial data survey equipment such as LiDAR sensors or TLS devices. However, due to the constraints of this study, a more traditional method was employed. To construct the ground truth model, the research team used a conventional approach to survey the architectural landscape, involving traditional measurements and subsequent reconstruction of the space using 3D modeling software (Figure 4). This endeavor demands meticulous precision and necessitates the involvement of seasoned professionals in the fields of architecture and interior design to participate in the measurement process.

The approach offers several advantages. Firstly, it effectively controls the polygon count in 3D models, ensuring that reconstructions of real spaces maintain a balance between polygon count and accuracy. Secondly, it addresses challenges posed by reflective surfaces such as glass or metal, ensuring accurate representation regardless of material properties. In this study, SketchUp software was used to create a ground truth model based on parameters measured from reality.

3.2. Spatial mapping with HoloLens 2 and iPhone 13 Pro

Employing spatial mapping methods for spatial data collection, the research team utilized two devices, namely the HoloLens 2 and the iPhone 13 Pro, to facilitate MR experiences. Initially, during the exploration of the functionalities of the HoloLens 2 at the start of the experiment, many applications in Microsoft's library enabling spatial data collection had been discontinued. Consequently, the study resorted to using the Microsoft Portal to establish connectivity between the HoloLens 2 and the computer in developer mode. Upon activation, the HoloLens 2 automatically initiates spatial mapping by scanning the surrounding environment and storing the acquired data in memory, thereby facilitating AR experiences for users.

In the Microsoft Portal interface, users have convenient control over the spatial mapping operations of the HoloLens 2, with the ability to comprehensively visualize the extent of collected spatial data. Opting to update information triggers changes in spatial data based on the most recent scanning session of the HoloLens 2 device in the respective area (Figure 5). Users can iteratively update the data until they ascertain its currency and completeness. The implementation of HoloLens 2’s spatial mapping in this study is iterative and protracted, aimed at achieving optimal data collection. The spatial mapping resolution utilized by the HoloLens 2 in this investigation is set at 500 triangles per cubic meter, which aligns with the device’s default mode. Subsequently, the spatial data obtained through the spatial mapping procedure can be immediately exported to a user’s computer as an OBJ file. This data can subsequently be readily utilized for tasks such as measurement, and the creation of VR and AR environments, among others.

For spatial mapping tasks employing an iPhone 13 Pro equipped with a LiDAR sensor, the procedure for scanning the room to collect data is similar to that of the HoloLens 2. However, in this experiment, a third-party application called 3D Scanner was utilized. This application allows users to acquire real-time spatial data akin to Near Eye Displays devices by leveraging the built-in LiDAR sensor. During the spatial survey process, users have the capability to continually monitor the collected spatial content and make appropriate adjustments. However, the immediate construction of 3D models is not feasible and requires a certain duration, contingent upon the desired content quality. In this investigation, the research team selected settings such as Smooth Scan, Texture Scan, and LiDAR Advanced. The final model is exported as an OBJ file (Figure 6). The experimenter scanned the entire room space using the content displayed on the phone screen, ensuring continuous updates until completion.

While the generation of a 3D model following real-time scanning is time-consuming, the 3D scanner software enables the iPhone 13 Pro to produce a model that incorporates the texture of the actual environment. Conversely, the mesh model produced by HoloLens 2 solely encompasses spatial data and lacks texture.

3.3. Photogrammetry and mesh construction

Currently, photogrammetry is widely adopted as an economical method for spatial data acquisition. Utilizing the computational power of computers, photogrammetry techniques enable users to reconstruct 3D representations of diverse objects, ranging from small items to entire interior and exterior spaces, or even extensive areas. Numerous applications available on both mobile devices and computers have been developed to assist users in generating 3D models from sequences of images (Figure 7). In this study, images were captured using a mobile device according to a specified protocol to ensure comprehensive data utilization before being imported into Agisoft Metashape software. Subsequently, the software analyzes the images to generate a point cloud mesh, which is further processed using algorithms to produce 3D models that can be exported as OBJ files.

The quantity of photos captured varied, ranging from approximately 180 to 220 images, depending on the complexity of each research environment. Environments 2 and 3, which contained numerous objects in addition to open space, were categorized into distinct groups. Image sequences capturing the surroundings of these object groups were also acquired to ensure data processing accuracy using the Agisoft Metashape software (Figure 8). To prevent duplication of image data and facilitate point matching for seamless connectivity, photos were captured from specific distances, ensuring that various objects within the space were encompassed within each frame.

The 3D model generated through photogrammetry includes texture data, similar to the approach involving the iPhone 13 Pro. However, this method is comparatively more time-consuming than other techniques as it necessitates post-data collection processing using computer software. Moreover, due to the lack of real-time spatial mapping, users may experience difficulties in ascertaining the adequacy of the collected data during the process. Furthermore, another potential constraint involves the precision of the scale and proportions within the spatial model generated through photogrammetry. Because geometric details are solely derived from captured image data, the resulting spatial model can accurately represent the relative sizes of objects in the environment but may not replicate their precise dimensions. Consequently, during post-processing, practitioners must employ scaling techniques based on real-world dimensions to refine the model.

3.4. Generated 3D models with spatial data of different methods

The results obtained reveal a clear distinction between the products created from three methods: spatial mapping using HoloLens 2, spatial mapping using iPhone 13 Pro, and photogrammetry (Figure 9). Certain differences are immediately recognizable through observation. Based on the output observation, iPhone 13 Pro is found to exhibit superior capability of collecting spatial information, particularly with reflective and metallic surfaces, compared with the other two methodologies. Conversely, HoloLens 2 encounters challenges in identifying the spatial surfaces of glass windows and objects with reflective attributes such as metal. Furthermore, due to the default limitation of 500 triangles per cubic meter in HoloLens 2, the device struggles with recognizing objects of smaller sizes and those with intricate shapes (such as plants, fire extinguishers, and baskets), as evidenced in experiment environment 3. When comparing the outcomes of the photogrammetry and spatial model methods produced by the iPhone 13 Pro's LiDAR sensor, substantial differences are not noted.

3.5. Using CloudCompare to compare generated 3D models with ground truth models

As outlined in the workflow and research methodology, in the post-processing phase of the experiments, to calculate the differences between 3D models created by three spatial mapping methods—using HoloLens 2, iPhone 13 Pro, and Photogrammetry—with ground truth models, the research team uses CloudCompare software. This is an open-source software specifically used to edit, measure, and process spatial data.

The two models to be compared are imported into CloudCompare as OBJ files. In all scenarios of comparing different environments in experiment environments 1, 2, and 3, the ground truth model is set as the reference model for rotation, scaling, movement, and distance variation computations. Firstly, the experiment practitioner adjusts the position and rotation of the model to be compared in order to match with the ground truth model and subsequently proceeds with cloud registration. Through software calculations, the two models are perfectly aligned. The default parameters for the cloud registration process include 20 iterations, a root mean square difference of 1.0 × 10⁻⁵, 100% final overlap, and a random sampling limit of 50,000 (Figure 10.1). Alignment can be adjusted along all three XYZ axes. For models generated by spatial mapping with HoloLens 2 and iPhone 13 Pro, no scale adjustment is necessary. However, because photogrammetry-generated models may lack scale accuracy due to their reconstruction of geometry characteristics only, the cloud registration step requires scale adjustment during this step.

Once the two models are aligned, the research team conducts a mesh distance computation, allowing the software to calculate two parameters: standard deviation and mean distance. This step involved comparing nine different scenarios, with the maximum distance selected for comparison set at 1.5 m (Figure 10.2). In the final step, after the variation of spatial data between the two models has been completely calculated, the research team supports the observation and assessment of the model’s spatial data accuracy compared through two forms:

Spatial data variation heat map with parameters in the following order: Location difference at 1.5 m—red; position difference at 0.1 m—yellow; different positions at -0.1 m—green; different positions at -1.5 m—blue.
The diagram shows the variation level of spatial data, with the x-axis being the distance difference parameter, and the y-axis is the number of faces/polygons of the model (Figure 10.3).

All model parameters in the nine scenarios of spatial data collection and 3D model reconstruction are shown in Table 2.

The final results of the comparison can be observed in Figure 11, where spatial data variation is visualized in the following order: location difference at 1.5 m—red; position difference at 0.1 m—yellow; different positions at -0.1 m—green; different positions at -1.5 m—blue (Figure 11). All three methods perform efficiently in collecting spatial data in free-space environments, such as Environment 1, which is less complicated. In the next two environments, all three methods demonstrate difficulty in reconstructing space in areas where interior objects are located. In general, across all spatial data collection instances, the collected spatial data tends to be a positive value. This indicates that when collecting spatial data via all three methods, inaccurately collected data typically scatters closer to the sensor or camera location rather than in the opposite direction. This finding contributes significantly to the navigation operation of AR content when using MR devices in this study, suggesting improvements in proximity-based data handling for enhanced accuracy.

3.6. Standard deviation export via CloudCompare

This study primarily aimed to assess the optimization capacities of various devices in constructing environments suitable for AR interactions within the context of architectural design. Consequently, the generated 3D environments may not necessarily appear more realistic but are optimized for functional use in AR interactions. AR interactions require capturing precise spatial data for navigation and accurately rendering physical elements within real-space environments, as facilitated by MR devices. The collected environmental data must be adequate to create satisfactory conditions for effective AR interactions. While high-quality data collection may occasionally result in a significant increase in file volume, the enhancement in accuracy is not always proportionate.

The output models generated by the three different methods employed in this study exhibit varying numbers of faces/polygons for each research environment. Ranked in ascending order of faces/polygons, the methods are as follows: HoloLens 2, iPhone 13 Pro, and Photogrammetry. The models produced by the iPhone 13 Pro contain approximately 2.5 times the number of faces/polygons compared with those generated by HoloLens 2. Conversely, models developed through the photogrammetry method feature 16 times the number of faces/polygons compared with the iPhone 13 Pro and approximately 40 times more than those from HoloLens 2 (Table 2). The results on the number of faces/polygons of the methods for all three experiment environments reveal that this number also increases as the complexity of the environment increases with the number of objects featuring special shapes.

As previously stated, assessing the adequacy of spatial data collected for constructing a 3D environment cannot solely rely on observation. Therefore, a comparison with the ground truth model is necessary to precisely evaluate spatial differences. In this study, mean distance and standard deviation calculations were employed to gauge the extent of variance between models generated by MR devices and real spatial data (Table 2). These calculations are facilitated by CloudCompare software, which was utilized by the research team to compute these parameters based on the 3D models derived from all three experiments and the ground truth models. These analytical steps are critical in quantifying the effectiveness and precision of each method in various architectural settings.

μ = \frac{1}{N} \sum_{i = 1}^{N} x_{i}

(1)

σ = \sqrt{\frac{1}{N} \sum_{i = 1}^{N} {(x_{i} - μ)}^{2}}

(2)

N: Number of all distance; xi= Distance from point i to its reference.

Standard deviation is a statistical measure that quantifies the amount of variation or dispersion in a set of data points. It provides a numerical value, indicating the extent to which individual data points differ from the mean (average) of the dataset. A higher standard deviation indicates greater variability among the data points, while a lower standard deviation suggests that the data points are closer to the mean. According to this interpretation, when utilizing standard deviation to assess the precision of spatial data collected in research methodologies, a lower value of this parameter corresponds to a higher level of data accuracy.

4. Discussion

The discussion based on the research tasks given at the beginning of this article is as follows.

O1 - Evaluate the accuracy of spatial data collected by the solutions proposed in the article, based on data variation: The analysis indicates that across all three experimental settings (1, 2, and 3), the photogrammetry-generated model consistently exhibits the highest level of accuracy, as evidenced by its lowest standard deviation value. Conversely, the spatial data captured by the LiDAR sensor with the iPhone 13 Pro device slightly surpasses those obtained using the depth sensor of HoloLens 2. Notably, in experiment environment 1, the photogrammetry approach produced deviation data that was only half that of the other two methods. Interestingly, all three data collection methodologies yielded the highest standard deviation outcomes for environment 2, indicative of the least accuracy in this setting (Figure 12). As mentioned before, the number of faces/polygons of models reconstructed by the photogrammetry method is significantly higher than the other two methods. Consequently, the volume of OBJ files created by this method is also substantially higher, which can present a challenge for this option in practical projects due to the increased demands on storage and processing capacity.

O2 - Calculate the performance of each method, based on the following criteria: complexity of the environment, data file size, and data variation: To identify the most effective method for collecting spatial data to support the creation of AR interactive content in architectural design, several key factors must be considered. Merely ensuring the lowest standard deviation parameter in a scenario does not automatically qualify it as the optimal solution. In our study, we constructed three experimental environments of varying complexity, which serves as the first factor influencing the outcome of spatial data collection. We hypothesized that as the complexity of the environment increases, characterized by the presence of more intricately shaped objects, the number of faces/polygons in the ground truth model would likewise increase. Thus, the spatial data collection tools in our study perform most effectively in spaces predominantly comprising flat surfaces, while encountering greater challenges in the remaining two environments.

The second critical factor our research team considers architectural space surveys for evaluating the accuracy of each method. This study assumes that models with lower standard deviation data demonstrate superior performance. Finally, storage resources represent a significant consideration, as they are determined by the number of faces/polygons comprising the reconstructed model. Constructing AR interactive content necessitates a balance between spatial information accuracy for navigation tasks and maintaining a compact data size. This ensures efficient resource utilization, prevents unnecessary data accumulation, and minimizes processing time on devices. The research team assumes that the productivity of the scenario is higher when the number of faces/polygons of the model is lower. According to this explanation, this study proposes a calculation equation for evaluating spatial mapping operation performance, referred to as Equation (3).

P = \frac{F_{t}}{F . σ}

(3)

P: Performance of the spatial mapping operation

Ft: Total polygons of ground truth mode

F: Total polygons of comparing model/ scenario

σ: Calculated standard deviation of scenario

Based on the standard deviation data collected through the CloudCompare application for each scenario and the number of faces/polygons of each model, the research team can effectively calculate the spatial mapping operation performance of different methods (Table 3).

The findings demonstrate that HoloLens 2 outperforms the other methods in spatial mapping across all three environments, achieving scores consistently threefold higher than those obtained using the iPhone 13 Pro's built-in LiDAR sensor. While photogrammetry exhibits the most precise spatial data collection, indicated by its lowest standard deviation values across all experimental settings, its use within the study's context is comparatively limited. This limitation primarily stems from the substantial number of faces/polygons it generates, which does not translate into a corresponding increase in spatial data accuracy.

HoloLens 2’s real-time spatial mapping exhibits commendable accuracy for immediate interaction with AR content. However, the 3D models derived from spatial data gathered by HoloLens 2 lack versatility as they feature a constrained number of faces/polygons and lack texture data. Consequently, if architectural design endeavors necessitate usable 3D data for environment recreation, the iPhone 13 Pro proves to be a fitting and efficient device, balancing detail and real-time processing capability.

The photogrammetry approach, while effective in configuring environments for AR interactive content, involves a comparatively longer processing duration than the other two alternatives. The research team believes that this method can be efficiently employed in projects requiring meticulous reconstruction of real spaces into high-precision 3D models for architectural visualization, especially where the highest level of detail and accuracy is crucial.

5. Conclusions

In this investigation, a comparative analysis of spatial data collection and 3D model reconstruction was conducted across three distinct interior environments, each featuring varying degrees of complexity achieved through the arrangement of additional furniture groups. Three methodologies were employed: spatial mapping utilizing the HoloLens 2 depth sensor, spatial mapping leveraging the LiDAR sensor of the iPhone 13 Pro, and photogrammetry. This resulted in nine unique scenarios, corresponding to nine 3D models reconstructed from the collected spatial data. The accuracies of these scenarios were evaluated by computing spatial data discrepancies using the standard deviation extracted from the CloudCompare application.

In assessing the optimal level of spatial mapping activities for the development of AR content in architectural design, the research team considered three pivotal factors directly influencing the outcomes. The first factor pertains to the complexity of the environment, gauged by the number of faces/polygons constituting the ground truth model. The second factor revolves around the precision of spatial data, evaluated through the standard deviation metric. Lastly, the capacity of the resulting 3D file is dictated by the number of faces/polygons comprising the model itself.

Following the computation of these parameters, the research team concluded that across all scenarios, encompassing both simple and intricate environmental setups necessitating spatial data collection, the HoloLens 2 device exhibited the most optimal performance within the parameters outlined in this study. Subsequently, the iPhone 13 Pro LiDAR sensor ranked next in efficacy, followed by the photogrammetry method. Different priorities may arise depending on the specific requirements of the intended 3D model. While the HoloLens 2 facilitates real-time spatial data collection with minimal storage demands, optimizing AR environment construction, the resulting spatial data lack texture data and feature a lower polygon count, rendering it unsuitable for post-processing architectural design operations. Conversely, users can generate a 3D model with marginally superior accuracy compared with HoloLens 2, supplemented with texture data, by utilizing the 3D Scanner application in conjunction with the built-in LiDAR sensor of the iPhone 13 Pro. Despite its advantages, this approach entails a significant increase in storage requirements, resulting in three times the number of faces and polygons compared with the HoloLens 2 device.

Conversely, the Photogrammetry method, while yielding the highest spatial data accuracy along with texture data inclusion, involves multiple processing steps across various software platforms, resulting in a 3D model with an excessively high number of faces and polygons. However, this method can be refined through various optimization techniques, particularly when the project objective involves accurately replicating the physical environment in virtual space. Users have the flexibility to employ various strategies during photogrammetry to eliminate redundant data, complemented by post-processing steps such as retopology and 3D modeling.

This study paves the way for further exploration into the utilization of visualization technology in architectural design. Future research endeavors could delve into leveraging the 3D model outputs generated through spatial mapping techniques outlined in this study to fulfill diverse visualization objectives such as: Improving design and operation support by integrating in project workflow; Enhancing user interaction with the generated AR content; Optimizing the data handling processes.

Author Contributions

Conceptualization, Tam Do, Leehwan Hwang and Seunghyun Lee; Data curation, Tam Do and Kang Sanhae; Formal analysis, Tam Do, Jumamurod Ugli and Yi Dongsu; Funding acquisition, Seunghyun Lee; Investigation, Tam Do and Kang Sanhae; Methodology, Tam Do and Leehwan Hwang; Project administration, Leehwan Hwang and Seunghyun Lee; Software, Tam Do, Kang Sanhae and Jumamurod Ugli; Supervision, Leehwan Hwang and Seunghyun Lee; Validation, Tam Do and Yi Dongsu; Visualization, Tam Do; Writing – original draft, Tam Do and Narzulloev Mirzaevich; Writing – review & editing, Tam Do and Narzulloev Mirzaevich.

Acknowledgments

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2024-2020-0-01846) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation), The present research has been conducted by the Excellent researcher support project of Kwangwoon University in 2023.

Conflicts of Interest

The authors declare no conflicts of interest.

References

Al-Kodmany, K. , Visualization tools and methods in community planning: from freehand sketches to virtual reality. J. Plan. Lit. 2002, 17, 189–211. [Google Scholar] [CrossRef]
Korkut, E.H. and E. Surer, Visualization in virtual reality: a systematic review. Virtual Real. 2023, 27, 1447–1480. [Google Scholar] [CrossRef]
Rubio Tamayo, J.L. Barro Hernández, and H. Gómez Gómez, Digital data visualization with interactive and virtual reality tools. review of current state of the art and proposal of a model. Digital Data Visualization with Interactive and Virtual Reality Tools, 2018.
Shakeel, H.M. , et al., A comprehensive state-of-the-art survey on data visualization tools: Research developments, challenges and future domain specific visualization framework. IEEE Access 2022, 10, 96581–96601. [Google Scholar] [CrossRef]
Arena, F. , et al., An overview of augmented reality. Computers 2022, 11, 28. [Google Scholar] [CrossRef]
Carmigniani, J. and B. Furht, Augmented reality: an overview. Handbook of augmented reality 2011; pp. 3–46.
Hwang, L. , et al., A study on optimized mapping environment for real-time spatial mapping of HoloLens. Int. J. Internet Broadcast. Commun. 2017, 9, 1–8. [Google Scholar]
Hou, J. , et al., Indoor Mapping with Entertainment Devices: Evaluating the Impact of Different Mapping Strategies for Microsoft HoloLens 2 and Apple iPhone 14 Pro. Sensors 2024, 24, 1062. [Google Scholar] [CrossRef] [PubMed]
Weinmann, M. , et al., Efficient 3d mapping and modelling of indoor scenes with the microsoft hololens: A survey. J. Photogramm. Remote Sens. Geoinf. Sci. 2021, 89, 319–333. [Google Scholar]
Groenendyk, M. , Emerging data visualization technologies for map and geography libraries: 3-D printing, holographic imaging, 3-D city models, and 3-D model-based animations. J. Map Geogr. Libr. 2013, 9, 220–238. [Google Scholar] [CrossRef]
Rohil, M.K. and Y. Ashok, Visualization of urban development 3D layout plans with augmented reality. Results Eng. 2022, 14, 100447. [Google Scholar] [CrossRef]
Trinh, T.A. Trinh, T.A., et al. Framework Proposal of Smart City Development in Developing Country, A Case Study-Vietnam. in International Conference on Computational Science and Its Applications. 2022, Springer.
Echebarria, C., J.M. Barrutia, and I. Aguado‐Moralejo, The Smart City journey: a systematic review and future research agenda. Innov. Eur. J. Soc. Sci. Res. 2021, 34, 159–201.
Gracias, J.S. , et al., Smart Cities—A Structured Literature Review. Smart Cities 2023, 6, 1719–1743. [Google Scholar] [CrossRef]
Newcombe, R.A., et al. Kinectfusion: Real-time dense surface mapping and tracking. in 2011 10th IEEE international symposium on mixed and augmented reality. 2011, Ieee.
Díaz Vilariño, L. , et al., 3D mapping of indoor and outdoor environments using Apple smart devices. ISPRS-International Archives of the Photogrammetry Remote Sensing Spatial Information Sciences, 2022.
Hübner, P. , et al., Evaluation of HoloLens tracking and depth sensing for indoor mapping applications. Sensors 2020, 20, 1021. [Google Scholar] [CrossRef] [PubMed]
Microsoft, About HoloLens 2. 2023.
Noghabaei, M. , et al., Trend analysis on adoption of virtual and augmented reality in the architecture, engineering, and construction industry. Data 2020, 5, 26. [Google Scholar] [CrossRef]
Lee, J.G. , et al., End-Users’ augmented reality utilization for architectural design review. Appl. Sci. 2020, 10, 5363. [Google Scholar] [CrossRef]
Delgado, J.M.D. , et al., A research agenda for augmented and virtual reality in architecture, engineering and construction. Adv. Eng. Inform. 2020, 45, 101122. [Google Scholar] [CrossRef]
Teruggi, S. and F. Fassi, Hololens 2 spatial mapping capabilities in vast monumental heritage environments. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2022, 46, 489–496. [Google Scholar] [CrossRef]
Shan, J., et al., Toward a unified theoretical framework for photogrammetry. 2020, 23, 75–86.
Firzal, Y.J.G.J. Architectural photogrammetry: a low-cost image acquisition method in documenting built environment. GEOMATE Journal 2021, 20, 100–105. [Google Scholar] [CrossRef]
Murtiyoso, A., et al., Towards semantic photogrammetry: generating semantically rich point clouds from architectural close‐range photogrammetry. Sensors 2022, 22, 966.
Askar, C. and H.J.G. Sternberg, Use of Smartphone Lidar Technology for Low‐Cost 3D Building Documentation with IPhone 13 Pro: A Comparative Analysis of Mobile Scanning Applications. Geomatics 2023, 3, 563–579.
Moyano, J., et al., Validation of close‐range photogrammetry for architectural and archaeological heritage: Analysis of point density and 3D mesh geometry. Remote sensing 2020, 12, 3571.

Figure 1. Spatial data collection with built-in depth sensor of MR device (a) Depth camera inside HoloLens 2 (b) LiDAR sensor inside iPhone 13 Pro.

Figure 2. Experimental methodology and workflow.

Figure 3. Experiment environment (a) Environment 1 (b) Environment 2 (c) Environment 3.

Figure 4. Ground truth models of three experiment environments: (1) Experiment environment 1 (2) Experiment Environment 2 (3) Experiment Environment 3 (a) View from Northeast (b) View from South-West.

Figure 5. Operation of spatial mapping with HoloLens 2: (a) Microsoft portal interface (b) Room scanning process.

Figure 6. Operation of spatial mapping with iPhone 13 Pro and 3D scanner. (a) Settings for 3D Scanner. (b) LiDAR sensor is used to capture spatial information.

Figure 7. Operation of photogrammetry. (a) Sequence of images is acquired around the room space to capture spatial information as the point cloud mesh. (b) Point cloud mesh generated in Agisoft Metashape from a sequence of images.

Figure 8. Sequences of images are acquired around each group of objects in the 2nd and 3rd experiment environments.

Figure 9. Generated 3D models with spatial data of different methods.

Figure 10. Interface of CloudCompare and the step-by-step calculation for model comparison. (1) Cloud registration for aligning entities; (2) Distance computation for deviation; (3) Calculation of mean distance and standard deviation.

Figure 11. Generated 3D models with spatial data of different methods.

Figure 12. Spatial data mapping accuracy comparison among nine scenarios, with three data collection and model reconstruction methodology

Table 1. List of objects in three different experiment environments.

Components	Environment 1	Environment 2	Environment 3
Room	Total Area: 21.4 m²
Furniture	(1 Object) Waste Storage	(7 Objects) Waste Container Boxes Table Chair Monitor Books Waste Bin	(14 Objects) Waste Container 2 Boxes Groups Table Chairs Monitor Books Waste Bin Plant Fire Extinguisher Basket Spray Bottle Hand Truck Bag

Table 2. Model’s information and deviation results for nine scenarios, acquired from the CloudCompare calculation.

EE	Ground Truth Model		HoloLens 2 Model				iPhone 13 Pro Model				Photogrammetry Model
EE	File Size (mb)	Faces	File Size (mb)	Faces	MD	SD	File Size (mb)	Faces	MD	SD	File Size (mb)	Faces	MD	SD
1	0.426	3,161	2.19	30,655	-0.0356	0.0656	7.78	81,639	-0.0295	0.0634	195	1,289,965	0.0056	0.0301
2	1.77	10,035	2.23	31,468	-0.0328	0.1003	9.31	88,971	0.00009	0.0899	196	1,309,114	0.0180	0.0556
3	14.2	96,972	2.81	39,103	0.0063	0.0703	16.8	130,123	0.00933	0.0695	202	1,428,345	0.0188	0.0512

* EE: Experiment Environment; SD: Standard Deviation, MD: Mean Distance

Table 3. List of objects in three different experiment environments.

P	HoloLens 2 Model	iPhone 13 Pro Model	Photogrammetry Model
EE 1	1.5711374	0.610993	0.0814077
EE 2	3.1802083	1.2553794	0.1377818
EE 3	35.287676	10.718464	0.1371681

* EE: Experiment Environment; P: Performance Calculation

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.

MDPI Initiatives

Important Links

Choose an area of interest and we will send you notifications of new preprints at your preferred frequency.

Disclaimer