Preprint
Article
Pre-Dam Vltava River Valley—A Case Study of 3D Visualization of Large-scale GIS Datasets in Unreal Engine
Altmetrics
Downloads
115
Views
54
Comments
0
A peer-reviewed article of this preprint also exists.
This version is not peer-reviewed
Submitted:
29 July 2024
Posted:
29 July 2024
You are already at the latest version
Alerts
Abstract
This article explores the potential of the Unreal Engine game engine as a tool for creating highly realistic 3D visualizations of GIS data, specifically in the context of significant historical areas. The research will focus on the computational techniques used in the 3D visualization of GIS data and the Unreal Engine software and its potential applications in geomatics. The article also presents several relevant scientific studies and projects that have successfully used game engines for similar purposes, explores the potential for using GIS data within a game engine, including plug-ins that provide additional functionality to GIS software, and the use of procedural modeling to create 3D objects through the CityEngine software plug-in. Finally, the research is applied to a dataset of the historical Vltava Valley covering an area of 1670 km2 to demonstrate the capabilities of Unreal Engine in creating realistic visualizations of large historical landscapes, and the resulting visualizations are presented. This practical application of the research provides valuable insights into the potential of the Unreal Engine as a powerful tool for creating highly realistic 3D visualizations of large historical areas with potential applications in the field of geomatics.
Keywords:
Subject: Environmental and Earth Sciences - Geography
1. Introduction
This article deals with the recreation of 3D GIS visualization of the historic pre-dam Vltava River valley in Unreal Engine. The original GIS visualization and all the data used comes from the project DH23P03OVV055 Vltava II—Transformations of the Historical Landscape, the River as a Link and a Barrier, hereafter referred to as Project Vltava, which is part of the Czech Ministry of Culture's NAKI program (Program to Support Applied Research and Development of National and Cultural Identity). One of the goals of the Vltava Project was to create a 3D visualization of the Vltava River valley as it looked before the construction of the Vltava cascade, which took place between 1930 and 1992. This was achieved using the ESRI GIS platform. The visualization used period maps as the primary data source for creating the digital terrain model (DTM) and modeling buildings and other objects based on land use information. Furthermore, photographs, plans, drawings, and other materials aided this reconstruction period. This resulted in the creation of DTM with an area reaching almost 1700 km2, procedurally created models for more than 28000 buildings, more than 10 million trees covering forests, and manually modeled significant buildings (churches, castles, etc.). More information on the NAKI program and the Vltava projects can be found on the Czech Ministry of Culture's website or related articles [1,2,3,4,5,6]. The goal of this article, however, is to present the process used to recreate the exact 1670 km2 visualization with the same GIS data using Unreal Engine 5 (hereafter referred to as UE), focusing on the differences in the handling and processing of large-scale GIS data between the GIS and UE Visualization.
Creating 3D visualizations of large-scale landscapes, such as counties, valleys, or plains, using GIS data presents significant challenges regarding computer hardware requirements. These projects can span tens, hundreds, or even thousands of square kilometers, which requires a substantial amount of processing power (CPU and GPU), RAM space to process, and storage space to store the vast amount of data necessary to generate the 3D models, create the visualization, and view the finished product [7]. However, since introducing the first microprocessors in the 1970s, CPU and GPU performance has followed an exponential growth trend known as Moore's Law. Over the past few decades, this prediction has held, resulting in an astonishing improvement in computing power [8,9]. This constant computation performance uplift has enabled the renaissance of old technologies previously limited by processing power and memory constraints. [10]
One of those is using Game Engines, which were previously barred behind unattainable (or too expensive) hardware demands. These powerful software frameworks, initially designed for creating interactive video games, have transcended their gaming origins to become essential platforms for rendering and simulating complex virtual environments. Leveraging advanced graphics rendering capabilities, physics simulations, and scripting systems, game engines empower geomatics practitioners to construct captivating and realistic visual representations of spatial data. In this context, game engines serve as the bridge that unites gaming technology and geospatial science, offering a flexible and dynamic medium for exploring, analyzing, visualizing, and communicating intricate mathematical concepts within geomatics [11,12]. Game engines provide developers with the tools, libraries, and resources to create and develop video games or interactive virtual experiences. These engines are designed to handle various aspects of game development, including graphics rendering, physics simulations, input handling, audio playback, or scripting [13,14,15]. Furthermore, Virtual Reality (VR), whose concepts date to the 1960s, enables users to immerse themselves in a virtual environment, providing a highly realistic and engaging experience [16,17]. While the creation of VR apps is not limited to game engines, it is deeply intertwined with them [18].
Thanks to those advancements, game engines began to see greater use outside of game-making [19], firstly for smaller and less graphicly intensive projects (such as [7]) and now even large-scale and graphically intensive projects [20,21]. This can be showcased in the project Rome Reborn, initiated in 1996, which aimed to create a highly detailed and accurate 3D reconstruction of Rome during different periods, allowing viewers to explore and experience the ancient city virtually. The visualizations of Rome Reborn depict different periods in Rome's history, from the first settlement in the late Bronze Age (ca. 1,000 B.C.) to the early Middle Ages (ca. A.D. 550). Each period showcases the city's architectural evolution and urban development. The project utilized a combination of historical research, archaeological data, and advanced digital technologies to recreate Rome's architecture, streets, and landmarks. Various software tools were employed to create the visualizations, including 3D modeling software such as 3DS Max and Maya and procedural modeling techniques using CityEngine [22,23,24]. The use of game engines came with the Rome Reborn 3.0 in 2018. However, with it, the project shifted from free to download to a paid model (flyoverzone.com), and no more information about the project was published. Other large-scale visualization projects, such as Virtual Angkor (virtualangkor.com), don’t publish any usable methodology information to recreate such visualizations either [21,25]. Sadly, it seems that today, such a fate, where research/visualization methods used in large-scale visualization projects are never published or retrospectively locked or removed, is standard practice. Since there aren’t many such large projects to begin with, this creates a gap in the literature regarding large-scale visualizations and their specific needs for used methods and data. For this reason, this article focuses on individual methods tested and used for visualizing the large-scale landscape of the Vltava River valley, covering 1670 km2 using standard GIS data and data formats as used in the Vltava Project (the original GIS visualization was previously described in the article [1]).
2. Methodology
To contextualize different measures, observations, comparisons, and results of this study, it is essential to specify what hardware was used during the works on the visualization of the Vltava River in UE. Since specific procedures were evaluated on a different computer during the author’s stay at Adam Mickiewicz University in Poznań, both computer builds are listed in Table 1.
Furthermore, the software and its versions that were used are summarized in Table 2. The leading software used, Unreal Engine, has many versions (4.27 - 5.3), but version 5.2.1 was used for this case study since this is the newest version that the plugins used in this study support. CityEngine 2021.1 was used to create rule packages. Lastly, ArcGIS PRO 2.8 was used to generate the input data, and updated version 3.2.1 was used to further edit the input GIS data.
Lastly, Table 3 lists plugins that were directly used while creating the 3D visualization of the Vltava River valley. Plugins tested but not used are mentioned in the text at the appropriate locations but are not part of this table.
2.2. GIS Data from Project Vltava
Two primary data sources were used to create a 3D visualization of the pre-dam Vltava River valley in the Vltava project. One source was dedicated to land use information (use of land, locations of buildings, roads, etc.), and the other was dedicated to creating DTM. Other smaller sources, such as period photographs, local maps, plans, etc., were also used within Project Vltava but not in this case study.
The first data set was created from scans of the Imperial obligatory imprints of the stable cadaster, shown in Figure 1a, which captures the land use information between 1821 and 1840. Roughly two cadastral areas on each side along the whole river valley (orange highlighted areas) were manually georeferenced, vectorized, and generalized. The final vectorization was then used as a source of land use data for the final visualization and as footprints for the creation of 3D models of buildings using procedural modeling. The final vectorization used in this case study is stored mainly as a geodatabase (*.gdb), and work data may be stored as shapefiles (*.shp). [1,3,6]
The second data set was created from the State Map 1:5000 – Derived, released in the 1950s. This map is the oldest continuous source of historical height data depicting the whole river valley. It contains height contour lines, as shown in Figure 1b. Map lists containing the area around the Vltava River valley (orange highlighted areas) were chosen and vectorized using a semi-automatic vectorization method. This vectorization was then used with other sources of height data, such as individual height coordinates, local maps, river maps, river elevation sections, and others. Those sources were combined and processed into one digital terrain model (DTM), which covers 1670 km2 from the river's spring to the border of Prag and is stored as a single GeoTIFF (*.tif) file. [2,4,5,26]
2.3. Importing of GIS Data
The first step in creating visualizations in the UE is the importation of GIS data. This step may seem trivial, but it’s the first step where problems occur because UE, by default, doesn’t support standard GIS file formats. There are, however, multiple ways GIS data can be imported using external tools, plugins, and extensions. The most common ways to import GIS data are:
- 1)
- Directly by dragging files into the content browser. This, however, only works for specific supported formats. This is comparable to ArcGIS Pro and importing data into the map from the catalog window (it may work simply by dragging files over or requiring a proper importing function to be used even for supported formats).
- 2)
- Using the built-in plugin Datasmith, the official way to import data from outside the UE ecosystem. It works in two ways: as an extension for external software called Datasmith Exporter or as a plugin for UE called Datasmith Importer. The extension is mainly used with CAD and 3D modeling software to allow exporting models in formats that UE can work with, while the plugin version is for importing files with supported file formats into UE.
- 3)
- Using third-party plugins such as Landscaping or TerraForm PRO. Those plugins mainly add functionality to import DTM data to create landscape actors and GIS vector lines to create landscape splines or blueprints. The disadvantage of using such third-party plugins is that they are paid (bought using the Unreal Engine marketplace) and may not support future versions of UE.
Since UE cannot import GIS file formats immediately, and there seemed to be no Datasmith extension for GIS software, the third option was to find suitable plugins to import GIS data from the Vltava project into UE. Unreal Engine Marketplace was used to find such plugins. This official marketplace is available from web browsers and Epic Games Launcher. It allows sharing and selling assets, such as textures, models, animations, code, etc., that can be added to individual projects or installed in the engine.
Two plugins were tested: Landscaping and TerraForm PRO, both allowing the import of vector data and DTM files. Landscaping is severely limited regarding coordinate systems since it only works with UTM-style coordinates. This caused the first problem of transforming all the data into an appropriate coordinate system. Secondly, it took a long time to import DTM, up to several hours if importing the whole 1670 km2 DTM (on the Poznań PC). Lastly, this plugin proved to be quite unstable when importing large datasets of vector data. TerraForm PRO offers the same functionality as Landscaping. The options and settings for importing DTM are comparable with the Landscaping plugin, with the advantage of not being limited to specific CRS. Importing DTM using the default settings takes roughly 15 minutes, which is a significant improvement over the Landscaping plugin; however, there seems to be a major problem with the resulting coordinates of the DTM. The imported DTM is transformed to match the engine’s coordinate origin (X=0, Y=0, Z=0), which means any attempts to import any other data outside of TerraForm, for example, 3D models from SketchUp using Datasmith, will not match with the imported landscape. There doesn’t seem to be a way to avoid this behavior. This means that if data from different sources needs to be imported using other methods, TerraForm PRO cannot be used. For those reasons, TerraForm PRO was rejected as a reliable form of importing GIS data for use within this study.
Luckily, during the work inside CityEngine, it was discovered that this software, also used in the Vltava project for procedural modeling, contains an inbuilt Datasmith exporter not mentioned in the official Datasmith documentation. This was significant because CityEngine is ESRI software, which allows it to import standard GIS data file formats and ESRI-specific GIS formats. Those can then be exported in Datasmith file format (*.udatasmith ). This method allows the export of DTM, 3D models, and shapes (vector data) into the UE. Importing *.udatasmith files using Datasmith Importer inside UE looks pretty stable compared to using plugins such as Landscaping reasonably fast. For example, importing 1670km2 of DTM with over 28000 polygons took 2 minutes to complete. After extensively testing all the methods mentioned above and plugins with the data from project Vltava and because of the ease of use, speed, and stability of CityEngine used as a middleman for importing GIS data into UE, an approach using the Datasmith exporter using CityEngine was chosen to be used within this study.
It is important to note that other ways to import GIS data exist, but the choice is heavily affected by the availability of licensing for CityEngine and ArcGIS Online. The options are CityEngine (licensed), ArcGIS Maps SDK for Unreal Engine to import maps and terrain directly from ArcGIS Online (licensed), and plugins such as TerraForm PRO (paid).
2.4. GIS Data from Project Vltava
During the initial testing of UE, it was found that most of the imported GIS data would need to be further edited in GIS to be usable or practical to use and work within UE to create a 3D visualization of the Vltava Valley.
While the DTM raster file containing height data of the terrain can be imported as a single file, reducing the amount of data is necessary to create a more stable and less memory-demanding visualization in UE. As is displayed in Figure 2a, the original one-piece raster contains a large amount of no-data space represented by a black background. This black area is still made up of pixels with their positions and data values. The only difference is that the value is set to zero (or other arbitrary number), which a GIS software would identify as such and would not display or use in calculations. This means that even though those areas do not contain usable data for the terrain visualization, they still contain data, thanks to how the raster data is saved. However, in UE and other non-GIS software, those pixels are taken at their face value, resulting in one big rectangular terrain with a size of over 12700 km2 (as in Figure 2a) compared to the 1670 km2 of the area of Vltava valley shown on Figure 2b, and result in terrain being created in locations with no valuable data. This results in UE rendering the 12700 km2 area, significantly reducing performance and increasing hardware demands.
Because of how raster files are saved, the way to avoid those empty (black) places is to split the original DTM raster into smaller rectangular rasters, where each raster fully contains part of the area of interest. Thankfully, the area of the DTM of Vltava Valley is primarily created based on SMO-5 maps and follows the shape of the SMO-5 rectangular map sheet layout. The division of the original raster into smaller rectangular pieces, which resulted in the creation of 68 individual rasters, is shown in Figure 2b. For the division of the DTM, first, the outline of the area containing the Vltava River valley was created using the Raster Domain geoprocessing tool. This Outline was then manually divided into 68 rectangles. The individual rectangles were designed to use as few as possible but made them similar in size, requiring some larger areas to be further split into smaller rectangles. The aim was to create rectangles covering the whole area but not containing any areas outside the Vltava River valley. The final rectangles are shown in Figure 2b in orange color. The next step was splitting the original DTM using the Split Raster geoprocessing tool. The resulting rasters were tested in UE, and through an iterative process of trial and error of partially building the visualization, the best parameters for the final raster attributes were found. First, it was found that there needs to be an overlap between the rasters to ensure the continuity of the terrain. However, the overlap should be as small as possible since, in the later stages of the creation of the visualizations, the procedurally created 3D objects like trees may become too dense in the areas of the overlap. Furthermore, due to the follow-up processes done on top of the terrain, it was decided that the individual raster's precision (pixel size) should be increased from 5m to 1m.
While this precision increase may seem counterproductive in terms of the reduction of memory demands of the visualization, the removal of the empty spots of terrain means the removal of more than 80% of the raster size, and together with the later use of World Partition more than makes up for it. As is shown in Table 4, while the upscaled DTM takes 25 times more memory due to the increased pixel count, after removing the “no data” background, the size is only 3,3 times bigger than the original. On average, the file sizes of the 68 DTM pieces are in the tens of MB. This is significant because later in the process of creating the scene in UE, a process called World Partition is used, which renders only objects near the camera/player position. This means that only a few of the 68 DTM pieces are rendered at a time, leading to lower memory usage than the single original DTM with a resolution of 5m. The final 68 DTMs allow further use of world partition (tiling) in UE and, together with the outline polygon, are further used for editing and creating land use data.
Secondly, rasters used for importing terrain into UE through CityEngine should be a 32-bit float pixel type, which is significant information since the rasters used for terrain masks used for applying textures require different pixel types (16-bit unsigned). If the terrain raster had only 16bit depth, the created terrain in UE would be made of flat pixels resembling stairs, creating an unsmoothed terrain model, which would require either a smoothing pass for the entire landscape in UE or recreation of the imported raster in GIS.
The second dataset was created by manual vectorization of scans of the Imperial obligatory imprints of the stable cadaster, which captures the land use information between the years 1821 and 1840. While in the original GIS visualization, the whole vectorization shown in Figure 3a was used thanks to the global height layer available in ArcGIS Online, this layer is not available for use in UE. For this reason, the first change to the vectorization was to reduce its scope to match the area covered by the DTM, as shown in Figure 3b.
This was done using the Clip geoprocessing tool with the DTM outline polygon shown in Figure 3b as the clipping feature layer. After reducing the land use layer to the area of DTM, the whole vector layer had to be converted to raster. This is because while UE can import and display vector data, it converts all polygons into 3D splines. This creates a vast number of actors, which causes performance issues. On top of that, those do not interact with the terrain in any way and cannot be used to visualize land use on the terrain. In comparison, in a 3D GIS scene, 2D layers are projected on top of the terrain, and with the help of Symbology, those layers can apply textures to parts of the terrain according to the land use information. In UE, however, the way to apply textures (in UE, called materials, which on top of textures contain further details regarding physics and lighting-related attributes) is vastly different; terrain (in UE called landscape) can only have one single material assigned to a landscape. However, this material can be customized using a Blueprint visual scripting system to create multiple layers with different attributes that can be assigned to various landscape parts. This is available as a paint option, allowing manual landscape painting with those layers. However, to use them on a larger scale, an option to import the layer´s area in a format of raster masks must be used. This means that the whole set of land use data had to be converted into a raster form of the exact dimensions as the original landscape, and specific file formats with specific naming schemes had to be used to apply different textures to different areas depending on the land use information.
Before converting land use data into raster form, it is essential to understand how the final raster is used inside UE so it can be tailored and changed beforehand. Typically, rasters in UE are used in two ways: to import the landscape itself or to apply textures (materials) using landscape attributes like slope or aspect in raster format. These are then used to apply materials and create seamless material blending between the individual mask layers. The individual pixel values of the mask are either used as a value of fact (for example, slope values) or as a weight for material blending. This visualization uses a process similar to material blending in the importing sense. In this instance, the land use rasters in the UE are applied to the individual landscape parts as a raster mask. From this comes the first change made to the land use dataset: aggregation of all the separate areas of the same land use under one ID. The goal was to make all the regions of the same land use look the same in UE. Since there were no plans for material blending and there is no need for differencing between different areas with the same land use, having different values for each polygon represented in the final raster is unnecessary and may cause problems in UE when applying materials and working with tens of thousands of polygons in GIS (may) cause instability, cause errors when using geoprocessing tools and other problems. Due to those reasons, all polygons of the same land use were aggregated (merged). After aggregating all the values of the dataset by the land use type into fourteen groups using the Select by attribute tool, merging them, each land use record representing the individual way the land used was converted to an individual raster format, creating 14 rasters in total using the Polygon to Raster geoprocessing tool. However, some rasters, such as water bodies or buildings, were not used further since those are modeled in the final visualization in different ways. Another reason for the aggregation was that using the Polygon to Raster geoprocessing tool with selection as part of the input feature creates one raster for each selected polygon, creating thousands of rasters, which is unnecessary and unsuitable for further data processing.
What needs to be highlighted is the use of Snap Raster option, which is mandatory at this point of the process because, without it, the output rasters would have different dimensions than the original DTM raster(s), where the areas on the edges of the processing extend may be cut out (for example, because there may not be any forest in the border areas during the creation of land use raster of forests) creating problems further down in the process where the final land use rasters would be difficult if not impossible to correctly align with the landscape in UE during the import of the land use rasters as raster masks.
After creating the individual land use rasters displaying each land type (forests, fields, gardens, etc.) in this way for the whole Vltava River valley, they were each split to match the rectangular areas of the 68 DTM parts. The process for this further division was very similar to dividing the original DTM using the Split Raster geoprocessing tool. The difference here was the use of the Snap Raster option, where the rasters used for snapping were the individual 68 rasters, to ensure that the resulting land-use masking rasters would match the individual DTM rasters.
2.5. Work inside Unreal Engine
The first step in creating the visualization of the Vltava River valley in the UE was to import terrain rasters as a landscape. This was done using Datasmith Importer with CityEngine as a middleman. After the landscape layers were imported into UE and datasets were prepared in the GIS, further preparations on the UE side were necessary to import the land use data as raster masks. Firstly, to correctly display and work within UE, many settings needed to be tweaked/changed to ensure maximal visual quality and functionality of the used processes and, in general, help with creating the visualization both in UE editor and within the final visualization. Most required settings are mentioned further in the text at appropriate places. However, for ease of use and the ability to set everything beforehand, all used Editor Preferences and Project Settings settings are listed in Table 5 and Table 6. The remainder of this chapter describes the steps to prepare for the remaining data to be correctly imported and processed in the UE and the steps made within UE to create the final Vltava River visualization.
The first step is to display and use the land use data on top of the imported landscape. To do this, landscape layers within landscape material had to be defined. For this purpose, a plugin named Landscape Pro 3 was used. This plugin contains already-prepared materials for the landscape visualization and many textures, 3D models, actors, and other assets, as shown in Figure 4.
Those were needed to visualize the land use information and procedurally generate grass, trees, rock formations, and other 3D models based on it to fill up the landscape and make it look organic, realistic, and detailed. After enabling the Landscape Pro 3 plugin, its landscape material was applied to the imported landscape by dragging the material file from the Content Browser plugin directory LandscapePro_v3 > Environments > Landscape to the landscape material attribute inside the landscape actor Details panel. This material file contains the definitions of each material layer, where each layer is made of parameters, textures, normals, masks, and functions. Those layers are then blended to create the final landscape material. Furthermore, this material file contains a Grass Output section that, depending on what layers are blended, procedurally generates ground foliage and 3D objects on top of the landscape. After applying the material, Layer Info must be manually created. To do so, the UE editor needs to be switched to Landscape Mode, and then under the Paint tab, for each layer, Layer Info must be created using the Create Layer Info (the + icon next to each layer) as shown in Figure 5.
After the Layer Info is created, the base (first) layer will be applied to the selected landscape, which will result in textures being applied where primary terrain attributes like slope are taken into account to apply either rock or grass textures, as shown on Figure 6, together with procedurally generated foliage matching the texture.
Unlike the base layer, all the other layers (from layer two upwards) are not applied automatically and must be applied manually or imported as raster masks. For that, each layer needs its own Layer Info to be created. If a layer is missing Layer Info, the importing feature will be greyed out, and no import will be allowed. The importing of layer masks is located under Landscape Mode in the Manage/Import tab. Each layer with Layer Info allows importing raster files as raster masks. Rasters must match specific requirements that are not readily apparent to be imported as layer masks. After all those requirements are met, the rasters can be imported as material layer masks, after which the layers are automatically applied, adding textures and generating foliage and other objects. The requirements are as follows.
- The file format of the imported rasters needs to be RAW or PNG
- The rasters must have a color depth of 16bit (greyscale) with unsigned values
- All imported rasters need to have unique names within their directory
- No auxiliary files can be present in the same folder as the imported raster
- Rasters must contain an alpha channel or the file format may not be recognized
Another step in the creation of landscape visualization is filling it with content. In this case, it is vegetation and its procedural generation. This step is tightly tied with the landscape materials and material layers. The plugin used for their creation also contains procedural generation capabilities. In this case, the procedural generation of vegetation is done in two ways: globally generating ground vegetation and objects on the whole landscape and locally generating vegetation based on individual land use layers coverage.
The global procedural generation that applies to whole material layers is defined directly within the material layer. It aims to generate the vegetation that fills the entire landscape depending on the layers used. The first layer is the base (main) layer for the whole landscape. It doesn’t require location definition as the other layers do and is applied automatically after the material is assigned to the landscape. After the assignment, it applies textures and generates 3D vegetation to the landscape. The generated vegetation consists of small and tall grasses, flowers, bushes, wooden branches, and other ground foliage shown in Figure 7, which is fully animated.
Furthermore, the vegetation placement considers the terrain slope and adjusts which and how dense vegetation will be generated. This process is fully automated and doesn’t require any further actions. This base layer can be overlayed at specified locations with the other defined layers, such as layers representing forests, whose locations are imported as land-use raster masks. If the non-base material layers are applied over the base layer, a blend of those layers is created, resulting in the generation of foliage specific to the layers blend as is defined in the Grass Output part of the material. This feature, however, is not used because of how the original land use data (vectorization created in GIS) was made. It does not contain any overlaps between individual entries, resulting in no possibility of such layers blending in UE outside the base grass layer.
To procedurally generate any of the larger objects, such as broadleaf and needle trees, rock formations, and others, Procedural Foliage Spawner actors must be used. The spawner creates a bounding box, shown in Figure 8a, that interacts with specific predefined material layers. This results in procedural generation being only done on top of the matching material layer and within the bounds of the bounding box. Furthermore, the spawner controls what objects will be generated, as shown in Figure 8b. However, the Procedural Foliage option must first be enabled in the Editor Preferences to make procedural generation using the spawners work. Without it, the spawner actors cannot be added to the scene. Another tweaking was done within the Project Settings, where the anti-aliasing method was changed to Temporal Anti-Aliasing (TAA) to remove flickering and shimmering effects from the animated foliage to create organic, realistic, and detailed foliage as in Figure 9.
The primary way of creating 3D models of over 28000 buildings was procedural modeling done in CityEngine, as shown in Figure 10. However, those are not part of the input data for this case study since those models were only exported in a format compatible with ArcGIS Online at the time of their creation. Since one of this study's goals is recreating Vltava Valley's visualization, recreating those 28000 models is also needed. The easiest way to create those models again is to use CityEngine and import them, either in the *.udatasmith format or any other format supported by UE (for example, glTF). However, because the case study aims to explore the usage of GIS data within UE, another solution has been used.
CityEngine is 3D modeling software locked behind a license but has a freely downloadable software development kit, CityEngine SDK. This C++ API allows for implementing procedural modeling, as is used within CityEngine. Based on this SDK, a plugin for UE was developed named Vitruvio. Vitruvio is free for personal, educational, and non-commercial use; however, as input, Vitruvio requires Rule Packages (RPK), which are created in CityEngine and require a license [27,28]. While using Vitruvio, the procedurally created buildings stay procedural and can be changed easily with a parametric interface (height, style, and appearance). An RPK file includes assets and a CGA rule file, which encodes an architectural style.
To use Vitruvio, polygons that represent footprints of the modeled building must be imported into UE. This is done through CityEngine and Datasmith Importer. This creates Actors for each imported polygon in the project. Once the polygons are imported, a new action menu option will appear: Select all viable initial shapes in the hierarchy. This will select all Shapes within the selected Actors, which can then be converted into Vitruvio Actors with the procedural modeling functionality. Before conversion into Vitruvio Actor, the RPK file must be present within the project. To do so, the RPK file created in CityEngine must be dragged into the Content Browser. After selecting from the context menu, Convert to Vitruvio Actor and select the desired RPK file. All shapes will be converted to Vitruvio Actor, and the original polygon will be modeled into the desired shape according to the RPK file, which, in this case, will be fully textured models of buildings.
3D modeling over 21000 buildings was quick and took a couple of minutes on both tested machines. This is comparable with procedural modeling done in CityEngine. The process was also stable during the testing, and repeated importing and procedural modeling were performed on both tested machines. After converting Shapes into Vitruvio Actors, it is possible to change the attributes used for procedural modeling using the Vitruvio Actor details inside the VitruvioComponent section under Attributes. The implementation is very similar to procedural modeling inside ArcGIS Pro, which is available under polygon symbology as an option under the fill symbol layer. Both methods use CityEngine RPK files to create models procedurally based on the polygon footprint. In both cases, each will have its attributes created based on the rule file that is part of the RPK. Those attributes can be changed to edit the look of individual models. Neither of those implementations (UE or ArcGIS) allows for creating or editing the original rule file or alerting the original assets that are part of the RPK, which is limited only to CityEngine in which the rule file was created. The UE version also contains more options related to physics and material, which are unique to game engines.
The last unique step for the UE is the conversion from Vitruvio Actors to Static Mesh Actors. This process preserves the changes made to Vitruvio Actors and saves the procedurally modeled objects. Without this conversion, Vitruvio Actors would continue to exist until the project's closure, after which they would cease to exist, and the original shapes would need to be imported again. During the conversion of Vitruvio Actors to Static Mesh Actors, a process of so-called Cooking is used. UE stores content assets in particular formats that it uses internally, such as PNG for texture data or WAV for audio. However, this content needs to be converted to different formats because the platform uses a proprietary format, does not support Unreal's format used to store the asset, or a more memory- or performance-effective format exists. The process of converting content from the internal format to the platform-specific format, together with compiling shaders, is called cooking. This process can take longer with more complex assets (like meshes with lots of polygons, etc.) [29]. This process is done the same way as the initial conversion, meaning that after selecting all Vitruvio Actors in the context menu, there will be a new option: Convert to static mesh actors. Unlike procedural modeling, this process can be volatile due to its very high hardware (memory) requirements, and the time needed is also severely longer, where finishing these conversions takes hours. Solutions to these problems are to split the number of Actors processed at a time into smaller chunks, in this case, 10000 at one time, or to increase available memory either by installing more RAM or significantly increasing the size of the system's virtual memory via increasing the page file size in Windows.
During model cooking, the CPU and memory are utilized significantly. While the CPU speeds things up, insufficient memory leads to UE crashes. Resource Monitor reveals that UE commits over 170GB of memory just for itself at the time of cooking.
Another part of the visualization is creating water bodies. To add and work with water, a plugin of the same name, Water, is used. This plugin is built into the UE and is currently in the experimental version, as shown in Figure 11.
This plugin brings multiple types of water bodies to the UE. Still, only two are being used to visualize the Vltava Valley: Lake for modeling smaller water bodies like fish farms, water reservoirs, or ponds and River for modeling the Vltava River and smaller streams, as shown in Figure 12.
The simpler of the two is an Actor named Water Body Lake. This actor creates a lake defined by three Spline points with the possibility of adding additional spline points. Those spline points are connected by arc-shaped curves whose attributes can be changed using the detail panel or graphical representations of these attributes directly in the editor scene. This actor is more straightforward than a river because a lake only has one height value (the lake must be flat). Specific attributes like the speed of water flow don’t matter for small-scale water bodies, and there are significantly fewer attributes in general. By default, the lake actor contains the attribute option Affects Landscape, which allows the lake to interact with and change the landscape it is situated on. While this feature can be disabled, it must be used since the smaller water bodies for which the actor Water Body Lake is used are not modeled within the DTM. In the original datasets, they are only represented by land-use vectorizations and, consequently, by texture (only their position is known). The lack of lake bottoms in DTM and their representation only in vectorization create additional problems. Since many water bodies contained in the land use vectorizations are made of sharp angles, which are significantly harder to achieve using the spline points, it’s lengthy and complicated to match the position of the banks with the vectorization perfectly. Furthermore, the exact measurements of the bottom of those water bodies (depth, angle, etc.) are unknown, at least in the datasets used for this modeling. This results in higher creative freedom in how the water bodies are created; however, it is at the cost of historical accuracy and the accuracy of following the base GIS data, in this case, the vectorizations of old maps.
The actor named Water Body River is the more complex one. Like the lake actor, it creates a river defined by multiple spline points; however, those have many more associated attributes. Firstly, each spline point has its height (Z coordinate), making the manual creation of rivers more accurate but at the cost of the increased difficulty of working with the river actor. Furthermore, each spline point has its depth point, river width, flow orientation, and flow strength/speed. The actor Water Body River also contains the attribute option Affects Landscape; however, in the case of the Vltava River, the situation is reversed. Since the used DTM already includes the bottom of the river basin, there is no need to alert the terrain. Furthermore, doing so could lead to terrain continuity problems, where it could/would be challenging in specific spots to make the terrain changes made by the river actor unnoticeable and seamless with the original terrain. Another reason not to use the Affects Landscape option is that by not using it, the whole workflow of creating rivers can be significantly simplified. Since the water level is always lower than the surrounding terrain and the bottom of the river is already modeled within the DTM, those attributes can be made purposefully bigger to fill the river basin. This reduces the whole process of creating the river by removing the need for having to set up each single spline point with depth, width, and terrain angles to only creating the individual spline points and setting their location, which is already a significantly lengthy process considering the shape of the river and its length and the fact that the spline points are all created and positioned manually. However, a combination of both Affects Landscape options is used for creating streams according to individual stream locations and the terrain around them. Since those streams are substantially smaller than the Vltava River, using the Affects Landscape can help highlight the stream in the terrain, which is welcomed in certain situations.
After the scene is filled with content, meaning it contains the finished landscape with applied material layers according to the land use information, with procedurally generated vegetation on top of it, all the water bodies, and the procedurally modeled buildings, the last step is to package the whole scene. Packaging a project is the final step, which allows the scene to be opened via an executable file and no longer requires UE to be installed. The first step to packaging the project is to fill out all the information in Project Settings under Project – Description, namely Project Display Title (also used by the operating system), publisher information if applicable, project version, and others. Next, choosing default maps (levels) to be opened in the Project – Maps & Models section is crucial. Lastly, the Build Configuration needs to be set to Shipping in the Project- Packaging section, allowing the opening of the final packaged project without needing UE. All other settings can be changed per the individual project’s needs; however, they are left with default values for this study. After the project is set up for packaging, it can be selected from the main toolbar under the Platforms, as shown in Figure 13.
However, this step has excellent potential for causing errors and failure. The most straightforward errors to solve are those related to required SDKs, which can be fixed by installing and updating .NET Core 3.1, .NET 9.0, Windows SDK, and Visual Studio 2022. Another set of problems may arise from deleted and unused content within the project. The packaging process packages all levels, including any (testing) unused levels. The problem is that those usually contain invalid references to deleted content, which will cause the packaging to fail. For this reason, it is essential to “clean up” the whole project before packaging, for example, using the Content Browser filtering option Not Used In Any Level. After all those steps are completed and potential errors are resolved, the project packaging process can be completed. It is important to note that the whole packaging process takes a significant amount of time and fully utilizes the computer CPU for compiling shaders and packing the project per the chosen options in Project – Packaging within the Project Settings. After the packaging process, the application executable file and all the packaged assets are created within the selected folder. This application opens the previously set default level of the created visualization with basic movement and camera controls. Within the application, no further modifications to the visualization can be made (outside of programmed interactions); however, the project within UE can still be used and worked on, and a new version of the application can be packaged.
3. Results
The case study provides information that is usually missing in similar projects. It describes the used GIS data, its origins, used software and plugins, and delves into the recreation of the 3D visualization of Vltava River valley in Unreal Engine in detail with focus on practices for importing and processing GIS data in UE, techniques for creating realistic-looking scene as shown on Figure 14, performance optimization techniques for ensuring a seamless viewing experience, and possibilities for sharing and or publishing of the resulting visualizations and lastly by focusing on the workflow of 3D visualization in general. Furthermore, the results of this case study compare the individual parts of the visualization process, requirements, and suitability with visualization created as part of the Vltava project using GIS software. This allows readers to understand better what processing and visualizing GIS data in UE entails and how it compares with GIS so that they may gain the basic knowledge to start creating visualization works in UE, should they choose to.
3.1. Summary of the Visualization Workflow
First, before any attempts to create the visualization in UE were made, the question of importing GIS data into UE had to be answered. For this purpose, multiple ways to import GIS data were tested and discussed. While each way to import data has pros and cons, importing using CityEngine was used. This method allows the import of all required GIS data types (3D models, rasters, landscape splines, etc.) with the correct coordinates and relative position to each other, which was the major problem with the importing solutions using plugins, where the importing process transformed the data, which causes data types that were unsupported by the individual plugins to be imported separately.
After importing the first GIS datasets, the process of creating the Vltava River Valley visualization began by trying to work with all the unedited GIS datasets and using an iterative process of trial and error to progress further with the creation of the 3D scene in UE. Each hurdle in this process was then documented and, in many cases, resulted in the need to edit the base GIS data in GIS software to better suit the UE’s needs. The changes to the original GIS data can be summarized into optimization and functional changes. Optimization removes unnecessary data and primarily reduces memory usage in UE, while functional changes make the GIS data usable for use in UE.
The next part of the article aims to describe and explain each step required to create the basic visualization of the Vltava River valley. It began with the problem of using GIS vector data as a source of information for applying textures to the terrain in UE. In GIS, this is done by simply projecting polygons onto the terrain with their symbology. However, UE doesn’t work that way. Instead, polygons must be applied as raster masks directly to the terrain rasters. For this reason, the precision of those rasters had to be increased to allow smaller areas and detail in the vectorization not to be lost during the conversion from vector polygons to raster pixels. Those raster masks were then used for procedural modeling of vegetation via the used plugin´s material. Another step was to create models of more than twenty thousand buildings by importing vector footprints and procedurally modeling them in UE via a plugin Vitruvio (based on CityEngine SDK) to test the plugin's functionality and avoid problems with importing models.
One of the parts of the visualization process is vastly different from the others: the creation of water bodies of rivers, lakes, and streams. Unlike the other parts of the visualization, which were at least partially automated, the water bodies were all created fully manually using the built-in plugin Water. This is the only part of the visualization that doesn’t precisely follow the underlying GIS data. While there was an effort to follow the available underlying GIS data, due to the limitations of the manual creation of the water bodies, at many places, an approach based more on the visual aspect of this visualization was used, where the water line follows but not fully match the original land use data. Furthermore, during the visualization process of some water bodies, the terrain had to be altered and remodeled since the DTM data was incomplete and did not contain the bottoms of water bodies like lakes.
The last part was the creation of an application that contains the visualization of part of the Vltava River valley outside of the UE editor. This was done using project packaging with the default settings for the flyover controls. This was done due to the upcoming plans for this visualization to be used as a basis for further applications such as VR or prerecorded animations, which don’t use traditional controls (animations) or require specific hardware (VR).
3.2. Comparison of GIS and Unreal Engine Visualizations
One of the arguments for recreating the Vltava River visualizations in UE was the possibility of achieving more realistic graphical fidelity. For this reason, this section contains image comparisons representing the resulting visualization created as part of the case study in Unreal Engine and images from the original GIS visualization from the exact locations, focusing on individual content types (terrain, vegetation, buildings, water). For each of those comparisons, an explanation of the differences is provided.
The most basic comparison comes in the form of a faraway view of the broader landscape. Figure 15a is the final GIS visualization hosted on the project Vltava website. This GIS visualization uses bright colors to differentiate between different land use areas and lacks any form of shadows due to the static lighting directly above the scene. While this can be changed inside ArcGIS Online, the final visualization hosted on the project's website doesn’t have this feature. On the other hand, the image from UE uses more realistic textures, which are noticeably darker, and uses dynamic light that helps to define the terrain, as shown in Figure 15b.
While from the air, those visualizations look very similar, upon moving closer to the landscape, this changes drastically, as shown in Figure 16a; in GIS, the trees are simpler and behave differently in terms of LOD. In GIS, those trees use a simple model without regard to the distance rendered; however, density changes based on distance. On the other hand, in UE, the density stays the same, but the model changes based on distance, which is shown in the second image of Figure 16b. The LOD in UE can be split into three levels: The closest trees have full details and shadows, trees further away are simplified and have the shadows removed, and trees in the far distance are removed.
What can also be noticed in Figure 16b is the slightly different position of the river. This is caused by the terrain layer, which contains some errors caused by over-smoothing of the terrain during its creation, that could be overlooked in GIS since the river there is represented only by the texture, which is projected to the terrain. However, in UE, since the water is a physical object, it is not feasible for the water to flow uphill or not downhill, and as such, it is in different positions compared to the land use data. While this could be fixed by manually editing the landscape, it has not been corrected because of the size and a new version of the terrain already in the work within Project Vltava.
In GIS, outside of the trees, which are disabled in the visualization by default, there is no other form of foliage or vegetation, and in general, the GIS visualization is not optimized for up-close viewing. This is showcased in Figure 17a, where bare ground without any ground foliage and the pixelated division between two areas (forest road and meadow) can be seen. On the other hand, the UE visualization in Figure 17b is covered with details that are better suited for viewing from up close. While there is currently a problem in UE with distinguishing the borders between areas because of the use of rasters as raster masks for applying textures to the landscape, which may be apparent in specific situations, the blending between neighboring areas is an advantage in others.
In terms of building models, there are only minor differences. From above, the buildings seem the same, but up close, the models look better in UE, even though they are the same models created in the same way. This is showcased in the Figure 18Error! Reference source not found.. The cause for this sudden upgrade in quality is unknown. However, a better rendering technique, shaders, shadows, or lighting may play a role. The only downgrade to the building models in UE comes in color due to different light settings, exposure, and texture attributes in UE.
4. Discussion
During the literature research, it was found that the majority of projects that visualize large landscapes (small projects workflow is unsuitable for large projects due to the differences in approach to optimization, unreasonable levels of manual work, hardware requirements, etc.) that were found during the literature research provide little or no data and data processing procedures and focus primarily on the introductory information and images of the final visualizations which were shown on two large projects, creating a gap in the literature regarding the data processing process. This is a significant problem, as other researchers cannot validate the procedures used or suggest improvements. Furthermore, new researchers starting with game engines have nowhere to adopt already proven procedures. A case study was created to help bridge this gap and find more information to answer each research question.
4.1. Comparison with Other Projects
4.1.1. Rome Reborn
Rome Reborn was an ambitious project, currently in version 4.0, that aims to digitally recreate the ancient city of Rome as it appeared at various points in history [30]. However, only the first half is somewhat documented before the project changed course and became a closed-source paid product with version 3.0 in 2019 [31].
In the first versions of Rome Reborn, which formed the basis of its visualizations, various software tools were employed, including 3D modeling software such as 3DS Max and Maya and procedural modeling techniques using CityEngine. Similar to the Vltava project, buildings were split into two categories: one was precisely modeled using 3D modeling software, and the other (made up of primarily common houses) was created using CityEngine. [23] CityEngine is a significant help in modeling buildings for which only basic information is known. Furthermore, procedurally modeled buildings are often of low importance but in large quantities for which manual modeling using CAD or 3D modeling software would be highly time-consuming.
In terms of scale, it is safe to say that the visualized area was smaller than the one in the Vltava Project and smaller than the attempted scale of this case study. Rome Reborn focuses on an urban area with a high density of buildings (7000) and a high density of significant buildings (250) [23]. In this case study (under the Vltava project), even though the visualized area is mainly made of rural areas with large quantities of dense fields and forests, it has overall more procedurally modeled buildings (over 21000), although in significantly less detail.
The use of game engines, namely Unity, came in Rome Reborn 3.0, which means that practically no information on what and how has been done in Unity, which causes a significant gap in the literature regarding this project. From the available images and the project webpage, it can be deduced that many assets from previous project versions were imported. At a minimum, a VR application was created for the virtual tours that are now offered for sale. However, due to this lack of transparency, any new DOI-issued articles regarding Rome Reborn after version 3.0 was released only contain basic descriptions of the project and the virtual tours [24], so much so that as a source of information for any other project, they are useless and only serve as adds for the sale of the virtual tours. This is probably the most significant difference between Project Vltava, within which the results of this case study will be freely available. This study aims to provide as much information as possible regarding using the UE as a visualization tool and to bridge those literature gaps created by projects like Rome Reborn that only show what is possible but don’t show how.
4.1.2. Virtual Angkor
Virtual Angkor is another significant project that aims to digitally reconstruct the ancient city of Angkor in Cambodia. While not all information regarding this project is public, compared to Rome Reborn, significantly more is available. The project's goal was to create a virtual representation of Angkor during its peak period in the 12th century and showcase the architectural wonders of Angkor Wat, the Bayon Temple, and other iconic structures, as well as the elaborate urban infrastructure and water management systems that characterized the city at that time. [21,32] Virtual Angkor created three main applications.
First, the area of Greater Angkor (3500 km2) was visualized using a GIS-based solution that incorporated LiDAR data, DEM, and GIS mapping layers. Furthermore, a significantly smaller area of Angkor Wat (the exact area is not defined, but from available sources, the estimate is around 2 km2) was visualized using the Unity game engine. Like many other projects, including Vltava, the Virtual Angkor began with works outside game engines that were later imported into Unity to transition away from 3D scenes built to render single images and animated sequences. This approach of visualization of large areas in GIS and smaller areas more in detail using game engines was also utilized in project Vltava. However, this case study aims to recreate the whole area in Unreal Engine, which is used to evaluate and highlight the scalability of UE in terms of the visualized area instead of highlighting the depth of the possibilities of the game engine, such as is showcased in the Angkor Wat Unity scene.
Second, Virtual Angkor capitalizes on the sheer number of game engine capabilities. It creates a profoundly detailed simulation of Angkor Wat with fully animated and living characters going through their daily routine. While neither this case study nor the Vltava project has the ambition to create a simulation of living beings, it highlights the capabilities of game engines.
Third, virtual reality is one of the primary forms of visualization for Virtual Angkor. It provides intractability and allows users to immerse themselves in the fully simulated area of Angkor Wat. While a VR application for the project Vltava is likely to be created in the future, it has not been done for the current case study due to the sheer amount of work with the base visualization, which needs to be completed first.
Sadly, even though the Virtual Angkor project provides quite a lot of basic information, it is still not valid as a source of information for any other project to follow in its steps. Once hosted on the official project website (based on multiple archived internet articles referring to it), the official GIS-based interactive map has been stripped away. Furthermore, the official website only contains descriptive information with no hint of any workflow that could be followed, either for recreating the GIS-based interactive map or the simulation in Unity. Next, the sources listed on the project website are tied to the area, but it is not stated that those are related to the project. The only things that stand to interest on the project website are teaching modules, which do not provide information on where they are deployed or how they could be obtained for education and 360° video views. Sadly, this approach of locking everything from public access and not even providing basic workflow or some other way to share the research behind those projects further aids in creating a literature gap, where new research is purposefully gated by lack of usable documentation from similar research.
Yet again, the most crucial difference for the further development and usage of game engines is the aim of this study, which is to provide as much information as possible regarding the use of the UE as a tool for visualizations and to bridge those literature gaps created by projects like Virtual Angkor that only show what is possible but don’t show how.
4.1.3. Virtual Scene of Poyang Lake
A case study of Poyang Lake proposes a workflow of game engine-based virtual wetland scene construction for the rapid modeling of virtual wetland scenes. The visualized area averages 3500 km2 (depending on the season) and consists mainly of the wetlands of the Poyang Lake [20]. Compared with the Vltava case study, the visualized area doesn’t contain buildings, trees, or area-wide grass coverage. Instead, the region comprises wetlands, specific grasses, and plants. While this may not sound like a big difference, considering that a big part of the area is filled with a lake (deep water without any foliage modeled), this results in significantly lower performance required and thus allows visualization of an even bigger area than in the Vltava case study, and as such can be used with less powerful PC.
In the lake study, UAV-gathered data is processed into information about the wetlands to create the informational basis for setting individual material layers. Information such as soil type and height above or depth underwater level are used to determine which layer with what weight will be applied and blended. This is then used to generate specific types of grasses or plants. [20] The Vltava case study uses a similar process of assigning material layers for slightly different purposes. Land use information from vectorized old maps is used as the basis for applying material layers, which are used for the procedural modeling of buildings and the procedural generation of vegetation.
The lake study uses various virtual reality solutions and technologies to display the virtual wetland scene and realizes multiple forms of virtual experience. One such solution is hosting the visualization online through WebVR and HTML. This is an interesting choice since the lake study was published in 2023. However, the support for rendering to HTML has been deprecated and removed from UE with the update 4.24, released on December 9, 2019. While the possibility of hosting the visualization from the Vltava case study on the project webpage sounds enticing, since the Vltava study is made in the newer UE version (5.2), it would require downgrading and porting of the content, which may be significant work, not to mention problems with used plugins and their compatibility issues.
The other VR solutions in the lake study include a standard head-mounted display and a CAVE (Cave Automatic Virtual Environment). While a standard headset VR implementation is planned within Project Vltava, the case study focuses on the visualization's construction.
In conclusion, the case study of Poyang Lake, while having different goals and using different types of data, uses similar techniques that achieve, in theory, the same results. To automate the process of placing objects in the visualization, in the case of the lake study, it would be wetlands-specific plants, and in the case of the Vltava case study, it would be general foliage and vegetation; both studies utilize material layers. While in the lake study, those layers are based on terrain attributes obtained from UAV scanning, such as soil type, terrain angle, and height (depth), the Vltava case study obtains those layers as terrain uses information obtained from vectorized maps. The big difference between the two is in the resulting forms in which the visualizations are presented. Currently, the Vltava case study only creates a PC application with a VR application planned for later, while the Lake study provides only VR applications for multiple VR solutions.
4.2. Possible Improvements to the Case Study Visualization
As part of the case study, a portion of the Vltava River valley area was visualized using UE. Although only a part of the whole area was finished at the time of this article's writing, it is being further worked on within Project Vltava. Several other shortcomings within the visualization process were identified.
One oddity of this case study is the shape of the visualized area and the fact that different types of data (terrain and the rest) don’t match the coverage of each other; in fact, the used terrain is, in many places, smaller than the land use data. However, this discrepancy is not visible within the original GIS visualization, and the full extent of both datasets is used. This is because, in GIS, a secondary terrain layer was used, namely the global terrain layer, which is present in ArcGIS Online. The fact is that this layer was supposed to be part of the case study at the beginning, imported using the ArcGIS Maps SDK plugin, and would add another example of ESRI’s pursuit of connection to the game engines. However, after adding this plugin to the project, it wasn’t working correctly, and subsequently, it was removed from the case study entirely due to time constraints. By adding this surrounding terrain, more of the original datasets could be used for even bigger visualizations. This would, however, significantly impact performance and hardware demand but could also further highlight the capabilities of game engines to work with such a massive amount of data.
The original GIS visualization also contained a layer with the water levels caused by the individual dams to showcase which areas of the Vltava River valley are nowadays filled and underwater. This is currently not present in the UE visualization because of the time requirements for creating water bodies. The addition of the current dam water level could further showcase the capabilities of game engines, such as the possibility to run simulations of what could happen if such a dam broke and how the individual objects that are currently underwater would react to the flooding of the dam reservoirs, etc. This would, however, require further programming and development of new functionality for visualization to make those things work.
A process that hasn’t been used within the case study that could improve the visualization in a significant way is the use of Landscape Splines. Landscape splines work similarly to river body actors used in the case study. They are made of spline points and possibly other adjected points that can symbolize width or other geometrical attributes. However, instead of filling the vicinity of the spline with water, it is filled with static mesh, such as a road. This means that depending on the used mesh actor, roads and other linear objects created in this way could be entirely made in 3D, meaning there would be an actual 3D mesh model of a road, possibly with other features like alleys of trees, bollards, and other barriers and objects. This method would significantly improve the visual fidelity of the visualization, considering the number of roads or other objects that could be created in this way. However, as with water bodies, this would most likely mean not strictly following the underlying land use data and omitting some details like road widths because of the sheer number of roads. Manually setting attributes like width for each road would be overwhelming, if not impossible. Furthermore, this would require significant work in GIS to extract those central lines, split them into individual roads, and remove roads where this approach would not be suitable.
4.3. Suitability of Unreal Engine for Visualization
This article establishes that game engines can indeed be used to create realistic-looking 3D models and scenes for large-scale landscape visualizations using GIS data. However, game engines, in this case, UE, are very different from GIS and other software that may have been used for creating 3D visualizations before. Thus, another question arises: How should game engines be used, or more precisely, what for?
To answer this question, it is essential to look at the differences in the outputs of GIS and UE. The GIS visualizations from Project Vltava are hosted online via ArcGIS Online, allowing the visualizations to be viewed by anyone with an internet connection and browser without downloading. Furthermore, the viewing doesn’t require a powerful computer since the visualizations created in GIS are usually way simpler, with less focus on graphics and more on transferring information via the visualization. On the other hand, UE visualization doesn’t have this opportunity for online sharing and must be shared as an application, meaning it cannot be viewed but needs to be fully downloaded first. Furthermore, visualization created in UE is heavily focused on graphics by creating a realistic representation of the visualized area together with phenomena like wind, animations, collisions, and others, which results in heavily hardware-demanding visualization. This then requires the user to own a powerful computer, as the visualization itself requires the user to open and use the UE visualization. Those facts significantly limit the possible use cases of the realistic UE visualization and make it more suitable for controlled environments where the visualization can be prepared for the user, such as at exhibitions. This is further underlined by the possibilities of using additional advanced technologies in tandem with UE visualization, such as VR, which most people don’t own. Since the UE visualization is already suited for use in controlled environments such as project exhibitions and is already more performance-demanding, VR capability and hardware can be added to the already more demanding UE visualization in such a controlled environment to enhance the viewer’s experience further.
Several use cases have already been discussed, focusing on visualizations of historic buildings and landscapes, virtual reality utilizing VR headsets, or even more exotic solutions like VR CAVE to simulations of people and their daily lives, as discussed under the Virtual Angkor project. With the focus on visualizations using realistic graphics, what has been omitted are the use cases of game engines that don’t rely on graphical quality yet use the 3D interactable environment, such as visualizing events as a game or visualizing calculations.
One specific use case of game engines is the creation of educational games. The purpose of such games isn’t to have complex visualizations or realistic graphics but to envoy a message or knowledge to the player by using simple graphics with an interactive environment. For example, in flood management educational games [33], a village area is visualized using simple graphics. This area then gets flooded, and the player aims to collect necessities and evacuate. This goal is gamified using a points system, where the better the player behaves and the more correct actions the player takes in the game, the more points he is rewarded. In the case of such educational games, a less realistic graphic approach is preferred since it allows those games to run on minimal hardware and mobiles. This allows those educational games to be spread and used by as many users as possible, which in this example allows the majority of users to learn how to behave during floods, which are frequent in the area, which helps to save lives.
5. Conclusions
The literature research on this topic found that the use of game engines outside of the gaming and filmmaking industry is on the rise thanks to the constant developments in computer performance and the renaissance of older technologies that can now be fully utilized thanks to this available performance. The fact that even regular desktop computers can now achieve performance capable of using game engines and other technologies, such as virtual reality, brings a rise to smaller independent projects utilizing game engines both for gaming and non-gaming projects, such as visualizations of (historic) large-scale landscapes of which multiple examples were found. This is further underlined by companies like ESRI that now develop software development kits such as CityEngine SDK or Arc Map SDK [28,34] and are implementing direct exporting options that allow them to interconnect their software with game engines Unity and Unreal Engine, further advocating the use of those game engines. This treatment is currently only aimed at Unity and Unreal Engine, the leading game engines on the market, where the rest is too focused on games [14,15].
However, most projects that visualize large landscapes found during the literature research provide little or no data and data processing procedures and focus primarily on the introductory information and final visualizations, creating a gap in the literature regarding the spatial data processing process in game engines. This is a significant problem, as other researchers cannot validate or suggest procedure improvements. Furthermore, new researchers starting with game engines have nowhere to adopt already proven spatial data processing procedures. A case study was created to help bridge this gap and find more information to answer each research question. Within the case study, a portion of the 1670 km2 Vltava River valley was visualized based on historical GIS data from the project DH23P03OVV055 Vltava II. While only a portion of this area was fully visualized, all the steps in the visualizing process, from importing, editing, and visualizing GIS data, were tested and documented with the complete datasets covering the whole area.
During the visualization process, several fundamental problems were identified. Firstly, how to import data into the UE is highly dependent on the data itself, the project, and access to licensed and paid content and plugins. While the functionality to import raster heightmaps is built-in in UE, importing rasters in a specific coordinate system, in this case with EPSG 5514, and keeping the coordinates in UE editor is not possible without using additional tools like plugins or Datasmith importer. While multiple solutions for importing GIS data into UE were tested in this case study, it is not feasible to test all the solutions because of the prohibitive cost of obtaining licenses for all the software and plugins. As a result, the importing solutions need to be considered for each project separately according to its needs. Furthermore, UE and its default scene content like sun, sky, clouds, and fog, which are contained in some of the default presets, do not correctly work if the visualized areas are within extreme XY coordinate values, such as was the case in this case study with coordinate values reaching -750 000 and 1 000 000 respectively. This can be solved by manually transforming the imported data near the coordinate system origin (X = 0, Y = 0). That can, however, be problematic in cases where additional data needs to be (re)imported, which, thanks to this case study’s testing nature, was a reoccurring problem. Alternatively, a plugin or an asset pack containing the functionality of the sun, sky, clouds, and fog needs to be found at the marketplace, bought and tested, or would need to be of own creation of the user, which requires deep understanding and knowledge of the UE inner workings.
One of the significant differences between GIS and UE visualization is how land use information is presented on top of terrain. In GIS, vector polygons containing land use information are projected onto the terrain, and the visualization is achieved by simply applying symbology to the projected polygons. However, UE doesn’t support importing vector data without plugins, and vector data imported using plugins have little to no visualization value without landscape splines. In UE, the textures (materials) are applied to the terrain using raster masks, which creates a whole set of challenges. If one would still like to use imported vector polygons, the only use would be guiding lines for the manual application of textures (material layers), which is not feasible for more extensive projects. The other solution is to convert vector data containing land use information into separate raster files and import them as mask layers. However, this has its problems in the loss of precision of the land use dataset and a considerable increase in the size of the data and memory requirement, especially if there are more layers we want to visualize.
Memory requirements for the visualization process are currently the biggest obstacle. While the minimum required RAM for any UE project is 32GB, it scales up with the project size without a limit. A system page file on fast SSD was set up to avoid crashing the UE during specific computational tasks. During memory-demanding peaks, over 200GB of RAM and page file memory combined were used. While there are ways to negate this problem for the end user by using world partition in UE (equivalent to tiling in GIS), this doesn’t negate memory problems in the UE editor used while creating the visualization. An older solution of splitting the whole 1670 km2 area into 68 sections was used to reduce memory demands in the editor. Those sections are then visualized in separate levels and manually connected, which significantly reduces the memory strains during the visualization process in UE. Furthermore, much of the input data was edited to reduce the number of actors (objects) and reduce textures' resolutions.
Another important takeaway is the entirely different use case of UE visualization from that of GIS visualizations. Firstly, the GIS visualizations from project Vltava are hosted online on ArcGIS Online, allowing the visualizations to be viewed by anyone with an internet connection and browser. On the other hand, UE visualization doesn’t have this opportunity and must be shared as an application, meaning it cannot be viewed but needs to be fully downloaded first. Secondly, a comparably more powerful computer is required to open and use for the UE visualization. Those facts significantly limit the use case of the UE visualization and make it more suitable for exhibitions and use in controlled environments. In contrast, GIS visualizations are better suited for viewing by individuals at home. This is further underlined by the possibilities of using additional advanced technologies in tandem with UE visualization, such as VR, for which most people don’t own the required hardware. Since the UE visualization is already suited for use in controlled environments such as project exhibitions and is already more performance-demanding, VR capability and hardware can be added to the already more demanding UE visualization in such a controlled environment to enhance the viewer’s experience further.
In summary, the initial literature research found a significant gap in the literature, which helped set the direction for the case study, during which a part of the 1670 km2 Vltava River valley area was visualized using Unreal Engine. Each part of the visualization process was documented with recommendations and solutions to problems found and solved in the case study to bridge this literature gap and provide enough information to make further research on using game engines (namely Unreal Engine) more accessible. This is important since game engines are becoming more popular each day with their wide range of capabilities, intertwined technologies that can be implemented and used with game engines such as VR, and use cases applicable to a wide range of industries from geomatic and hydrology to faraway sectors like medicine. While landscape visualization may not be helpful in such faraway industries, parts of this study may still be useful as sources of information on the inner workings of UE.
The work on the visualization of the Vltava River valley will continue within Project Vltava, where all outputs, including the final visualization, will be freely available.
Funding
This project was funded by the Ministry of Culture of the Czech Republic, grant number DH23P03OVV055.
Data Availability Statement
Due to the use of licensed content, plugins, and underlying data used in the creation of the visualization within this case study, the ArcGIS Pro, City Engine, and Unreal Engine projects will be available only after request to the author of this article until the whole visualization in UE is finished. After that, the final visualization, together with as much as possible from the individual projects (aka the licensed content will be cleared out), will be made available either separately through GitHub, to which link will be included as part of the project Vltava II at the website or it will be available directly for viewing and download at the project website: https://vltava.fsv.cvut.cz/.
Acknowledgments
I want to thank my tutor, Prof. Ing. Jiří Cajthaml, Ph.D., for his professional guidance and valuable advice and comments while preparing my dissertation and throughout my doctoral studies, which allowed for the creation of this article.
Conflicts of Interest
The author declares no conflicts of interest. The funders had no role in the design of the study, the collection, analysis, or interpretation of data, the writing of the manuscript, or the decision to publish the results.
References
- Janovský, M.; Tobiáš, P.; Cehák, V. 3D Visualisation of the Historic Pre-Dam Vltava River Valley—Procedural and CAD Modelling, Online Publishing and Virtual Reality. ISPRS International Journal of Geo-Information 2022, 11, 376. [Google Scholar] [CrossRef]
- Cajthaml, J.; Kratochvilova, D.; Janata, T. 3D Model of Historical Vltava River Valley: Combination of Sources. Proceedings of the ICA 2019, 2, 1–5. [Google Scholar] [CrossRef]
- Krejčí, J.; Cajthaml, J. Historical Vltava River Valley–Various Historical Sources within Web Mapping Environment. ISPRS International Journal of Geo-Information 2022, 11, 35. [Google Scholar] [CrossRef]
- Pacina, J.; Cajthaml, J.; Kratochvílová, D.; et al. Pre-dam valley reconstruction based on archival spatial data sources: Methods, accuracy, and 3D printing possibilities. Transactions in GIS 2022, 26, 385–420. [Google Scholar] [CrossRef]
- Kratochvílová, D.; Cajthaml, J. Using the Automatic Vectorisation Method in Generating the Vector Altimetry of the Historical Vltava River Valley. Acta Polytechnica 2020, 60, 303–312. [Google Scholar] [CrossRef]
- Janovsky, M.; Janata, T.; Cajthaml, J. Visualization of the Vtava River valley: illustration of work procedures on data from the Kamyk reservoir surroundings. In Proceedings of the 20th International Multidisciplinary Scientific GeoConference Proceedings SGEM 2020; 2020; pp. 469–476. [Google Scholar] [CrossRef]
- Laksono, D.; Aditya, T. Utilizing A Game Engine for Interactive 3D Topographic Data Visualization. ISPRS International Journal of Geo-Information 2019, 8, 361. [Google Scholar] [CrossRef]
- Flamm, K. Measuring Moore’s Law: Evidence from Price, Cost, and Quality Indexes; National Bureau of Economic Research: Cambridge, MA, USA, 2018. [Google Scholar] [CrossRef]
- Leiserson, C.E.; Thompson, N.C.; Emer, J.S.; et al. There’s plenty of room at the Top: What will drive computer performance after Moore’s law? Science 2020, 368, eaam9744. [Google Scholar] [CrossRef] [PubMed]
- Liu, E.; Llamas, I.; Cañada, J.; et al. Cinematic Rendering in UE4 with Real-Time Ray Tracing and Denoising. In Ray Tracing Gems: High-Quality and Real-Time Rendering with DXR and Other APIs; Haines, E., Akenine-Möller, T., Eds.; Apress: Berkeley, CA, USA, 2019; pp. 289–319. ISBN 978-1-4842-4427-2. [Google Scholar] [CrossRef]
- Jungherr, A.; Schlarb, D.B. The Extended Reach of Game Engine Companies: How Companies Like Epic Games and Unity Technologies Provide Platforms for Extended Reality Applications and the Metaverse. Social Media Society 2022, 8, 20563051221107641. [Google Scholar] [CrossRef]
- Alene, G.H.; Vicari, H.; Irshad, S.; et al. Realistic visualization of debris flow type landslides through virtual reality. Landslides 2023, 20, 13–23. [Google Scholar] [CrossRef]
- Salama, R.; Elsayed, M. Basic elements and characteristics of game engine. Global Journal of Computer Sciences: Theory and Research 2018, 8, 126–131. [Google Scholar] [CrossRef]
- Vohera, C.; Chheda, H.; Chouhan, D.; et al. Game Engine Architecture and Comparative Study of Different Game Engines. In Proceedings of the 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT): 2021 12th International Conference on Computing Communication and Networking Technologies (ICCCNT); 2021; pp. 1–6. [Google Scholar] [CrossRef]
- Christopoulou, E.; Xinogalos, S. Overview and Comparative Analysis of Game Engines for Desktop and Mobile Devices. International Journal of Serious Games 2017, 4. [Google Scholar] [CrossRef]
- Hamad, A.; Jia, B. How Virtual Reality Technology Has Changed Our Lives: An Overview of the Current and Potential Applications and Limitations. International Journal of Environmental Research and Public Health 2022, 19, 11278. [Google Scholar] [CrossRef] [PubMed]
- Slater, M.; Sanchez-Vives, M.V. Enhancing Our Lives with Immersive Virtual Reality. Frontiers in Robotics and AI 2016, 3. [Google Scholar] [CrossRef]
- Pavelka, K.; Landa, M. Using Virtual and Augmented Reality with GIS Data. ISPRS International Journal of Geo-Information 2024, 13, 241. [Google Scholar] [CrossRef]
- Sobota, B.; PietrikovÁ, E.; Sobota, B.; et al. The Role of Game Engines in Game Development and Teaching. In Computer Science for Game Development and Game Development for Computer Science; IntechOpen: London, UK, 2023; ISBN 978-1-83769-732-8. [Google Scholar] [CrossRef]
- Lu, S.; Fang, C.; Xiao, X. Virtual Scene Construction of Wetlands: A Case Study of Poyang Lake, China. ISPRS International Journal of Geo-Information 2023, 12, 49. [Google Scholar] [CrossRef]
- Stein, E.N. Virtual Angkor: Reimagining Cambodia’s Temple-City. Ars Orientalis 2021, 51. [Google Scholar] [CrossRef]
- Frischer, B.; Abernathy, D.; Guidi, G.; et al. Rome Reborn. In ACM SIGGRAPH 2008 new tech demos; Association for Computing Machinery: New York, NY, USA, 2008; ISBN 978-1-4503-7847-5. [Google Scholar] [CrossRef]
- Dylla, K.; Haegler, S. Rome Reborn 2.0: A Case Study of Virtual City Reconstruction Using Procedural Modeling Techniques. 2010. [CrossRef]
- Sulkow, C. Rome Reborn. 2019. [CrossRef]
- Chandler, T.; Clulow, A. Modeling Virtual Angkor: An Evolutionary Approach to a Single Urban Space. IEEE Computer Graphics and Applications 2020, 40, 9–16. [Google Scholar] [CrossRef]
- Kratochvílová, J.C.; Tobiáš, D.P. 3D model of the historic Vltava River valley in the area of Slapy Reservoir. In Advances and Trends in Engineering Sciences and Technologies III; CRC Press: Boca Raton, FL, USA, 2019; ISBN 978-0-429-02159-6. [Google Scholar]
- Vitruvio - Plugin for Unreal Engine. Available online: https://esri.github.io/cityengine/vitruvio (accessed on 7 April 2023).
- CityEngine C++ SDK. Available online: https://esri.github.io/cityengine/cityenginesdk (accessed on 6 February 2024).
- Unreal Engine 5.2 Documentation. Available online: https://docs.unrealengine.com/5.2/en-US/ (accessed on 7 February 2024).
- NOVEDGE. Rome Reborn: A Virtual Journey Through 3D Cities with Dr. Bernard Frischer. NOVEDGE. Available online: https://novedge.com/blogs/news/rome-reborn-a-virtual-journey-through-3d-cities-with-dr-bernard-frischer (accessed on 21 December 2023).
- Rome Reborn. 2023. Available online: https://en.wikipedia.org/w/index.php?title=Rome_Reborn (accessed on 21 December 2023).
- Virtual Angkor. Virtual Angkor. Available online: https://www.virtualangkor.com (accessed on 24 December 2023).
- Rismayani, A.; Nurhidayani, P.A.; et al. Fundamental Design of Flood Management Educational Games Using Virtual Reality Technology. International Journal of Online and Biomedical Engineering (iJOE) 2022, 18, 19–32. [Google Scholar] [CrossRef]
- ArcGIS Maps SDK for Unreal Engine | ArcGIS Developers. ArcGIS Maps SDK for Unreal Engine. Available online: https://developers.arcgis.com/unreal-engine/ (accessed on 7 April 2023).
Figure 1.
Map sheets and examples: (a) Imperial Obligatory Imprints 1:2880.; (b) SMO-5.
Figure 2.
DTM of Vltava River valley: (a) as the original raster file; (b) divided into 68 pieces.
Figure 3.
Vectorization of Imperial Obligatory Imprints 1:2880: (a) original full size; (b) reduction to match terrain based on SMO-5 maps.
Figure 3.
Vectorization of Imperial Obligatory Imprints 1:2880: (a) original full size; (b) reduction to match terrain based on SMO-5 maps.
Figure 4.
Landscape Pro 3 foliage models.
Figure 5.
Layer Info for material layers in UE.
Figure 6.
Layer Info for material layers in UE.
Figure 7.
Procedural generation of grass by landscape material in UE.
Figure 8.
Procedural Foliage Spawner: (a) inside the bounding box; (b) details tab.
Figure 9.
Procedurally generated needle tree forest in UE: (a) close view; (b) broader view.
Figure 10.
Example of procedurally modeled buildings – CityEngine.
Figure 11.
UE water plugin.
Figure 12.
UE water plugin actors: (a) Water Body Lake; (b) Water Body River.
Figure 13.
Packaging Project in UE.
Figure 14.
Buildings along the River Vltava, UE visualization.
Figure 15.
A faraway view of the broader landscape: (a) GIS; (b) UE.
Figure 16.
Comparison of trees LOD optimization: (a) GIS; (b) UE.
Figure 17.
Vegetation comparison up-close: (a) GIS; (b) UE.
Figure 18.
Procedurally modeled buildings up-close comparison: (a) GIS; (b) UE.
Table 1.
Used hardware.
| Computer/parameters | PC1 - personal | PC2 - Poznań PL |
|---|---|---|
| CPU | AMD Ryzen 9 7950X3D | Intel Core i9 10940X |
| GPU | NVIDIA GeForce RTX 4090 | NVIDIA GeForce RTX 3070 |
| RAM | 64GB DDR5 6600MHz CL32 | 64GB DDR4 3200MHz CL22 |
Table 2.
Used software.
| Name | Version | Software type |
|---|---|---|
| Unreal Engine | 5.2.1 | Game engine |
| ArcGIS CityEngine | 2021.1 | 3D modeling software |
| ArcGIS Pro | 2.8.0 ; 3.2.1 | GIS software |
| GIMP | 2.10.36 | Photoshop |
Table 3.
Used Plugins.
| Plugin name | Version |
|---|---|
| Vitruvio CityEngine Plugin | 1.4 |
| Landscape Pro 3 - Automatic Natural Environment Creation Tool | - |
| Water | Experimental v0.1 |
Table 4.
Comparison of DMTs in the context of memory size.
| Name | Area of DTM (km2) |
Pixel size (m) |
LZW Compression size (GB) |
Uncompressed size (GB) |
|---|---|---|---|---|
| Original DTM | 12 705 | 5 | 0,263 | 1,89 |
| Upscaled DTM | 12 705 | 1 | 5,76 | 47,33 |
|
Upscaled 68 DTMs together |
1 670 +5m overlaps |
1 | 2,32 | 6,425 |
|
Upscaled 68 DTMs individually |
Smallest = 5,07 | 1 | 0,00711 | 0,0193 |
| Largest = 75,3 | 0,103 | 0,287 |
Table 5.
Used Editor Preferences in UE.
| Location | Name | Value |
|---|---|---|
| General - Experimental | Procedural Foliage | Enable |
Table 6.
Used Project Settings in UE.
| Location | Name | Value |
|---|---|---|
| Engine - Rendering | Anti-Aliasing Method | TAA |
| Project - Packaging | Build Configuration | Shipping |
| Project – Maps & Modes | Default maps | Select the starting level name |
| Project – Supported Platforms | Windows | |
| Project - Description | About, Displayed | Fill in information |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.
MDPI Initiatives
Important Links
© 2024 MDPI (Basel, Switzerland) unless otherwise stated