The 3D Ultraviolet Fluorescence Imaging in Cultural Heritage: A Review of Applications to Multi‐Material Artworks

1. Introduction

Ultraviolet fluorescence (UVF) imaging, such as other imaging techniques, is a diagnostic tool widely used in conservation and restoration of cultural heritage for obtaining information about the surface state of conservation of artworks in a completely non-invasive and nondestructive modality [1,2,3,4,5,6,7,8]. For this reason, UVF imaging is an almost imperative step to be taken before starting the restoration, as it allows us to reveal the presence of inpainting, grouting, fluorescent materials, previously restored area and in general details not visible at naked eyes [9,10,11,12,13,14,15,16,17,18,19,20,21,22]. In Figure 1 four examples of UVF images are shown to highlight their great utility for the study, the interpretation and the restoration phases of an artwork.

UVF imaging is a simple but at the same time powerful tool for restorers, art historians, archaeologists, archivists, and conservators to “look” at the artwork surfaces under a different way, that is the fluorescence response in the visible range induced by ultraviolet radiation [23,24,25,26]. The great relevance of UVF technique in restoration is demonstrated in pioneering works by Rorimer [27], Eibner [28,29,30], Lyon [31] and other authors such as De la Rie whose papers made the story of UVF photography applied to artworks’ examination [9,10,11].

Figure 1. Some examples of UVF images from our laboratory archive demonstrating the utility of the technique for the study of the artworks, especially during the restoration activities. (A) a 18th century tetramorphic wooden lectern from the Church of the Certosa of San Martino showing the status of the cleaning operation (upper part cleaned, lower part uncleaned), unpublished photograph; (B) the wall painting in Palazzo Nuzzi at Orte (Central Italy) showing yellow fluorescence in the area interested by inpainting with zinc white [21]; (C) the 16th century panel panting representing a Crucifixion and attributed to Michelangelo’s workshop, exposed in Museum of Colle del Duomo in Viterbo, showing the presence of cracks, inpainting, grouting [22]; (D) a 19th century large canvas representing the wall paintings of Cappella Mazzatosta by Lorenzo da Viterbo showing an interesting use of two different kinds of white: yellow fluorescent zinc white and lead white [12].

Figure 1. Some examples of UVF images from our laboratory archive demonstrating the utility of the technique for the study of the artworks, especially during the restoration activities. (A) a 18th century tetramorphic wooden lectern from the Church of the Certosa of San Martino showing the status of the cleaning operation (upper part cleaned, lower part uncleaned), unpublished photograph; (B) the wall painting in Palazzo Nuzzi at Orte (Central Italy) showing yellow fluorescence in the area interested by inpainting with zinc white [21]; (C) the 16th century panel panting representing a Crucifixion and attributed to Michelangelo’s workshop, exposed in Museum of Colle del Duomo in Viterbo, showing the presence of cracks, inpainting, grouting [22]; (D) a 19th century large canvas representing the wall paintings of Cappella Mazzatosta by Lorenzo da Viterbo showing an interesting use of two different kinds of white: yellow fluorescent zinc white and lead white [12].

UVF photography has been widely applied for paintings, sculptures, manuscripts, textiles, contemporary artworks, etc. and several papers can be found in the literature where this technique helped in the investigation of the surface state of artworks, but a very limited number of papers can be found for 3D photogrammetric application of UVF, as we will see later in this article.

The research in scientific literature was made by using various keywords: 3D ultraviolet fluorescence imaging for cultural heritage, 3D multi-band imaging, 3D digital twins under ultraviolet fluorescence, ultraviolet fluorescence photogrammetry, ultraviolet 3D models, 3D luminescence imaging. If, on the one hand, several papers can be found concerning UVF imaging in 2D, from the other one a very limited number of articles reporting 3D imaging under UVF were obtained [12,32,33,34,35,36,37,38,39,40,41,42].

Papers concerning topics related to the 3D UVF imaging production such as: ultraviolet image acquisitions, photogrammetry, digital model creation, and applications of UVF imaging will be reported in the references’ section because they are relevant for the topic treated in the present review.

The first published papers proposing an innovative approach to creating 3D digital models under UVF were published in 2017 by Lanteri and Agresti [32], by Lanteri et al. in 2018 [33] and in 2019 [34]. In these papers, the authors reported for the first time a method to output UVF 3D models and to share them with the end-users starting from photogrammetry and Structure from Motion workflow with a variation in the acquisition procedure, as we will see later in the paper, in order to make simpler and more rapid the process. Subsequently other authors presented the same approach for obtaining multispectral and UVF 3D models without citing the original idea previously published and without any proposal on how to share the 3D models [35,36].

It is worth noting that a 3D multi-band approach was also employed for paintings (not for 3D artworks) to produce surface models at different wavelengths, but with a different methodology [43,44].

In the present review the attention will be focused on the papers proposing the use of ultraviolet fluorescence imaging for obtaining 3D digital models of artworks in UVF as a powerful instrument for cultural heritage studies.

Detailed research has been performed on the instrumental set-up used for obtaining ultraviolet fluoresce images by considering the UV sources, the filters and the cameras A focus will be addressed to underline the modalities of making the digital twins usable for restorers and experts in the field to easily observe the fluorescence of the surfaces.

2. Methodologies for obtaining the UVF 3D digital models

The acquisition of photographic images of the fluorescence induced by ultraviolet radiation (UVF), follows a procedure now standardized in the field of diagnostics for cultural heritage: the shots are taken in a darkroom, using digital cameras equipped with lens characterized by intermediate focal length (50 mm), in order to reduce as much as possible the distortions and aberrations that affects wide-angle lens, UV sources, filters and a tripod [12,45].

UVF photography requires the use of a tripod to fix and stabilize the camera during acquisition. This is particularly relevant to ensure the camera stability during quite long exposure times, usually around 15 seconds or higher.

Starting with the UV sources, which are the first essential requirement to obtain UVF images, different kinds of lamps may be used, such as Wood lamp, mercury vapor lamps, xenon arc-lamps, tungsten-halogen lamps, and more recently light-emitting diodes (LEDs) [5,15,38,46,47,48,49,50,51,52,53]. These latter sources offer considerable advantages in terms of handling and the possibility of being powered by rechargeable batteries, to be able to operate in situ, even when there is no power supply available [12,21,54]. Furthermore, LED lamps offer a controlled emission peak at 365 nm, ideal for UVF photography.

Another relevant part of the instrumental set-up is the UV cut filters, mounted in front of the photographic lens [12,55,56,57,58]. Sandwiches composed of UV-IR cut filters are often used to eliminate residual infrared and UV components that could affect the visible fluorescence response [12,46]. A wide panorama of filters used in UVF photography are reported by Crowther in a recent interesting paper [55].

Lastly, a digital camera (Reflex or Mirrorless), must be used for the capture of the images. Also in this case, different kinds of apparatus may be used [12,50,59,60,61,62], including low-cost compact digital camera [63].

The camera typologies and settings are generally detailed in the published papers [36,46,64,65,66,67].

As said in the introduction, UVF photography is widely used as 2D technical documentation and diagnostic tool for painting and 3D objects by using a typical set-up, as reported by Cosentino [3]. But the present review focuses on 3D applications of UVF imaging, particularly useful for sculptures and in general tri-dimensional artworks. The procedure for creating a 3D model in UVF has been proposed for the first time by the group of Lanteri in 2017 [32] and further tested in 2018 and 2019 [33,34].

The technique used does not differ from the Structure from Motion (SfM) workflow, which usually uses images taken in the visible range [68,69,70,71,72]. The basic principles of photogrammetry are still valid, in fact a set of images must be acquired having few important characteristics: 60%-80% overlapping between frames, limited optical and perspective distortions [73,74,75,76,77,78,79,80,81].

The workflow of the SfM process, which by its nature involves reconstructing the 3D digital structure starting from images taken by a moving sensor, involves the creation of a photographic studio set in a darkroom and very long acquisition times for UVF images also due to the continuous movements of the camera and UV lamps (at least two) around the object to be acquired, in order to obtain complete coverage of the surface.

To reduce the acquisition times of the photographic set, it has been proposed by Lanteri [32,33] and subsequently by other authors [35,36,40] to position the subject on a rotating support (360° around the vertical axis), and to keep the camera and lamps in fixed position. The acquisition of the frames and the management of the lighting is much easier and faster, drastically reducing the time to obtain the photographic set of images (Figure 2).

Figure 2. Creation of the 3D UVF model starting from the images acquired by rotating the object, in this case a reliquary bust representing St. Rosalia, and maintaining fixed the camera and the UV sources [42].

Figure 2. Creation of the 3D UVF model starting from the images acquired by rotating the object, in this case a reliquary bust representing St. Rosalia, and maintaining fixed the camera and the UV sources [42].

This procedure, in clear contrast with the principle of SfM, is not “perceived” during the image processing phase with the most used commercial software [12].

In a nutshell, if the image matching algorithm should recognize, in the set of images, the object in the foreground with different positions and the background with the same characteristics for the entire set, it would abort the calculation process (because it is based on a fixed object). Fortunately, in the case of UVF photographs this is avoided. This is made possible by the fact that the image matching algorithms, when analyzing UVF images, process only the pixels that have captured the fluorescence of the irradiated surface of the object, therefore recognizable on multiple images, but do not distinguish the background pixels. In fact, the pixels that capture the background, generally very dark, are highly uniform with confidence values so low that they are discarded by the image matching algorithm.

Regardless of the phenomenon described most of the photogrammetry software ae equipped with specific tools that allow to create cutout masks on the individual images, once loaded on the software. The masks allow us to exclude the background pixels from the calculation, concentrating the “image matching” process only on the pixels that represent the fluorescence produced from the object’s surface.

The digital product that is obtained at the end of the workflow is a 3D model characterized by photorealistic rendering and real dimensions, these last obtained by inserting a scale bar in the scene, or by knowing the real distance between two well-defined points of the model [12]. The model, although created with images of UV-induced fluorescence, returns a metrologically correct digital twin, i.e., a faithful 1:1 reproduction of the acquired artwork.

As highlighted in the literature, the photogrammetry software used for the creation of 3D models starting from UVF images are both open-source and commercial.

As emerged from our bibliographic research, the commercial software generally used for creating 3D UVF models is Agisoft Metashape [82]. Open-source available alternative is AliceVision Meshroom [83].

Agisoft Metashape is a stand-alone software that performs photogrammetric processing of digital images and generates 3D spatial data. The software allows processing images from RGB or multispectral cameras, transforming them into spatial information in the form of dense point clouds, textured meshes and orthomosaics.

AliceVision Meshroom is a free and open-source 3D reconstruction software based on the Alice Vision framework. The software has a good quality node-based interface, which allows managing almost the entire workflow: photo alignment, sparse point cloud creation, high-quality mesh and photorealistic texture generation. A limitation of this program is the inability to generate and display a dense point cloud within the program’s workspace. The dense point cloud can be exported, which requires additional work.

The last relevant step of the procedure concerns the use of the 3D UVF models. The question is: how can we make available these models to the end-users, such as conservators, restorers, art-historians, archaeologists, students, scholars, etc.?

In fact, in all examined papers from the literature there are only 2D images extracted from the 3D models. So, our idea was to use Sketchfab on-line platform, specifically developed for uploading and sharing 3D models [84].

Sketchfab is a commercial platform that allows uploading 3D models that could be made available through the specific link so that the end-user can use them from different kinds of devices: laptop, tablet, mobile. For example, a restorer can make a check during work, by simply accessing the model and exploring it (rotating, zooming).

3. Discussion

The present review has been focused on UVF imaging technique due to its great relevance in conservation and restoration field. More in detail, the attention was focused on the production of 3D UVF models that could be interactively explored by end-users for their needs.

The research in the literature gave a very wide numbers of papers on UVF imaging, demonstrating the high relevance and diffusion of this technique in cultural heritage applications. But UVF imaging is generally used as 2D method applied for obtaining information from photographs both for paintings and 3D objects.

On the other hand, very few papers specifically devoted to 3D UVF imaging are published. One of the main obstacles to the production of photorealistic 3D UVF models is the necessity to use a set of overlapped images acquired according to a photogrammetric approach, i.e., the object is fixed, and the acquisitions are made by moving the camera and the irradiation sources around the object. This is difficult in case of UV irradiation because we have to move the sources and the camera in a completely dark environment, necessary for obtaining the UVF acquisitions.

So, some years ago, we proposed an innovative methodology based on a different perspective in respect to traditional acquisition. Why do not maintain the camera and sources fixed and moving the object? Working in a completely dark environment could guarantee that the background is completely black, so the camera acquires only the UVF response of materials, as detailed in paragraph 2 about methodology.

The first application of this idea was successfully performed in 2017 on a reliquary bust made of papier-mâché representing the Pope Pio V (1504-1572 AD) exposed in the Museum of Colle del Duomo in Viterbo (Italy) [32]. From that time, other models were created in collaborations with restorers and students from our course in Conservation and Restoration of Cultural Heritage in Viterbo. The restorers, in fact, found these models much more useful for their daily work in respect to 2D photographs usually employed for diagnostic documentation. In view of this result of practical interest, the same procedure was also tested to obtain a 3D diagnostic model starting from NIR photographic shots, which highlighted the potential of the model to analyze the underdrawings of the wooden polychromatic bust [34].

Other authors [35,36,37,38,40], after our first publication, proposed similar approaches, with some little differences in the experimental conditions and post-processing.

After having found the modality to acquire the UVF set of images and produced the 3D model representing the fluorescence of the object surface, how can we make this model available to the restores or in general to other users? In fact, in all papers reporting 3D models in UVF or in multi-band modality, we have observed 2D images (screenshot) extrapolated during the software workflow for obtaining the digital model, or single UVF frames, without specifying how the 3D models would be displayed and used [40].

To solve this problem, we chose to use a commercially available platform, named Sketchfab [84] where all models produced in our laboratory have been published.

Each model is visible at a specific link of Sketchfab, as reported in Table 1.

By accessing though the link it is possible to observe the model, to rotate it, to zoom on specific zones to check fluorescence details. The models can also be downloaded if authorized by the authors.

Moreover, the original models obtained through the photogrammetric software, are available in our laboratories in the traditional formats (.obj., .ply, .3ds, .pdf and others) to make further processing operations by using 3D modelling and/or mapping software. Mapping is highly relevant in conservation, and it is normally used by restorers to have in graphical form information about the artworks and the operations performed on it such as mapping of the state of conservation, of the cracks, of the surface detachments, cleaning, consolidation, inpainting, etc.

4. Conclusions

To conclude the paper, we can list some key points summarizing the main findings of the research about 3D ultraviolet fluorescence imaging in cultural heritage:

• The literature research revealed the presence of several papers underlying the great utility of ultraviolet fluorescence photography as diagnostic tools to support the restoration, starting from the first published articles about a century ago.
• If from one hand the use of UVF imaging in 2D is widespread, the same one in 3D is much less applied, due to the difficulty of the procedure, especially in the phase of acquisition of the images necessary for creating the 3D model.
• Starting from 2017, some authors proposed an innovative approach aimed at simplifying the acquisition. The approach is based on photogrammetry and SfM but something has been modified. In fact, the camera and the UV sources are fixed and the object to be acquired is rotated on a turntable. The acquisition is possible thanks to the completely dark background required for UVF capture, as described in detail in paragraph 2.
• One relevant aspect is the visualization and use of the 3D UVF models. In fact, the output is generally reported by showing 2D images that are extrapolated from the 3D model workflow or even single frames used for creating the digital model. The use of online platforms, such as Sketchfab, may give valid help in this regard. Sketchfab allows publishing 3D models that can be interactively investigated from simple devices, i.e., mobile, tablet, laptop, etc.
• Lastly the original models can be further used for other applications, such as mapping of the state of conservation in case of restoration activities, this being a usual procedure in the phases of documentation of artworks that precede the practical operations on them.

Table 1. Sketchfab links to the 3D UVF digital models and the publication place.

Link to the 3D UVF model	Publication
https://skfb.ly/6EMy7	Lanteri et al. [33]
https://sketchfab.com/3d-models/san-icodono-ridotto-623eeb3a3e6c40888f0c9663c5fc4adf	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-rodonio-tempo-1-187ef9beb2de45388bde63390591a778	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-filomelo-3ef74f8fe0af4d2291b6898d6f72d391	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-leonardo-7bcff9ed23ae4221a50ba52b06285679	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/santa-rosalia-8b4cf6c05837433c8c383b6266bf46fb	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/santo-stefano-ac0f26ccea1d466a822a499df5df1d3b	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/pio-v-3d-uvf-model-063c014bb7734581b3ebfca95d1ab85e	Lanteri and Pelosi [12]
https://sketchfab.com/3d-models/copricapo-uvf-45e43bcc474b470e8c8b876db917ba3e	Colantonio et al. [39]
https://sketchfab.com/3d-models/tomba-degli-scudi-tarquinia-7a183dcbe1e84bb199babcc0b30c3905	Rinaldi et al. [85]
https://sketchfab.com/3d-models/modello-3d-bambinello-corona-8d6a084e53144fb98174b8a0671ca4d5	Ceci et al. [41]
https://sketchfab.com/3d-models/madonna-del-carmine-3d-model-672bf63e67f44f40b102dd789658bb0a	Published on Sketchfab
https://sketchfab.com/3d-models/sant-andrea-3d-uvf-5f10ae5fb00242e293aaa297473dad68	Published on Sketchfab

Table 1. Sketchfab links to the 3D UVF digital models and the publication place.

Link to the 3D UVF model	Publication
https://skfb.ly/6EMy7	Lanteri et al. [33]
https://sketchfab.com/3d-models/san-icodono-ridotto-623eeb3a3e6c40888f0c9663c5fc4adf	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-rodonio-tempo-1-187ef9beb2de45388bde63390591a778	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-filomelo-3ef74f8fe0af4d2291b6898d6f72d391	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/san-leonardo-7bcff9ed23ae4221a50ba52b06285679	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/santa-rosalia-8b4cf6c05837433c8c383b6266bf46fb	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/santo-stefano-ac0f26ccea1d466a822a499df5df1d3b	Lanteri and Pelosi [42]
https://sketchfab.com/3d-models/pio-v-3d-uvf-model-063c014bb7734581b3ebfca95d1ab85e	Lanteri and Pelosi [12]
https://sketchfab.com/3d-models/copricapo-uvf-45e43bcc474b470e8c8b876db917ba3e	Colantonio et al. [39]
https://sketchfab.com/3d-models/tomba-degli-scudi-tarquinia-7a183dcbe1e84bb199babcc0b30c3905	Rinaldi et al. [85]
https://sketchfab.com/3d-models/modello-3d-bambinello-corona-8d6a084e53144fb98174b8a0671ca4d5	Ceci et al. [41]
https://sketchfab.com/3d-models/madonna-del-carmine-3d-model-672bf63e67f44f40b102dd789658bb0a	Published on Sketchfab
https://sketchfab.com/3d-models/sant-andrea-3d-uvf-5f10ae5fb00242e293aaa297473dad68	Published on Sketchfab