1. Introduction

Figure 1. Trends in forgery detection during the last years.

Figure 1. Trends in forgery detection during the last years.

Figure 2. Image forgery detection overview.

Figure 2. Image forgery detection overview.

2. Inpainting methods

• each pixel is defined as follows: n represents the coordinate system-usually 2
• m represents the color space representation of that pixel (usually 3 for an RGB space)

2.1. Diffusion based methods

The term diffusion (from a chemistry point of view) is the action in which items inside a region of higher concentration tend to move to a lower concentration area. From a mathematical point of view - let’s define it as: let Ω ⊂R^2 denotes the entire image domain 𝑓. The basic idea then is to propagate information from the border of the missing region into it, in such a way that the border of the missing region is no longer visible to the human eye. The border of missing region is going to be called 𝜕𝐷; Figure below ilustrates the inpainting steps.

Figure 3. Process of inpainting-based on PDE method.

Figure 3. Process of inpainting-based on PDE method.

Several authors [10], [9] have suggested a more detailed approach of splitting the inpainting diffusion class. They have suggested to further divide into the following sub-categories like: isotropic, anisotropic, total variation, PDE based. For the simplicity of this paper, we intented to organize all these methods under one big umbrella, because the starting points are the main ideas observed by [11], in which the inpainting process is inspired from the “real” inpainting of canvas and consists of the following steps:

• Global image properties enforce how to fill in the missing area
• The layout of the area $\delta D$ is continued into $D$ (all edges are preserved)
• The area D is split into regions and each region is filled with the color matching $\delta D$ (the color information is preserved from the bounding area $\delta D$ into the rest of the D area)
• Texture is added

The first step in almost all the inpainting algorithms is to apply some sort of regularization. It can be either isotropic, with some rather poor results, anisotropic, or any other type of regularization. This is done in order to ensure that image noise is removed, and thus it shall not interfere in the computation of the structural data needed in the next step.

In order to apply diffusion, the structural and statistical data of the low level image must be indentified. Based on this data, if on an edge on the δD area, we must conserve the edge identified and if δD area belongs to a consistent area, we can then easily replicate the same pixel information from the border. In order to retrieve image geometry, one can use isophotes - curve on surface connecting points of same values. For this one needs to compute first the gradient on each point on the margin area and then to compute the direction as a normal one to the discretized gradient vector.

Having performed these steps, the initial algorithm from [11] is just a succession of anisotropic filtering, followed by inpainting and then this repeated several times. From a mathematical point of view, based on [11], the intention is to achieve the following:

$$U\left(x\right)= unknown area- an array of pixels where x:R^n\to R^m$$

$$U_k\left(x\right)-is the k variant obtained.$$

$$U_k\left(x\right)= U_{k-1}\left(x\right)+{\alpha *\delta L}^k\left(x\right)*N^k\left(x\right)$$

$$L^k\left(x\right)=smoothness estimator \left[11\right]$$

$$N^k\left(x\right)=isophotes directions$$

In the original implementations the authors made some assumptions choosing α to be 0.1 and as for the smoothness estimator they have used the Laplacian. An interesting part is the choice of the N vector. In their original paper it is suggested that this vector has to be computed each time and is based on the current rotated gradient of the current block size to be inpainted. The problem with this vector is that it has to be computed at each iteration because with each iteration new information arises at the area to be reconstructed. Additionally, they also add an anisotropic diffusion step at each several steps, intending not to lose too much sharpness. From a forensic point of view, this is a very important step, because it does not tend to keep the same level of blur between the original area and the reconstructed area. Later on, the authors in [12] proposed an improved version of their initial algorithm. The idea was inspired from the mathematical equations of fluid dynamics, specifically the Navier-Stokes equations, which describe the motion of fluid. The proposal was to use the continuity and momentum equations of fluid dynamics to propagate information from known areas of the image or video towards the missing or corrupted areas. This was more or less an improved version of higher PDE version presented initially. As a follow up of his original work, Bertalmio proposed in [13] the use of 3rd order PDE, which are a better continuation of edges.

The algorithm starts by defining an initial velocity field that guides the propagation of information. This velocity field is then iteratively updated with the known image used as boundary conditions by the use of the Navier-Stokes equations. The result is obtained by advection of the original image along the final velocity field. The algorithm seems to perform slightly better than the initial paper in situations where the missing data is large, or the structure of the image is complex. The Navier-Stokes inpainting algorithm is a method for completing missing or corrupted parts of images or videos that uses the equations of fluid dynamics to propagate information from known areas to missing or corrupted areas. The final result is obtained by advecting the original image or video along the final velocity field.

At the same time, Chan & Shen developed similar algorithms [14], [15] in which they postulated the use of the local curvature of an image to guide the reconstruction of missing or obscured parts. Using Euler’s Elastica model, they can predict for what the missing parts of the image might look like. Both Euler’s Elastica and PDE-based inpainting are effective methods for image inpainting and have their own advantages and disadvantages. Euler’s Elastica is particularly well-suited for images that contain thin, flexible objects, while PDE-based inpainting is well-suited for images that are smooth and locally consistent. Depending on the specific characteristics of the image and the desired outcome, one method may be more appropriate than the other.

Based on the work described above, many methods continue in the same direction, trying to map real physical processes into the inpainting process (diffusion, fluid dynamics, osmosis). For e.g. in [16] the authors proposed curvature-preserving PDE. Their tensor PDE, is used for regularizing images, keeping into account the curvatures of specific integral curves. In this way, they estimate better the shape of the inpainting data, thus reducing the blurring effect on the resulted image. Another variant with very good results and implemented in the computer vision library (OpenCv) is presented in the paper [17]. The authors present a fast marching technique that estimates the missing pixels in one pass using weighted means of known calculated pixels. This is suboptimal algorithm compared to other inpainting algorithms, but it gains strength in its speediness compared to the example of Bertalmio, in which several iterations were needed and the result was affected by the number of iterations.

In the recent year the focus for diffusion based inpainting has moved towards more and more complex PDE forms. For e.g. in [18] using high order variational models is suggested, like low curvature image simplifiers or Cahn-Hilliard equation. Another recent paper that goes into the same direction is [19], which basically integrates the geometric features of image, namely the Gauss curvature. Still, even these methods introduce the blurring artifact also found in initial papers [11], [12]. In order to surpass these challenges in the current models, with second-order diffusion-based models that are prone to staircase effects and connectivity issues and fourth-order models that tend to exhibit speckle artifacts, a newer set of models has to be developed. The authors Sridevi & Srinivas Kumar proposed several robust image inpainting models that employ fractional-order nonlinear diffusion, steered by difference curvature in papers [20], [21], [22]. In their most recent paper [23], a fractional-order variational model is added to mitigate noise and blur effectively. In essence, a variation of DFT is used to consider pixel values from the whole image, not only by relying strictly on the neighboring pixels.

In an attempt to summarize the diffusion inpainting methods, it is found that they usually rely on 2nd or higher order partial derivatives, or via Total variation of energy, in order to be able to “guess” the missing area. One of the major drawbacks of these methods is that in some way, either locally or globally, some sort of anisotropic diffusion is introduced with a blurring effect, which in turn will affect the entire image. Due to this blurring effect nature, in theory image inpainting via PDE can be detected via some sort of inconsistency in the blurring effect of various regions.

2.2. Exemplar based methods

Approximately at the same a newer approach based on texture synthesis started to gain more momentum. The main inspiration came from [24] where A. A. Efros and T. K. Leung introduced a non-parametric method for texture synthesis, where the algorithm generates new texture images by sampling and matching pixels from a given input texture, based on their neighborhood pixels, thus effectively synthesizing textures that closely resemble the input sample. This approach was notable for its simplicity and ability to produce high-quality results, making it a foundational work in the area of texture synthesis. . The primary goal of these approaches was to enhance the reconstruction of the image section area which is missing. However, the challenges brought by texture synthesis are slightly different from those presented by classical image inpainting. The fundamental objective of texture synthesis is to generate a larger texture that closely resembles a given sample in terms of visual appearance. This challenge is also commonly referred to as sample-based texture synthesis. A considerable amount of research has been conducted in the field of texture synthesis, employing strategies such as local region growing or holistic optimization. Probably one of the main papers that gained a lot of attention was the work of [25]. In this paper, Criminisi presented a novel algorithm for the removal of large objects from digital images. The technique is known as Exemplar-Based Image Inpainting. The method is based on the idea of priority computation for the fill front, and best exemplar selection for texture synthesis. Given a target region Ω to be inpainted, the algorithm determines the fill order based on the priority function P(p), defined for each pixel p on the fill front ∂Ω. P(p) = C(p) * D(p) Where, C(p) is the confidence term, an indication of the amount of reliable information around pixel p. D(p) is the data term, a measure of the strength of isophotes hitting the front at p. The algorithm proceeds in a greedy manner, filling in the region of highest priority first with the best match from the source region Φ. This is identified using the Sum of Squared Differences (SSD) between patches. The novel aspect of the method is that it combines the structure propagation and texture synthesis into one framework, aiming to preserve the structure of the image, while simultaneously considering the texture. It’s been demonstrated to outperform traditional texture synthesis methods in many complex scenes and it has been influential in the field of image processing.

Figure 4. Criminisi algorithm [25].

Figure 4. Criminisi algorithm [25].

Based on these seminal works of Leung and Criminisi, the area started to be more and more researched and various places were investigated for further improvements: order of patch processing, faster way of computing the best patch, applying various operations on the best patch found in order not to disturb higher statistical data inside the image, multiscale and overcome global constraints, and even finding the best way of dealing with distances between patches.

Several methods are available for calculating the resemblance between patches of images. The most commonly employed metrics can be grouped into two categories: pixel-based metrics, with gauge similarity based on the difference or cross-correlation among pixel color values, and statistics-based metrics, which estimate the similarity between the probability distributions of pixel color values in patches. The first category includes metrics such as the sum of squared differences (SSD), the Lp norm, and normalized cross-correlation. The second category features statistics-based metrics like the Bhattacharyya distance, normalized mutual information (NMI), and Kullback-Leibler divergence. The SSD is the most frequently used metric when searching for similar patches. One key aspect in the designing of an inpainting forensic tool is that SSD tends to favor uniform regions, meaning it prefers copying pixels from those areas. To address this bias, a weighted Bhattacharya distance, denoted as d(SSD,BC), has been proposed. Nonetheless, when two patches have identical distributions, their Bhattacharya distance (dBC) is zero, implying that the weighted Bhattacharya distance is also zero, even if one patch is a geometrically modified version of the other.

Another area of improvement was one how fast/optimal the best patch is found. For this reason, some methods start by identifying the K-nearest neighbors (K-NNs) within the recognizable sections of the image. One simple approach to solving the nearest neighbor search problem is by calculating the distance from the target patch to all potential patches, regarding each patch as a point in multi-dimensional space. More efficient and approximate nearest neighbor search strategies are available, which structure the potential candidates using certain space-segmenting data structures like k-dimensional trees (kd-trees) or vantage point trees (vp-trees), guided by their spread in the search space. The nearest neighbor search can be performed effectively by using the properties of these trees to quickly discard vast sections of the search space, leaving only a minor portion of candidates for verification. The matching process based on kd-trees is one of the most used methods for identifying the nearest patch. However, the number of nodes examined expands exponentially with the dimension of the space. As a result, when the dimension is big, the search speed slows. A variety of nearest neighbor search algorithms are evaluated in a separate study [26] to determine their effectiveness in locating similar patches within images. A big improvement in this area was incorporated into Photoshop tool in recent years. The idea is based on the Patch Match algorithm [27]. Approximate nearest neighbor (ANN) search methods that are tree-based treat each query individually. PatchMatch, a randomized patch search algorithm introduced, takes advantage of the relationships between queries to facilitate collaborative searching. This method operates on the presumption that images maintain coherency. That is to say, once a similar pair of patches in two images is identified, their adjacent patches (those offset by a few pixels) are likely also similar. Consequently, the match result of a specific patch can be transferred to proximate queries, providing an advantageous initial guess that can then be updated with randomly selected candidates. PatchMatch is a speedy algorithm used for calculating dense, approximate nearest neighbor correspondences between patches in two image areas, with these correspondences collectively referred to as the nearest neighbor field (NNF). The algorithm initiates the search for the NNF as follows. The NNF is initially assigned either random values or prior information, with random guesses likely offering only a few beneficial guesses. The NNF is then continually fine-tuned by alternating between two operations known as propagation and random search, carried out on the patch level. The propagation step updates a patch offset using known offsets from its causal neighborhood, leveraging image coherency. During even iterations, offsets are propagated from the top and left patches, while during odd iterations, they are propagated from the right and bottom patches. The second operation carries out a local random search to establish initial patch matches, which are then disseminated by iterating a limited number of times. Although the algorithm is significantly faster than kd-trees, it offers less accuracy. It can get stuck in a local optimum due to the limited distance of propagation.

In the recent years the methods have become more and more complex and tried to exploit various artifacts inside the image and analyzing more in depth the structure near the area to be inpainted. Other approaches like [28], utilize a patch-based approach that searches for well-matched patches in the texture component using a Markov random field (MRF). Jin and Ye [29] proposed an alternative patch-based method that incorporates an annihilation property filter and a low rank structured matrix. Their approach aims to remove an object from an image by selecting the target object and restricting the search process to the surrounding background. Additionally, Kawai [30] presented an approach for object removal in images by employing a target object selection technique and confining the search area to the background. Authors have also explored patch-based methods for recovering corrupted blocks in images using two-stage low rank approximation [31] and gradient-based low rank approximation [32]. Another sub-area of focus for some authors was to represent the information by “translating” first the image into another format, so called sparse representation like DCT, DFT, DWT etc.. Here we just want to mention a few interesting research papers [33], [34]. They obtained good quality, while the area to be inpainting is rather uniform, but if the area is at edge of various different texture, the methods introduce some pretty ugly artifacts that make the methods unusable.

Some various authors like [7,8]–[10], [35], [36] have suggested that another classification of the inpainting procedure can be done. Either they suggest to add a sub-division based on sparse representation of images (like authors have suggest in [33], [37]–[39]) and then later try to apply existing algorithms on this representation, or a so called mixed / fussion mode – in which authors try to incorporate ideas from both world: from diffusion based and from texture synthesis (patch copying). In this latter category we can name a few interesting ideas, like the one Bertalmio explored in his [40] study, in which they combined PDE based solution together with patch synthesis and coherence map. The resulting energy function is a combination of the 3 metrics. As similar idea to Bertalmio’s above mentioned work, is the research of Aujol J, Ldjal S and Masnou S in their [41] article, in which they try to use exampler based methods to reconstruct local features like edges. Another inquest in the same idea was the work of Wallace Casaca, Maurílio Boaventura , Marcos Proença de Almeida and Luis Gustavo Nonato on [42] in which they combine anisotripic diffusion with transport equation to produce better results. Their suggested approach of using a cartoon-driven filling sequence has proven to be highly effective for image inpainting using as metrics both PSNR and speed.

If we were to summarize the exampler (or patch based) inpainting methods, they try to do 3 steps:

• find the best order to fill the missing area
• find the best patch that approximates that area
• try to apply if needed some processing on the copied patch in order to ensure that both local and global characteristics are maintained

Now if we look at the artifacts these methods introduce, we can easily categorize them into two groups: methods that simply “copy-paste” unconnected / unrelated regions (patches) into the missing area and methods that do some enhancement / adaptation of the patch values. The first category is straightforward to determine via a forensic algorithm: we rely solely on the fact that for a given region (usually several times greater than the patch used for inpainting) is comprised of patches that have “references” in other areas. The main problems are that how to determine the correct patch size to be able to determine “copied”, speed, and last but not least – how to correctly eliminate false positives (especially when the patch size is smaller and the window step is small as well). The second category of inpainting patch based methods – that do not simply copy the source patch to destination, is a little harder to detect. The above algorithm, where we search for similar (identical) patches can no longer be applied, we must introduce as well some heuristic, and probably evaluate some parameters on how much the patches resemble. Last, the problem of false positive with these approaches, increases exponentially because we are no longer finding identical patches, but rather nearly identical patches, and on images with smooth large texture, we might end up with a lot of false positives data.

3. Inpainting forgery detection mechanism

Figure 5. General steps to copy paste forgery detection as presented in P. Korus paper.

Figure 5. General steps to copy paste forgery detection as presented in P. Korus paper.

3.1. Image Inpainting detection

If we were to analyze the above-proposed framework, we can see that it works well on the copy-move scenario because it assumes that the copied area is at least bigger than the copy-paste algorithm patch size used in detection. But due to their nature, the inpainting algorithms do not copy a fixed size block, but rather, at each step they try to decide what is the best patch to fill in the missing information. The following will introduce several elements that form the above framework, and other variants that were added to it, unusable for these types of forgeries. To summarize the various inpainting techniques, the following was observed:

• Diffusion-based/PDE/Variational based – they are not suitable for filling large areas, and usually they do not copy patches, but rather propagate smaller information into the area to be reconstructed, thus they do not copy patches, but rather fill the area with information diffused from the known region. So applying a block-based detection will yield no results, as there are no copy regions, but rather diffused areas. Still, some traces can be analyzed, but they are more in the area of inconsistencies in blurring, CFA, and other camera/lens properties.
• Patch-based – at first they seem rather well suited to the above-mentioned framework. They work rather well if the region forged contains a lot of texture, but fail in case the object removed was rather large or surrounded by a continuous area (like sky, water, grass, etc). But at a closer look, the method may give unreliable results, due to the inpainting procedures: usually, patch-based methods reconsider the filling area at each step, and the currently selected patch may vary in location from the previously selected patch. Thus, if for the forgery method, we select a larger area that contains several inpainting patches we will not be able to properly determine a similar area. On the other hand, if for the forgery method, we select a smaller patch, we might need to confront 2 aspects: one will be the speed of the method and the other will be the necessity to add some other mechanism to remove false positives.

3.1.1. Classical Image forensic techniques

Considering the above constraints, various authors proposed some areas to circumvent these problems. One of the first proposed methods to detect object removal was [63] – Detection of Examplar Based Inpainting - DEBI. The authors proposed a zero connectivity algorithm and added a fuzzy logic to exclude areas likely to be false positives. They make the following observations from the Criminisi paper [25]: because of the way the object removal algorithm works, it will end up introducing some patches which are going to be disconnected. Their algorithm proposes to scan the entire image (or rather a region of interest), compare patches and if they are found similar, pass them through a fuzzy logic semi-trapezoidal membership function. When comparing patches to determine if they are similar – they proposed the following:

o

Compute all patches from the image (or from ROI). They are called the suspicious patches

o

For each patch in the suspicious patches apply the following algorithm:

▪

Compute all the other image patches and compare each one of them to the suspicious path:

▪

Create a difference between the two patches

▪

Binarize the difference matrix

▪

Find the longest connectivity (either 4way or 8way) inside the binarized matrix

▪

Compare the obtained value with the maximum longest connectivity obtained for the suspicious patch

o

In the end, apply fuzzy logic to exclude some of the false positive cases.

The algorithm works well for some test scenarios, but it has some serious drawbacks, especially if the targeted image is not altered via the Criminisi inpainting algorithm, but rather with a newer variant of patch-based (or ML based). First of all, the computation effort is very high. If we were not to specify an ROI (and usually this is the case), then for a given image of size MxN and a block size B, the algorithm has to compute (M - B + 1) * (N - B + 1) patches. Then they have to take each patch and compare it to the other patches, thus the time grows exponentially. Also, the fuzzy logic works rather well if the ROI area is given, but in case the entire image is ROI, then it fails. Also, the algorithm signals a lot of false positives for larger areas.

Several years later, in [64] they enhanced the original paper from 2008 with some extra steps. The first major change the authors did is to enhance the search mechanism. They are proposing a two-step search. First, they are using a weight-transformation of the blocks. The main idea is to be able to increase the performance of the block comparison by first grouping similar blocks into a structure similar to a dictionary, where several blocks that resemble are categorized in the same “key” of the dictionary. Later the comparison of blocks will be done only per key, that being said, they will take all blocks which belong to the same key and try to see what is the best match. The authors are comparing several mechanisms for various methods of block grouping like equal numbers, even numbers, odd numbers, prime numbers, and a weight transformation proposed by them.

Another relevant improvement in this search mechanism was later proposed by authors in [65], where they are using only the central pixel as the key, rather than computing a value by applying the weight transformation. After that, they use the information to search for similar blocks inside the grouped blocks. They are using again for block comparison the Zero Connectivity algorithm proposed in the [63]. After this operation, they are proposing a vector filtering mechanism to eliminate a lot of false positives. Shortly they are implying that if two blocks are similar, their distance should be short as possible in a normal situation. If the distance is greater (the authors in all papers do not present the values used) than a threshold they are marking that block as potentially forged. The next step in their research is to decrease even further the false positive rates. For this, they’ve noticed that, usually Criminisi’s algorithm copies patches from different areas. Based on this observation, they proposed to use a multi-region correlation. Shortly blocks identified in previous steps are grouped (if they are connected), thus obtaining some bigger regions. For each region, it is identified where are the reference blocks to it. If there is a self-relationship group or a pair-relation group then these area is marked as not forged. So if we were to summarize this last step, the region created by blocks that are connected is marked as forged, as long as there are referenced blocks from other regions (the authors proposed a value of 3 – thus if the region is referencing at least 3 other regions than that blocked is marked as forged). Again, as a result, the method overall seems to produce good results for Crimini object removal cases, but it still has problems with diffusion-based inpainting or with enhanced example-based object removal methods. Another two potential improvements can be addressed in the area of speed, it still takes a lot of time to calculate the hash value of each block, and secondly the fact that they apply zero connectivity mechanisms is good for areas at the inpainted region, but to apply some mechanisms somewhere in the middle of the inpainted region might seem as a redundant step. Last, but not least, from all the analyzed pictures it seems it produces good results when the inpainted area (removed object) is either in a fully textured region or the object removed is at the boundary between two texturized regions.

Another improvement was suggested in [66] in which the authors suggested a jump patch approach increases the overall results. Later in [65] authors suggested several improvements to the original 2013 paper [64]. The first difference compared to the original paper is that they improve the two searches by changing the weight approach with a simpler and faster approach. They take as “key” only the central pixel value of each block. Secondly then computing the differences matrix, they suggest replacing the AND with an OR between the color components, in this way the recent paper makes a stricter comparison of blocks. This algorithm seems an important improvement versus the previous ones, but it still suffers in different areas:

• They have tested only on recent Crimini’s variation paper, not on state-of-the-art (at that time) methods for inpainting (and especially for object removal) – they’ve used [67].
• The computation effort is still very high. Again, applying the GZL in the middle-forged area seems a little too exhaustive and probably will not affect the overall results.

The same authors, several years later proposed in [68] to extend the above framework with a machine learning-based approach. If the above frameworks do not generate any false positives, images are fed into an ensemble classifier. For feature extractions, the authors rely on the fact that there has to be a Generalized Gaussian Distribution between the DCT coefficients of various blocks. Other authors tried similar approaches like the authors proposed in the CMFD framework. For e.g. in [69] suggested using Gabor magnitude as feature extractions to extract the features. The rest of the methods resemble the CMFD proposed framework (like block comparing, sorting, and detection). The interesting thing regarding this paper is the claims that the method is robust against inpainting methods, although they didn’t mention what method they applied for object removal. Another interesting method was proposed [70]. Here the authors have analyzed the impact of sparsity-based inpainting on the color correlations. They’ve noticed that a modified canonical correlation analysis might be able to properly detect these artifacts. Again, we must emphasize that the method works only on sparsity-based inpainting. An interesting direction was in analyzing the reflectance of the forged and non-forged areas. The authors in [71] have identified that some inpainting methods leave these traces and they suggest that this approach should be combined with the CMFD framework. Below you can see a summary table of the methods described.

Table 1. Classical inpaiting forgery detection.

Table 1. Classical inpaiting forgery detection.

Reference article	Year	Observations
[63]	2008	The first found method which tackles inpainting methods. They test against Crimini datset. Method relies on detecting similar patches and applies a fuzzy logic for similar patches.
[64]	2013	They continue the work on [63] and add several mechanism to exclude a lot of false positives
[65]	2015	The authors come with 2 proposals – first do not compute the block differences – only compare the central pixel (this improves performance and the accuracy is not that much affected). Secondly they proposed an improved method comparing to [64] of filtering – eliminating false postives.
[66]	2013	Similar method as the on in [64]. For better results / faster computation – they suggest a jump patch best approach.
[68]	2018	The same authors which proposed [65], come with an additional step which consists of an ensemble learning. They rely on Generalized Gaussian Distribution between the DCT coefficients of various blocks
[69]	2015	The authors took the CMFD framework proposed in [62] but use as feature extractions the Gabor magnitude
[70]	2018	The authors took the CMFD framework proposed in [62] but use as feature extractions the color correlations between patches
[71]	2020	The focus was on analyzing the reflectance of the forged and non-forged areas

3.1.2. Machine learning-based methods

Before we start on a deep review of deep learning methods for image inpainting, we would like to summarize all the methods classical methods that constitute the basis for the deep learning approach and their limitations. The first focus was on analyzing the physical traces. Here some of the base ideas are searching for blur inconsistencies, chromatic aberrations inconsistencies, or radial lens distortions (for a more in-depth review we strongly suggest the work of [5] and [44]. Usually, the methods imply splitting the image into patches and searching for inconsistencies in one of the three areas mentioned earlier. The work does not target a specific type of forgery because it relies on the assumption that whatever type of forgery method was applied (copy-paste, resampling, inpainting, copy-move), they all disturb the overall balance on the above properties. Mention here a few relevant papers – which are going to be the basis for machine learning methods:

• For blur inconsistencies, one of the most cited paper is [72]. They rely on the assumption that if the targeted original image contains some blur, by combining parts from other images the blur will be inconsistent. They are proposing a method to analyze the gradients and detect inconsistencies among them. Of course, the method does not give good results in case the target digital image does not contain some blur artifacts.
• Some other researchers focused on other camera properties like lens traces. The authors in [73], postulated that in some cases it is possible to detect especially copy-move forgeries by analyzing the lens discrepancies at block level. Their method basically detects edges and extracts distorted lines and use this in classical block based approach to analyze discrepancies within the image. The problem with this approach is that if the targeted area is either to big – or too small the results yielded are not very satisfactory, also there is another problem with low resolution images, because they tend to yield false positive results.
• A very good camera-specific parameter that was heavily studied was the noise generated at image acquisition. Several authors have proposed different mechanisms to detect inconsistencies at block noise levels. Some authors even went in the direction that based on noise patterns they are able to uniquely identify the camera model. To name a few of the most cited works [74,75]. For e.g. in[75], the authors suggested computing noise for non-overlapping blocks and then unifying regions which have similar noise – thus partitioning the image into areas of the same noise intensity. The authors suggested using wavelet + a median filter approach on the grayscale image to compute the noise. Of course, the main limitations of these methods vary from false positives to the impossibility of the methods to detecting if noise level degradation is very small (a lot of anti-forgery mechanisms can exploit this method).
• Color filter array methods or demosaicking methods(CFA) – rely on the observation that most cameras capture only one color per pixel and then use some algorithms to interpolate these values. The forgery detection mechanism based on CFA – basically detects inconsistencies at block levels between the patterns generated by CFA. One of the most cited work is [76], in which the authors are proposing to use a small block (up 2x2) to detect inconsistencies in the CFA pattern. They are extracting the green channel from the image and calculate a prediction error and analyze the variance of the errors in order to mark the non-overlapping block as forged or not. The method yield good results as the original image does not suffer some post-processing operations like color normalization.

One of the first machine learning-based methods was the work of Linchuan Shen, Gaobo Yang, Leida Li, and Xingming Sun [77]. Here they propose a support vector machine classifier composed of the following features: local binary pattern features, gray-level co-occurrence matrix features, and gradient features (actually they suggest to use fourteen features extracted from patches). One key aspect to mention in their research is the robustness of this method especially on post-processing operations (like jpeg compression, scaling, noise, etc). Another variant of ML-based detection is the work [78] in which they employ a standard CNN model to be able to detect tampered regions or the work [79] which suggests using four Resnet modules to extract features from the image blocks or a later one which employs a hybrid model of LTSM and CNN [80]. Similar to the above was also the work in [81] in which they use a simple CNN and extract general features from inpainted vs non-inpainted areas. The main problem with the above methods is that they either test the network on Criminisi’s old paper, or they randomly select a center area on real images and apply the latest image inpainting methods, neither with the potential to be used in real-life scenarios.

In the last years, the focus for detecting inpainting images is to apply higher and more complex machine learning models, with some strong feature extract mechanisms. For example in [82] [83] or [84], the authors suggested that the noise variance is disturbing the inpainted area, thus by applying three Resnet feature blocks in a multi-scale network, they obtained very good results. They use for testing the latest state-of-the-art deep inpainting methods, but they random masks for applying the inpainting methods, thus not a very realistic approach. Another fusion-based approach was proposed in [85]. Their work was actually inspired by [79], [86,87], and they suggest to use of three enhancements blocks: Steganalysis Rich Model – to enhance noise inconsistency, Pre-Filtering to enhance discrepancy in high-frequency components, and a Bayar filter to enable the adaptive acquisition of low-level prediction residual features. After these blocks, the authors used a search block to detect the areas based on the three enhancement blocks. The last step is a decision block – because usually there is an inconsistency in how pixels are classified. If we take a closer look at the results, the authors obtained very good results compared to previous algorithms. An interesting approach is also on how they generate the data – they incorporate various datasets of pictures and use several inpainting methods to generate forged data. The problem still with their approach is that the mask data used for the inpainting process is generated randomly, not in a realistic approach. In the same category as IID-NET methods, others have suggested [88] incorporating more enhancement blocks to make the detection more reliable and for a more general range of forgeries, and they used several datasets some of which present not very realistic tampering (more on the datasets on the next chapter).

Another more complex architecture was added in [89]. The authors based their network architecture on the work of [90]. Again, like in previous machine learning approaches, the authors try to create a method to suit all types of forgery like splicing, copy-move, or inpainting. The novelty in [89] was to add a top-down and a bottom-up path encompassed by another color correlation module called the Spatio-Channel Correlation module. Basically in the top-down area, they are trying to detect features at different scales, while in the bottom-up part, they start from the mask to find and strengthen the correlation between forged and not forged areas. They claim to obtain top SOT compared with other systems like [91], [92]. Again, it seems the authors opted to create their own data set to tackle various forgery types like splicing, copy-move, and object removal. An interesting thing was that they didn’t compare their work with another claimed SOTA [85]. Also, the work based on noise inconsistencies [83] appears to surpass these approaches in terms of results, but not in terms of speed and model complexity (the authors in [89] claimed to use a small network model with approximately 3 M parameters only).

To summarize the above-presented machine learning based, we can see two trends regarding detection types: algorithms that focus solely on detecting inpainting (object removal) forgeries and methods that try to tackle all kinds of possible forgeries. From the dataset, point of view we see that most of the users either manually generate datasets by applying some random masks or use pretty old datasets. From the results point of view, we can see that there are several areas that offer good results, the most promising ones seem to be in the detection of a mismatch in either noise and or color information presented. In the below table, you should see a summary of the presented data above.

Table 2. Machine learning based inpainting forgery detection.

Table 2. Machine learning based inpainting forgery detection.

Reference article	Year	Observations
[77]	2017	The main idea was to use a SVM classifier composed of the following features: local binary pattern features, gray-level co-occurrence matrix features, and gradient features (actually they suggest to use fourteen features extracted from patches).
[78]	2018	Standard CNN model on which they trained original / altered patches.
[79]	2019	Same as principle ideea as [78] but they choose a Resnet model
[80]	2020	The authors employed a combination of Resnet + LSTM to better protray the differences between altered vs non-altered regions. All the above methods were mainly tested against Criminisi’s initial paper, thus not having to “compete” with latest image inpainting methods at that time.
[81]	2021	A tweaked version of a VGG model architecture
[83]	2022	A CNN model with focus on detecting noise inconsistencies
[84]	2022	A U-NET VGG model which adds an enhancement block of 5 filters (4 SRM + Laplacian) to be able to better detect inpainted areas.
[85]	2022	The authors suggest to use of three enhancements blocks: Steganalysis Rich Model – to enhance noise inconsistency, Pre-Filtering to enhance discrepancy in high-frequency components, and a Bayar filter to enable the adaptive acquisition of low-level prediction residual features similar to the Mantranet architecture.

4. Image inpainting datasets Datasets

4.1. General forgery datasets

One of the first truly realistic forgery datasets is done in [105] and in [62]. The MICC dataset was one of the widely accepted datasets for image forgery. It actually consists of 4 sub-datasets called F220, F2000, F8multi, and F600. The CMFD dataset contains updates that were done manually using professional tools. It is a generic dataset for copy-move scenarios, not only object removal use cases. Another relevant property worth mentioning is the fact that the dataset is grouped by the types of cameras which acquire the pictures. Analyzing the literature for usage of this dataset, it was observed that the dataset was not referenced, probably because it contains large images, and using a block-based comparison needed a lot of time to compute. Approximately at the same time, authors in [106] and [107] proposed a similar dataset again focused on copy-move. Still, some of the pictures can be considered object removal rather than copy-pasted and thus constitute a good measure to test inpainting methods. What differentiates these datasets from the CMFD one is the fact that the authors proposed here some variants of the forged ones which undergo some post-processing anti-forensic measures like jpeg compression, blurring, and noise adding. An important item to mention is that the images for the Comofod dataset were generated using Adobe Photoshop (especially for the splicing artifacts). The interesting, unique area for the Comofod dataset is actually that the authors tried to have images which naturally contain similar but genuine objects. For the Casia dataset, the authors actually introduce two sub-datasets namely Casia V1 and Casia V2. The second version contains pictures with several sizes (up to 900 x 600) and an important item is that they add also in some instances, some pre-processing and post-processing operations like resizing, distortion, blurring, noising, etc – thus making the detection harder. Some the images are quite realistic generated – the copy region is not a simple copy-move region, but the copied region is also resized, rotate and went through additional processing. Also is important to notice that the original dataset didn’t offered the masks, so the following [108] can be used as a reference for the masks and also some filtering of the initial dataset (miss-categorizations or duplicates removal). In the same category but newer versions are the COVERAGE dataset [109] or the work of Pawel Korus in his papers [110], [111]. The COVERAGE dataset although rather small (100 images with an average size of around 400x500 pixels) tries to make things even harder by suggesting pictures that contain similar but genuinely valid objects. The “copied” items inside their images, thus already resemble multiple instances of similar objects, and the difficult task is in detecting real–faked areas vs similar instances. The work of Pawel Korus contains one of the most realistic forgeries which focuses on several types of different cameras (some images were own made some were taken from [112]). The size of the images is relatively bigger than previous work (1920 x 1080), but in total there are around only 220 forged imaged, and the forgery methods vary from copy-move to object removal. An important item to mention is that they used GIMP + Affinity photo for generating the forgeries and the dataset consists of 4 different camera model. The authors basically suggest that when copy-move or object removal is applied on the input image they distorce the overall noise between patches inside the image. Another relevant dataset that started in 2015 with the Media Forensic Challenge for Darpa[113] [114], now consists of millions of media that cover several forged mechanisms including both images and videos. The majority of media data is collected from the internet from various sources and in different contexts to ensure a more generic dataset. From these millions of media, approximately 200k images are what the authors called High Provenance – basically the authors ensure that the images have not undergone any forgery attacks. For this large dataset, the authors employed experienced people for performing the various forgery attacks (and also not limited to using only a specific tool for the forgery generation). The MFC dataset consists of several sub-tasks – like image/video manipulation localization, splice image/video manipulation, provenance, graph building (detecting all the steps and sources a target image has undergone), GAN, etc. At this point, it is still considered one of the referenced forgery datasets both from quality and quantitively perspectives. The MFC dataset (2019 version) has approximately 180 000 original images, obtained directly from camera, thus ensure the images are not manipulated in any way and all device specific settings – like e.g. ISO are know beforehand. Also to ensure a large diversity of the images, the authors used more than 500 different cameras in order to have a large poolset of different noise that can be used for training (PRNU based algorthims). From this approximatelly 16k images have been altered. Still, the main problem with our object-removal (content aware fill) / inpainting tasks is that a relatively small from this dataset consists of the type of forgery we are focused on. Even the authors in MFC recognized that they add several methods under the generic umbrella of copy-move alterations.

Table 3. General forgery datasets - for which a subset can be used for inpainting/object removal detection.

Table 3. General forgery datasets - for which a subset can be used for inpainting/object removal detection.

Name	Dataset Size (GB)	Number of pristine / forged pictures	Image size(s)	Type*	Mode	Observation
MICC	6.5 GB	1850 / 978	722 x 480 to 2048 x 1536	CM/OR	M	Some of the images are not very realistic, but it tries to generate several types of copy-move by applying rotation and scaling. The problem is that always the forged area is a rectangular
CMFD	1 GB	48 / 48	3264 x 2448 3888 x 2592 3072 x 2304 3039 x 2014	CM/OR	M	Very realistic dataset. Some of the images due to the fact that they use professional tools – are a mixing of copy-move, object removal, and sampling. Group by camera type. Important thing to notice is that there is no post processing operations done on the image, but because of the high-quality / size researches can do their on post processing. Images were processed using GIMP
CoMoFoD	3 GB	200 + 60 / 200 + 60	512 x 512 3000x2000	CM/OR	M	Canon camera used only. They’ve used 6 post processing operations, for e.g. JPEG compression with 9 various quality level, or changing brighness, noise etc. The operations was done in Photoshop. 3Gb is only the small variant of the dataset
CASIA	3GB V2	7491 / 5123	160x240 to 900x600	CM+S	N	Contains different type of copied areas like resize, rotate and post-processing of the forged area.
COVERAGE	150MB	100/100	235x340 to 752x472	CP	N	Original images already contain similar objects, thus makes harder to detect. The forged is relative large – 60% of the images have at least 10% forged area.
Realistic Tampering Dataset	1.5 GB	220/220	1920x1080	CP/OR	M/A	The dataset contains 4 different types of camera, and focus on the inconsistencies at noise level between patches. The images were pre/post processed with GIMP
MFC	150GB	16k / 2 M	All sizes	CP/OR	N	They’ve used a series of techniques like simple copy-move, to content aware fill, seam carving etc.

*Type CP = Copy-Move, OR = Object removal, S = Splicing. Mode M=Manual, A=Automatic, N=Not mentioned.

4.2. Image inpainting specific datasets

One of the first datasets with a specific division of object removal was the work in [115]. They have indeed a very large dataset about 25000 images which were generated using [116]. In order to generate the dataset, they actually use the MSCOCO [117] which contains descriptive information. Then they select randomly from each image an descriptive region and dillate that raw mask and apply the inpainting algorithm and removing images for which standard deviation for the inpainting regions was below a given threshold. The problem with the dataset is that they generated the masks randomly, thus in highly texturized areas (or where the MSCOCO description is not accurate), or at border regions, the forged images are blurring and can be easily detected even by human observation. In the same methodology as the previous authors, is the dataset entitled IMD2020 [118]. In this dataset, the authors took 35k pictures from Flickr, manually reviewed them, and then used a combination of classical inpainting from OpenCV and a machine learning-based model for inpainting [119]. One issue pertaining to this dataset was the incomplete differentiation between the original and forged areas in the masks.. If we compare the pixels from the original vs forged area, we will have a lot of differences, not only the altered region (probably the images went some post-processing which were not mentioned in the paper, or the images were altered as part of the inpainting procedure). Another problem with this dataset is that not all forged regions are realistic. One of the recent datasets proposed especially for the inpainting (object removal) detection is the work [85]. They present about 10k images, but as a novelty they do not target one specific inpainting method but rather propose 10 methods (6 classical method and 4 machine learning based). Still we observe the same problem with the mask generated for the inpainting process. Because they’ve generated the mask with different sizes and shapes, and do not take into account the nature of each image, some of the generated inpaintings are not very satisfactory. Also another problem is that the dataset consists of very small images – all have the same 256x256 size.

Table 4. Image inpainting forensic dataset.

Table 4. Image inpainting forensic dataset.

Name	Dataset Size (GB)	Number of pristine / forged pictures	Image size(s)	Observation
DEFACTO INPAINTING	13 GB	10312 / 25000 (they’ve applied inpainting for same image but for different areas)	180x240 to 640x640	Some of the images are not very realistic inpainted due to the automatic randomized selection of the area from the MSCOCO dataset
IMD2020	38GB	35k / 35K	640x800 to 1024x920	Some of the images are not very realistic, and the forged image underwent some additional changes (probably some noise filtering / color unifirmatization). An interesting fact is that they manualy selecting area and than using an automated algorthim, that means no post-processing / enhacements
IID-NET	1.2GB	11k / 11k	256x256	Random masks (based on MSCOCO) + 11 different automated algorithms of filling / removing object/ The idea is interesting to try to tackle different inpainting algorithms, still there are some problems in how the mask inpainted area is choosen. Also another problem is that altough several inpainting algoritms are tested, they are applied on different images.

5. Results and Discussion

Figure 6. Original and mask image from dataset.

Figure 6. Original and mask image from dataset.

Figure 7. Inpainted results: a – Criminisi[25], b – Gimp[122], c – NonLocalPatch[121], Lama[45], Mat[123].

Figure 7. Inpainted results: a – Criminisi[25], b – Gimp[122], c – NonLocalPatch[121], Lama[45], Mat[123].

Figure 8. Sample artifact introduced by [121].

Figure 8. Sample artifact introduced by [121].

Figure 9. DEBI [63] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – NonLocalPatch[121], Lama[45], Mat[123].

Figure 9. DEBI [63] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – NonLocalPatch[121], Lama[45], Mat[123].

Figure 10. Mantranet [91] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 10. Mantranet [91] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 11. IID [85] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 11. IID [85] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 12. Focal [124] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 12. Focal [124] results on inpainted images with: a – Criminisi[25], b – Gimp[122], c – Non-LocalPatch[121], Lama[45], Mat[123].

Figure 13. Evaluation metric for the results of DEBI detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch.

Figure 13. Evaluation metric for the results of DEBI detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch.

Figure 14. Evaluation metric for the results of CMFD detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch.

Figure 14. Evaluation metric for the results of CMFD detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch.

Figure 15. Evaluation metric for the results of Focal detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 15. Evaluation metric for the results of Focal detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 16. Evaluation metric for the results of IID-NET detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 16. Evaluation metric for the results of IID-NET detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 17. Evaluation metric for the results of Mantranet detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 17. Evaluation metric for the results of Mantranet detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 18. Evaluation metric for the results of PSCC-NET detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 18. Evaluation metric for the results of PSCC-NET detection method applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 19. Summary evaluation metrics for the results of IID-NET,Focal,Mantranet,PSCC-NET detection methods applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

Figure 19. Summary evaluation metrics for the results of IID-NET,Focal,Mantranet,PSCC-NET detection methods applied on the inpainted Open Images Dataset V7 dataset with the following inpainting methods: Criminis, Gimp, NonLocalPatch, Lama and Mat.

References

1. M. A. Qureshi and M. Deriche, “A bibliography of pixel-based blind image forgery detection techniques,” Signal Process Image Commun, vol. 39, pp. 46–74, 2015. [CrossRef]
2. A. Gokhale, P. Mulay, D. Pramod, and R. Kulkarni, “A Bibliometric Analysis of Digital Image Forensics,”. [CrossRef]
3. F. Casino et al., “Research Trends, Challenges, and Emerging Topics in Digital Forensics: A Review of Reviews,” IEEE Access, vol. 10, pp. 25464–25493, 2022. [CrossRef]
4. NOVA, “NOVA | scienceNOW | Profile: Hany Farid | PBS.” https://www.pbs.org/wgbh/nova/sciencenow/0301/03.html (accessed Sep. 09, 2021).
5. P. Korus, “Digital image integrity – a survey of protection and verification techniques,” Digit Signal Process, vol. 71, pp. 1–26, Dec. 2017. [CrossRef]
6. K. Liu, J. Li, S. Sabahat, and H. Bukhari, “Overview of Image Inpainting and Forensic Technology,” 2022. [CrossRef]
7. O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, “Image Inpainting: A Review,” Neural Processing Letters 2019 51:2, vol. 51, no. 2, pp. 2007–2028, Dec. 2019. [CrossRef]
8. D. J. B. Rojas, B. J. T. Fernandes, and S. M. M. I. Fernandes, “A Review on Image Inpainting Techniques and Datasets,” Proceedings - 2020 33rd SIBGRAPI Conference on Graphics, Patterns and Images, SIBGRAPI 2020, pp. 240–247, Nov. 2020. [CrossRef]
9. J. Jam, C. Kendrick, K. Walker, V. Drouard, J. G. S. Hsu, and M. H. Yap, “A comprehensive review of past and present image inpainting methods,” Computer Vision and Image Understanding, vol. 203, p. 103147, Feb. 2021. [CrossRef]
10. Z. Tauber, Z. N. Li, and M. S. Drew, “Review and preview: Disocclusion by inpainting for image-based rendering,” IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 37, no. 4, pp. 527–540, Jul. 2007. [CrossRef]
11. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image inpainting,” in Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH ’00, New York, New York, USA: ACM Press, 2000, pp. 417–424. [CrossRef]
12. M. Bertalmío, A. L. Bertozzi, and G. Sapiro, “Navier-Stokes, fluid dynamics, and image and video inpainting,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 2001. [CrossRef]
13. M. Bertalmío, “Contrast invariant inpainting with a 3RD order, optimal PDE,” Proceedings - International Conference on Image Processing, ICIP, vol. 2, pp. 775–778, 2005. [CrossRef]
14. T. F. Chan and J. Shen, “Nontexture Inpainting by Curvature-Driven Diffusions,” J Vis Commun Image Represent, vol. 12, no. 4, pp. 436–449, Dec. 2001. [CrossRef]
15. T. F. Chan, S. H. Kang, and J. Shen, “Euler’s Elastica and Curvature-Based Inpainting,” vol. 63, no. 2, pp. 564–592, Jul. 2006. [CrossRef]
16. D. Tschumperlé, “Fast anisotropic smoothing of multi-valued images using curvature-preserving PDE’s,” Int J Comput Vis, vol. 68, no. 1, pp. 65–82, Jun. 2006. [CrossRef]
17. A. Telea, “An Image Inpainting Technique Based on the Fast Marching Method,”. vol. 9, no. 1, pp. 23–34, Jan. 2012. [CrossRef]
18. C. B. Schönlieb’ and A. Bertozzi, “Unconditionally stable schemes for higher order inpainting,” Commun Math Sci, vol. 9, no. 2, pp. 413–457, 2011. [CrossRef]
19. P. Jidesh and S. George, “Gauss curvature-driven image inpainting for image reconstruction,” vol. 37, no. 1, pp. 122–133, Jan. 2014. [CrossRef]
20. G. Sridevi and S. Srinivas Kumar, “p-Laplace Variational Image Inpainting Model Using Riesz Fractional Differential Filter,” International Journal of Electrical and Computer Engineering (IJECE), vol. 7, no. 2, pp. 850–857, Apr. 2017. [CrossRef]
21. G. Sridevi and S. Srinivas Kumar, “Image Inpainting and Enhancement using Fractional Order Variational Model,” Def Sci J, vol. 67, no. 3, pp. 308–315, Apr. 2017. [CrossRef]
22. G. Sridevi and · S Srinivas Kumar, “Image Inpainting Based on Fractional-Order Nonlinear Diffusion for Image Reconstruction,” Circuits Syst Signal Process, vol. 38, pp. 3802–3817, 2019. [CrossRef]
23. S. Gamini, V. V. Gudla, and C. H. Bindu, “Fractional-order Diffusion based Image Denoising Model,” International Journal of Electrical and Electronics Research, vol. 10, no. 4, pp. 837–842, 2022. [CrossRef]
24. A. A. Efros and T. K. Leung, “Texture Synthesis by Non-parametric Sampling,” 1999.
25. A. Criminisi, P. Pérez, and K. Toyama, “Region filling and object removal by exemplar-based image inpainting,” IEEE Transactions on Image Processing, vol. 13, no. 9, pp. 1200–1212, Sep. 2004. [CrossRef]
26. N. Kumar, L. Zhang, and S. Nayar, “What is a good nearest neighbors algorithm for finding similar patches in images?,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5303 LNCS, no. PART 2, pp. 364–378, 2008. [CrossRef]
27. “PatchMatch: A Randomized Correspondence Algorithm for Structural Image Editing.” https://gfx.cs.princeton.edu/pubs/Barnes_2009_PAR/ (accessed May 19, 2023).
28. T. Ružić and A. Pižurica, “Context-aware patch-based image inpainting using Markov random field modeling,” IEEE Transactions on Image Processing, vol. 24, no. 1, pp. 444–456, Jan. 2015. [CrossRef]
29. K. H. Jin and J. C. Ye, “Annihilating Filter-Based Low-Rank Hankel Matrix Approach for Image Inpainting,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3498–3511, Nov. 2015. [CrossRef]
30. N. Kawai, T. Sato, and N. Yokoya, “Diminished Reality Based on Image Inpainting Considering Background Geometry,” IEEE Trans Vis Comput Graph, vol. 22, no. 3, pp. 1236–1247, Mar. 2016. [CrossRef]
31. Q. Guo, S. Gao, X. Zhang, Y. Yin, and C. Zhang, “Patch-Based Image Inpainting via Two-Stage Low Rank Approximation,” IEEE Trans Vis Comput Graph, vol. 24, no. 6, pp. 2023–2036, Jun. 2018. [CrossRef]
32. H. Lu, Q. Liu, M. Zhang, Y. Wang, and X. Deng, “Gradient-based low rank method and its application in image inpainting,” Multimed Tools Appl, vol. 77, no. 5, pp. 5969–5993, Mar. 2018. [CrossRef]
33. L. Shen, Y. Xu, and X. Zeng, “Wavelet inpainting with the ℓ0 sparse regularization,” Appl Comput Harmon Anal, vol. 41, no. 1, pp. 26–53, Jul. 2016. [CrossRef]
34. B. M. Waller, M. S. Nixon, and J. N. Carter, “Image reconstruction from local binary patterns,” Proceedings - 2013 International Conference on Signal-Image Technology and Internet-Based Systems, SITIS 2013, pp. 118–123, 2013. [CrossRef]
35. Hong-an Li, Liuqing Hu, Jun Liu, Jing Zhang & Tian Ma (2023) A review of advances in image inpainting research, The Imaging Science Journal. [CrossRef]
36. D. Rasaily and M. Dutta, “Comparative theory on image inpainting: A descriptive review,” 2017 International Conference on Energy, Communication, Data Analytics and Soft Computing, ICECDS 2017, pp. 2925–2930, Jun. 2018. [CrossRef]
37. B. Shen, W. Hu, Y. Zhang, and Y. J. Zhang, “Image inpainting via sparse representation,” ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, pp. 697–700, 2009. [CrossRef]
38. Z. Xu and J. Sun, “Image inpainting by patch propagation using patch sparsity,” IEEE Transactions on Image Processing, vol. 19, no. 5, pp. 1153–1165, May 2010. [CrossRef]
39. P. Tiefenbacher, M. Sirch, M. Babaee, and G. Rigoll, “Wavelet contrast-based image inpainting with sparsity-driven initialization,” Proceedings - International Conference on Image Processing, ICIP, vol. 2016-August, pp. 3528–3532, Aug. 2016. [CrossRef]
40. A. Bugeau, M. Bertalmío, V. Caselles, and G. Sapiro, “A comprehensive framework for image inpainting,” IEEE Transactions on Image Processing, vol. 19, no. 10, pp. 2634–2645, Oct. 2010. [CrossRef]
41. J. F. Aujol, S. Ladjal, and S. Masnou, “Exemplar-Based Inpainting from a Variational Point of View,” vol. 42, no. 3, pp. 1246–1285, May 2010. [CrossRef]
42. W. Casaca, M. Boaventura, M. P. De Almeida, and L. G. Nonato, “Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images,” Pattern Recognit Lett, vol. 36, no. 1, pp. 36–45, Jan. 2014. [CrossRef]
43. D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context Encoders: Feature Learning by Inpainting,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-December, pp. 2536–2544, Dec. 2016. [CrossRef]
44. Z. Qin, Q. Zeng, Y. Zong, and F. Xu, “Image inpainting based on deep learning: A review,” Displays, vol. 69, p. 102028, Sep. 2021. [CrossRef]
45. R. Suvorov et al., “Resolution-robust Large Mask Inpainting with Fourier Convolutions,” Proceedings - 2022 IEEE/CVF Winter Conference on Applications of Computer Vision, WACV 2022, pp. 3172–3182, 2022. [CrossRef]
46. Z. Lu, J. Jiang, J. Huang, G. Wu, and X. Liu, “GLaMa: Joint Spatial and Frequency Loss for General Image Inpainting,” IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, vol. 2022-June, pp. 1300–1309, May 2022. [CrossRef]
47. P. Shamsolmoali, M. Zareapoor, and E. Granger, “Image Completion Via Dual-Path Cooperative Filtering,” ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, Jun. 2023. [CrossRef]
48. Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep Learning Face Attributes in the Wild *,” 2015. [CrossRef]
49. B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 Million Image Database for Scene Recognition,” IEEE Trans Pattern Anal Mach Intell, vol. 40, no. 6, pp. 1452–1464, Jun. 2018. [CrossRef]
50. A. Lugmayr, M. Danelljan, A. Romero, F. Yu, R. Timofte, and L. Van Gool, “RePaint: Inpainting using Denoising Diffusion Probabilistic Models,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2022-June, pp. 11451–11461, Jan. 2022. [CrossRef]
51. R. K. Cho, K. Sood, and C. S. C. Channapragada, “Image Repair and Restoration Using Deep Learning,” AIST 2022 - 4th International Conference on Artificial Intelligence and Speech Technology, 2022. [CrossRef]
52. Y. Chen, R. Xia, K. Yang, and K. Zou, “DGCA: High resolution image inpainting via DR-GAN and contextual attention,” Multimed Tools Appl, pp. 1–21, May 2023. [CrossRef]
53. P. Jeevan, D. S. Kumar, and A. Sethi, “WavePaint: Resource-efficient Token-mixer for Self-supervised Inpainting,” Jul. 2023, Accessed: Sep. 10, 2023. [Online]. Available: https://arxiv.org/abs/2307.00407v1.
54. P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high-resolution image synthesis,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 12868–12878, 2021. [CrossRef]
55. H. Farid, “Image forgery detection,” IEEE Signal Process Mag, vol. 26, no. 2, pp. 16–25, 2009. [CrossRef]
56. M. Zanardelli, F. Guerrini, R. Leonardi, and N. Adami, “Image forgery detection: A survey of recent deep-learning approaches,” Multimed Tools Appl, vol. 82, no. 12, pp. 17521–17566, May 2022. [CrossRef]
57. N. T. Pham and C. S. Park, “Toward Deep-Learning-Based Methods in Image Forgery Detection: A Survey,” IEEE Access, vol. 11, pp. 11224–11237, 2023. [CrossRef]
58. H. Li, W. Luo, and J. Huang, “Localization of Diffusion-Based Inpainting in Digital Images,” IEEE Transactions on Information Forensics and Security, vol. 12, no. 12, pp. 3050–3064, Dec. 2017. [CrossRef]
59. Y. Zhang, F. Ding, S. Kwong, and G. Zhu, “Feature pyramid network for diffusion-based image inpainting detection,” Inf Sci (N Y), vol. 572, pp. 29–42, Sep. 2021. [CrossRef]
60. Y. Zhang, T. Liu, C. Cattani, Q. Cui, and S. Liu, “Diffusion-based image inpainting forensics via weighted least squares filtering enhancement,” Multimed Tools Appl, vol. 80, no. 20, pp. 30725–30739, Aug. 2021. [CrossRef]
61. A. K. Al-Jaberi, A. Asaad, S. A. Jassim, and N. Al-Jawad, “Topological Data Analysis for Image Forgery Detection,” Indian Journal of Forensic Medicine & Toxicology, vol. 14, no. 3, pp. 1745–1751, Jul. 2020. [CrossRef]
62. V. Christlein, C. Riess, J. Jordan, C. Riess, and E. Angelopoulou, “IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY An Evaluation of Popular Copy-Move Forgery Detection Approaches”, Accessed: Feb. 06, 2022. [Online]. Available: http://www5.cs.fau.de/our-team.
63. Q. Wu, S. J. Sun, W. Zhu, G. H. Li, and D. Tu, “Detection of digital doctoring in exemplar-based inpainted images,” Proceedings of the 7th International Conference on Machine Learning and Cybernetics, ICMLC, vol. 3, pp. 1222–1226, 2008. [CrossRef]
64. I. C. Chang, J. C. Yu, and C. C. Chang, “A forgery detection algorithm for exemplar-based inpainting images using multi-region relation,” Image Vis Comput, vol. 31, no. 1, pp. 57–71, Jan. 2013. [CrossRef]
65. Z. Liang, G. Yang, X. Ding, and L. Li, “An efficient forgery detection algorithm for object removal by exemplar-based image inpainting,” J Vis Commun Image Represent, vol. 30, pp. 75–85, Jul. 2015. [CrossRef]
66. K. S. Bacchuwar, Aakashdeep, and K. R. Ramakrishnan, “A jump patch-block match algorithm for multiple forgery detection,” Proceedings - 2013 IEEE International Multi Conference on Automation, Computing, Control, Communication and Compressed Sensing, iMac4s 2013, pp. 723–728, 2013. [CrossRef]
67. J. Wang, K. Lu, D. Pan, N. He, and B. kun Bao, “Robust object removal with an exemplar-based image inpainting approach,” Neurocomputing, vol. 123, pp. 150–155, Jan. 2014. [CrossRef]
68. D. Zhang, Z. Liang, G. Yang, Q. Li, L. Li, and X. Sun, “A robust forgery detection algorithm for object removal by exemplar-based image inpainting,” Multimed Tools Appl, vol. 77, pp. 11823–11842, 2018. [CrossRef]
69. J. C. Lee, “Copy-move image forgery detection based on Gabor magnitude,” J Vis Commun Image Represent, vol. 31, pp. 320–334, Aug. 2015. [CrossRef]
70. X. Jin, Y. Su, L. Zou, Y. Wang, P. Jing, and Z. J. Wang, “Sparsity-based image inpainting detection via canonical correlation analysis with low-rank constraints,” IEEE Access, vol. 6, pp. 49967–49978, Aug. 2018. [CrossRef]
71. G. Mahfoudi, F. Morain-Nicolier, F. Retraint, and M. Pic, “Object-Removal Forgery Detection through Reflectance Analysis,” 2020 IEEE International Symposium on Signal Processing and Information Technology, ISSPIT 2020, Dec. 2020. [CrossRef]
72. P. Kakar, N. Sudha, and W. Ser, “Exposing digital image forgeries by detecting discrepancies in motion blur,” IEEE Trans Multimedia, vol. 13, no. 3, pp. 443–452, Jun. 2011. [CrossRef]
73. H. R. Chennamma and L. Rangarajan, “Image Splicing Detection Using Inherent Lens Radial Distortion,” IJCSI International Journal of Computer Science Issues, vol. 7, pp. 1694–0814, May 2011, Accessed: Jun. 05, 2023. [Online]. Available: https://arxiv.org/abs/1105.4712v1.
74. J. Lukáš, J. Fridrich, and M. Goljan, “Detecting digital image forgeries using sensor pattern noise,” vol. 6072, pp. 362–372, Feb. 2006. [CrossRef]
75. B. Mahdian and S. Saic, “Using noise inconsistencies for blind image forensics,” Image Vis Comput, vol. 27, no. 10, pp. 1497–1503, Sep. 2009. [CrossRef]
76. P. Ferrara, T. Bianchi, A. De Rosa, and A. Piva, “Image forgery localization via fine-grained analysis of CFA artifacts,” IEEE Transactions on Information Forensics and Security, vol. 7, no. 5, pp. 1566–1577, 2012. [CrossRef]
77. L. Shen, G. Yang, L. Li, and X. Sun, “Robust detection for object removal with post-processing by exemplar-based image inpainting,” ICNC-FSKD 2017 - 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, pp. 2730–2736, Jun. 2018. [CrossRef]
78. X. Zhu, Y. Qian, X. Zhao, B. Sun, and Y. Sun, “A deep learning approach to patch-based image inpainting forensics,” Signal Process Image Commun, vol. 67, pp. 90–99, Sep. 2018. [CrossRef]
79. H. Li and J. Huang, “Localization of deep inpainting using high-pass fully convolutional network,” Proceedings of the IEEE International Conference on Computer Vision, vol. 2019-October, pp. 8300–8309, Oct. 2019. [CrossRef]
80. M. Lu and S. Niu, “A Detection Approach Using LSTM-CNN for Object Removal Caused by Exemplar-Based Image Inpainting,” Electronics 2020, Vol. 9, Page 858, vol. 9, no. 5, p. 858, May 2020. [CrossRef]
81. N. Kumar and T. Meenpal, “Semantic segmentation-based image inpainting detection,” Lecture Notes in Electrical Engineering, vol. 661, pp. 665–677, 2021. [CrossRef]
82. A. Li et al., “Noise Doesn’t Lie: Towards Universal Detection of Deep Inpainting,” IJCAI International Joint Conference on Artificial Intelligence, pp. 786–792, Jun. 2021. [CrossRef]
83. X. Zhu, J. Lu, H. Ren, H. Wang, and B. Sun, “A transformer–CNN for deep image inpainting forensics,” Visual Computer, pp. 1–15, Aug. 2022. [CrossRef]
84. Y. Zhang, Z. Fu, S. Qi, M. Xue, Z. Hua, and Y. Xiang, “Localization of Inpainting Forgery With Feature Enhancement Network,” IEEE Trans Big Data, Jun. 2022. [CrossRef]
85. H. Wu and J. Zhou, “IID-Net: Image Inpainting Detection Network via Neural Architecture Search and Attention,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 3, pp. 1172–1185, Mar. 2022. [CrossRef]
86. B. Bayar and M. C. Stamm, “Constrained Convolutional Neural Networks: A New Approach Towards General Purpose Image Manipulation Detection,” IEEE Transactions on Information Forensics and Security, vol. 13, no. 11, pp. 2691–2706, Nov. 2018. [CrossRef]
87. P. Zhou, X. Han, V. I. Morariu, and L. S. Davis, “Learning Rich Features for Image Manipulation Detection,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1053–1061, Dec. 2018. [CrossRef]
88. Y. Zhou, H. Wang, Q. Zeng, R. Zhang, and S. Meng, “A Discriminative Multi-Channel Noise Feature Representation Model for Image Manipulation Localization,” ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, Jun. 2023. [CrossRef]
89. X. Liu, Y. Liu, J. Chen, and X. Liu, “PSCC-Net: Progressive Spatio-Channel Correlation Network for Image Manipulation Detection and Localization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 11, pp. 7505–7517, Nov. 2022. [CrossRef]
90. J. Wang et al., “Deep High-Resolution Representation Learning for Visual Recognition,” IEEE Trans Pattern Anal Mach Intell, vol. 43, no. 10, pp. 3349–3364, Oct. 2021. [CrossRef]
91. Y. Wu, W. Abdalmageed, and P. Natarajan, “Mantra-net: Manipulation tracing network for detection and localization of image forgeries with anomalous features,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2019-June, pp. 9535–9544, Jun. 2019. [CrossRef]
92. X. Hu, Z. Zhang, Z. Jiang, S. Chaudhuri, Z. Yang, and R. Nevatia, “SPAN: Spatial Pyramid Attention Network for Image Manipulation Localization,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12366 LNCS, pp. 312–328, 2020. [CrossRef]
93. H. Sharma, N. Kanwal, and R. S. Batth, “An Ontology of Digital Video Forensics: Classification, Research Gaps & Datasets,” Proceedings of 2019 International Conference on Computational Intelligence and Knowledge Economy, ICCIKE 2019, pp. 485–491, Dec. 2019. [CrossRef]
94. J. Zhang, Y. Su, and M. Zhang, “Exposing digital video forgery by ghost shadow artifact,” 1st ACM Workshop on Multimedia in Forensics - MiFor’09, Co-located with the 2009 ACM International Conference on Multimedia, MM’09, pp. 49–53, 2009. [CrossRef]
95. S. L. Das Gopu Darsan Shreyas Divya Devan, “Blind Detection Method for Video Inpainting Forgery,” Int J Comput Appl, vol. 60, no. 11, pp. 975–8887, 2012.
96. C. S. Lin and J. J. Tsay, “A passive approach for effective detection and localization of region-level video forgery with spatio-temporal coherence analysis,” Digit Investig, vol. 11, no. 2, pp. 120–140, 2014. [CrossRef]
97. S. Bai, H. Yao, R. Ni, and Y. Zhao, “Detection and Localization of Video Object Removal by Spatio-Temporal LBP Coherence Analysis,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11903 LNCS, pp. 244–254, 2019. [CrossRef]
98. M. Aloraini, M. Sharifzadeh, and D. Schonfeld, “Sequential and Patch Analyses for Object Removal Video Forgery Detection and Localization,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 31, no. 3, pp. 917–930, Mar. 2021. [CrossRef]
99. M. Aloraini, M. Sharifzadeh, C. Agarwal, and D. Schonfeld, “Statistical sequential analysis for object-based video forgery detection,” IS and T International Symposium on Electronic Imaging Science and Technology, vol. 2019, no. 5, p. 543, Jan. 2019. [CrossRef]
100. X. Jin, Z. He, J. Xu, Y. Wang, and Y. Su, “Object-Based Video Forgery Detection via Dual-Stream Networks,” in 2021 IEEE International Conference on Multimedia and Expo (ICME), 2021, pp. 1–6. [CrossRef]
101. A. V. Subramanyam and S. Emmanuel, “Video forgery detection using HOG features and compression properties,” 2012 IEEE 14th International Workshop on Multimedia Signal Processing, MMSP 2012 - Proceedings, pp. 89–94, 2012. [CrossRef]
102. R. Dixit and R. Naskar, “Review, analysis and parameterisation of techniques for copy–move forgery detection in digital images,” IET Image Process, vol. 11, no. 9, pp. 746–759, Sep. 2017. [CrossRef]
103. G. Kaur, N. Singh, and M. Kumar, Image forgery techniques: A review, no. May. Springer Netherlands, 2022. [CrossRef]
104. S. Teerakanok and T. Uehara, “Copy-Move Forgery Detection: A State-of-the-Art Technical Review and Analysis,” IEEE Access, vol. 7, pp. 40550–40568, 2019. [CrossRef]
105. Amerini, L. Ballan, R. Caldelli, A. Del Bimbo, and G. Serra, “A SIFT-based forensic method for copy-move attack detection and transformation recovery,” IEEE Transactions on Information Forensics and Security, vol. 6, no. 3 PART 2, pp. 1099–1110, Sep. 2011. [CrossRef]
106. “CoMoFoD — New database for copy-move forgery detection | IEEE Conference Publication | IEEE Xplore.” https://ieeexplore.ieee.org/document/6658316 (accessed Jun. 06, 2023).
107. Dong, W. Wang, and T. Tan, “CASIA image tampering detection evaluation database,” 2013 IEEE China Summit and International Conference on Signal and Information Processing, ChinaSIP 2013 - Proceedings, pp. 422–426, 2013. [CrossRef]
108. N. T. Pham, J.-W. Lee, G.-R. Kwon, and C.-S. Park, “Hybrid Image-Retrieval Method for Image-Splicing Validation”. [CrossRef]
109. B. Wen, Y. Zhu, R. Subramanian, T. T. Ng, X. Shen, and S. Winkler, “COVERAGE - A novel database for copy-move forgery detection,” Proceedings - International Conference on Image Processing, ICIP, vol. 2016-August, pp. 161–165, Aug. 2016. [CrossRef]
110. P. Korus and J. Huang, “Multi-Scale Fusion for Improved Localization of Malicious Tampering in Digital Images,” IEEE Transactions on Image Processing, vol. 25, no. 3, pp. 1312–1326, Mar. 2016. [CrossRef]
111. P. Korus and J. Huang, “Multi-Scale Analysis Strategies in PRNU-Based Tampering Localization,” IEEE Transactions on Information Forensics and Security, vol. 12, no. 4, pp. 809–824, Apr. 2017. [CrossRef]
112. D. T. Dang-Nguyen, C. Pasquini, V. Conotter, and G. Boato, “RAISE - A raw images dataset for digital image forensics,” Proceedings of the 6th ACM Multimedia Systems Conference, MMSys 2015, pp. 219–224, Mar. 2015. [CrossRef]
113. H. Guan et al., “MFC datasets: Large-scale benchmark datasets for media forensic challenge evaluation,” Proceedings - 2019 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2019, pp. 63–72, Feb. 2019. [CrossRef]
114. H. Guan et al., “NISTIR 8377 User Guide for NIST Media Forensic Challenge (MFC) Datasets”. [CrossRef]
115. G. Mahfoudi, B. Tajini, F. Retraint, F. Morain-Nicolier, J. L. Dugelay, and M. Pic, “Defacto: Image and face manipulation dataset,” European Signal Processing Conference, vol. 2019-September, Sep. 2019. [CrossRef]
116. M. Daisy, P. Buyssens, D. Tschumperle, and O. Lezoray, “A smarter exemplar-based inpainting algorithm using local and global heuristics for more geometric coherence,” 2014 IEEE International Conference on Image Processing, ICIP 2014, pp. 4622–4626, Jan. 2014. [CrossRef]
117. T. Y. Lin et al., “Microsoft COCO: Common Objects in Context,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8693 LNCS, no. PART 5, pp. 740–755, May 2014. [CrossRef]
118. A. Novozamsky, S. Saic, and B. Mahdian, “IMD2020: A Large-Scale Annotated Dataset Tailored for Detecting Manipulated Images,” Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision Workshops, WACVW 2020, pp. 71–80, Mar. 2020. [CrossRef]
119. Yu, Z. Lin, J. Yang, X. Shen, X. Lu, and T. S. Huang, “Generative Image Inpainting with Contextual Attention,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 5505–5514, Jan. 2018. [CrossRef]
120. R. Benenson, G. Research, and V. Ferrari, “From colouring-in to pointillism: Revisiting semantic segmentation supervision,” Oct. 2022, Accessed: Sep. 25, 2023. [Online]. Available: https://arxiv.org/abs/2210.14142v2.
121. A. Newson, A. Almansa, Y. Gousseau, and P. Pérez, “Non-Local Patch-Based Image Inpainting,” Image Processing On Line, vol. 7, pp. 373–385, Dec. 2017. [CrossRef]
122. The GIMP Development Team. GIMP. Retrieved from https://www.gimp.org.
123. W. Li, Z. Lin, K. Zhou, L. Qi, Y. Wang, and J. Jia, “MAT: Mask-Aware Transformer for Large Hole Image Inpainting,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2022-June, pp. 10748–10758, Jun. 2022. [CrossRef]
124. H. Wu, Y. Chen, and J. Zhou, “Rethinking Image Forgery Detection via Contrastive Learning and Unsupervised Clustering,” Aug. 2023, Accessed: Sep. 28, 2023. [Online]. Available: https://arxiv.org/abs/2308.09307v1.