Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data

*,†,‡ ORCID logo , †,‡ , †,‡
Current address: North Carolina Agricultural and Technical State University, 1601 E Market St, Greensboro, North Carolina 27411, USA
These authors contributed equally to this work.
Version 1 : Received: 16 February 2023 / Approved: 17 February 2023 / Online: 17 February 2023 (06:51:37 CET)

How to cite: Johnson, D.; Yuan, X.; Roy, K. Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints 2023, 2023020299. https://doi.org/10.20944/preprints202302.0299.v1 Johnson, D.; Yuan, X.; Roy, K. Using Ensemble Convolutional Neural Network to Detect Deepfakes Using Periocular Data. Preprints 2023, 2023020299. https://doi.org/10.20944/preprints202302.0299.v1

Abstract

Deepfakes are manipulated or altered images, or video, that are created using deep learning models with high levels of photorealism. The two popular methods of producing a deepfake are based on either convolutional neural networks (CNN), or autoencoders. Deepfakes created using CNN comparatively show higher qualities of realism, yet oftentimes leave artifacts and distortions in the generated media that can be detected using machine learning and deep learning algorithms. In recent years, there has been an influx of periocular image and video data because of the increase usage of face masks. By wearing masks, much of what is used for facial recognition is hidden, leaving only the periocular region visible to an observer. This loss of vital information leads to easier misidentification of media, allowing deepfakes to less likely be identified as fake. In this work, feature extraction methods, such as Scale-Invariant Feature Transform (SIFT), Histogram of Oriented Gradients (HOG), and CNN, are used to train an ensemble deep learning model to detect deepfakes in videos on a frame-by-frame level based on the periocular region. Our proposed model is able to distinguish original and manipulated images with accuracies around 98.9 percent, which is an improvement to previous works by combining SIFT and HOG for deepfake detection in convolutional neural networks.

Keywords

deepfake detection; CNN; deep neural network; computer vision; scale invariant feature transform; histogram of oriented gradients

Subject

Computer Science and Mathematics, Computer Science

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our Diversity statement.

Leave a public comment
Send a private comment to the author(s)
* All users must log in before leaving a comment
Views 0
Downloads 0
Comments 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.