Deepfake Image Detection using CNN (FaceForensics Dataset)

AI-generated face forgeries — deepfakes — now circulate in social feeds, messaging apps and even video calls, and the human eye is a poor detector of them. This is an academic study in media forensics: a convolutional neural network learns the subtle synthesis traces that manipulation methods leave behind — blending seams at the face boundary, inconsistent eye-region textures, and compression fingerprints. Students often stop at tutorials that classify MNIST digits; this project tackles a genuinely current problem on a real research dataset. The model is trained on FaceForensics++, the benchmark dataset from Rössler et al. (ICCV 2019) with 1,000 original videos manipulated by four methods, and evaluated with proper train/validation/test discipline. A forensic analyzer web demo makes the trained model tangible: feed it a face image and it reports a verdict with per-method probabilities and a heatmap of the regions the CNN weighed most.

1. FaceForensics++ videos are downloaded (after signing the dataset's terms of use); every 4th frame is sampled from the 720 training, 140 validation and 140 test videos.
2. A face detector extracts face boxes from each frame; each face is aligned by landmarks, cropped and resized to 299×299 — exactly the input the classifier expects.
3. XceptionNet is initialized with ImageNet pre-trained weights, its head replaced with a 5-way softmax (authentic + 4 manipulation methods), and fine-tuned with categorical cross-entropy.
4. Augmentation mirrors real conditions: random H.264 re-compression (c23/c40 levels), flips and small crops, so the model tolerates social-media-grade images.
5. Training is monitored on the validation split with early stopping; the best checkpoint is kept for evaluation.
6. The held-out test split is evaluated once: accuracy, per-class precision/recall/F1, ROC-AUC and a confusion matrix are generated for the report.
7. In the forensic demo, an uploaded face goes through detection → alignment → the saved model → verdict, with class probabilities and a Grad-CAM heatmap rendered alongside.

Tech stack:

• Python 3, TensorFlow/Keras
• XceptionNet (ImageNet-pretrained, fine-tuned)
• OpenCV (frame extraction, preprocessing)
• MTCNN-style face detection
• Jupyter Notebook (training & evaluation)
• Grad-CAM (explanation visuals)
• HTML · CSS · JavaScript (forensic demo)
• FaceForensics++ dataset

Dataset & model details

• Dataset: FaceForensics++ (Rössler et al., ICCV 2019, TUM/CISPA; source https://github.com/ondyari/FaceForensics — access requires signing the dataset's terms of use). 1,000 original video sequences from 977 YouTube videos with trackable, mostly frontal faces; 4,000 manipulated videos — 1,000 each from DeepFakes, Face2Face, FaceSwap and NeuralTextures. Three compression levels: raw (c0), high quality (c23), low quality (c40). The paper's automated benchmark contains approximately 1.8 million manipulated images. Official split used: 720 train / 140 validation / 140 test videos.
• Task: 5-class frame classification; input = 299×299×3 aligned face crop, output = probability distribution over {Authentic, DeepFakes, Face2Face, FaceSwap, NeuralTextures}. Video-level score is the mean of its frame scores.
• Model: XceptionNet (depthwise-separable convolution architecture, approximately 22.9 million parameters) with ImageNet pre-trained weights, fine-tuned on face crops; global average pooling → dense layer → 5-way softmax. Grad-CAM applied to the last convolutional block for explanation visuals.
• Metrics: Frame-level and video-level accuracy, per-class precision/recall/F1, ROC-AUC, confusion matrix. All numbers are measured on the held-out test split during the training run and documented in the report; none are claimed in advance. Design target: approximately 90%+ video-level accuracy on the c23 split.

• [XceptionNet frame classifier] A fine-tuned XceptionNet CNN takes a 299×299 face crop and outputs a 5-way decision — authentic, or one of the four FaceForensics++ manipulation methods — with per-class probabilities.
• [Face detection and alignment] An MTCNN-style face detector locates the face box and landmarks, crops and aligns the face so the classifier always sees the same normalized input it was trained on.
• [Grad-CAM explanation visuals] Activation heatmaps overlay the regions that drove the verdict (typically eyes, nose and blending boundary), giving the student a concrete explainability story for the viva.
• [Method attribution] Beyond real-vs-fake, the classifier distinguishes DeepFakes, Face2Face, FaceSwap and NeuralTextures — a harder task that demonstrates real depth.
• [Interactive forensic analyzer] A single-file web demo with upload support and synthetic sample faces, a four-stage pipeline walkthrough, verdict card, artifact checklist and heatmap view.
• [Full training notebook] Dataset download and frame extraction, preprocessing, ImageNet-pretrained fine-tuning, training loop, and evaluation — reproducible in one Jupyter notebook.
• [Honest evaluation report] Accuracy, per-class precision/recall/F1 and ROC-AUC measured on the official held-out test split; the report documents the achieved numbers, never pre-claimed ones.
• Optional customization: live-webcam detection mode and new-class fine-tuning can be developed on request — these are not in the base build.

• Complete training & evaluation Jupyter notebook (data loading to metrics)
• Trained XceptionNet classifier weights with preprocessing code
• Interactive forensic analyzer web demo (upload + synthetic samples)
• Grad-CAM explanation visuals and evaluation plots
• Project report PDF (background, dataset, architecture rationale, methodology, results)
• PPT presentation for final review
• Viva Q&A preparation document (CNNs, Xception, FaceForensics++, evaluation metrics)