Deepfake creation methods have evolved rapidly in recent years. Modern AI models can generate realistic faces, objects and entire scenes, leading to a proliferation of manipulated photos on the internet with an astonishingly close resemblance to the originals. The ability of generative AI to create lifelike images opens up new possibilities in areas such as creative applications. But this bears with it an increased risk of misinformation. Manipulated content can distort evidence and undermine trust. In the Realorrender project, researchers at Fraunhofer IOSB are collaborating with the German Federal Office for Information Security (BSI) to show how innovative AI methods can be used for reliable and robust distinction of deepfakes from real images. The project is funded by the BSI.

Robust mechanisms for detecting deepfakes are of critical importance for mitigating risks. The research team at Fraunhofer IOSB is taking a first-time hybrid approach, combining a traditional deep learning classification method with a method that checks how well an image can be reconstructed by a generative model. “First, we use an AI image generator to reconstruct the image. Then an AI model takes over the classification and carries out a hybrid calculation of the reconstruction error. We finally obtain an estimate displaying recognition accuracy as a percentage. The overall detection rate ranges between 85 and 91 percent, although it can certainly be higher in individual cases,” explains Andreas Specker, senior scientist in the Video-Assisted Security and Assistance Systems research group. The more successful the reconstruction, the more likely it is that the image in question is also an AI-generated image, i.e., a deepfake. Detection accuracy is significantly improved by combining both approaches.

Combining detection and explainability

But simply exposing counterfeits is not enough. It is just as important to understand why the method detects a deepfake. Explaining decisions along with detection is therefore a key goal for the researchers. “We use explainable AI (XAI) methods for this, which allows us to highlight the image areas and structures that contribute to the model's decision. It can thus be clearly understood why an image is classified as real or AI-generated. For example, distinctive textures or characteristic frequency patterns can hint at the synthetic origin of an image. XAI helps to show which features the model uses in its decision,” says Nadia Burkart, Head of the Applied Explainable AI group at Fraunhofer IOSB. The researchers use XAI methods that can be classified into heatmap-based and segment-based approaches based on their explanation output formats. Heatmap-based approaches visualize which areas of the image contributed most to the model's decision and the degree to which these factors influenced the evaluation. In contrast, segment-based methods analyze contiguous image segments and show which semantic regions contributed to the model's decision.

Effective technical protection against forged images

In tests on a large image dataset, the researchers were able to demonstrate that this hybrid, explanation-oriented approach outperforms current methods. This yields a new generation of intelligent systems that can learn and are transparent and trustworthy. Combining these methods improves the robustness of deepfake detection, even with regard to increasingly powerful image generators.

The research findings are being incorporated in a demonstrator that will initially assist the BSI in detecting deepfakes. The demonstrator is easy to use: After uploading the desired image, the detection result is displayed, followed by an explanation. A text summary also provides an overview of the results.

The research by Fraunhofer IOSB and the BSI is documented in two project reports that have already been published. This work not only benefits security and law enforcement agencies but also social media platform operators, media companies and photo agencies.