What is a deepfake?

Deepfakes are created using deep learning techniques and Artificial Intelligence (AI) to manipulate existing media by swapping one person with another, creating the illusion that someone is present when they were never actually there. Deepfakes can also be entirely new and fabricated content, showing people doing or saying things that they never did or said.

Although they were initially popular mainly in the entertainment industry and on social media, deepfakes have now become a significant concern in the realm of cybersecurity. Fraudsters use deepfakes to manipulate public opinion and influence elections, deceive biometric facial recognition systems, and register fake accounts on various commercial resources and government systems.

What is essential to know is that deepfakes are difficult to recognize. In this article, we explore how deepfakes pose a threat to cybersecurity, present ways to detect them (elaborated by Group-IB Digital Risk Protection experts), and share strategies to mitigate the risks involved.

CYBERSECURITY X AI: Building capabilities to defend assets and defeat attackers
CYBERSECURITY X AI

Test your AI readiness and implement essential
upgrades to your cybersecurity

Download

Why are deepfakes dangerous?

Digital cloning poses a serious threat, especially when used in cyberattacks. AI-powered text-to-voice services can make scam calls sound incredibly natural, which is leading to more and more effective vishing (voice phishing) campaigns. The demand for deepfake apps like HeyGen and the discussions surrounding them point to a growing interest among threat actors in bypassing the security measures in such apps – as noted by Group-IB experts in the 2023/2024 Hi-Tech Crimes Report.

Gartner has drawn attention to the fact that AI-driven cyber and fraud attacks, including those involving deepfake technology, are on the rise: “Enterprises must prepare for malicious actors’ use of generative AI systems for cyber and fraud attacks, such as those that use deep fakes for social engineering of personnel, and ensure mitigating controls are put in place.”

Scams and phishing are among the most common challenges, and they are only expected to become even more dangerous as AI continues to evolve. Threat actors will continue to use realistic and convincing deepfakes to manipulate individuals and businesses, making it even more challenging to distinguish between genuine and fraudulent communications. These risks mean that organizations must adapt their cybersecurity strategies accordingly.

Typical deepfake attack schemes

Deepfakes are used in various malicious ways:

  • Blackmail: Victims’ faces are placed in compromising videos, for example in pornography.
  • Impersonation scams: Executives and employees have their identities forged and used in phishing attacks with a view to stealing money and sensitive information.
  • Brand damage: False videos or audio clips are often created to imitate brands and mislead customers into visiting phishing or scam websites, or to spread false information. These attacks can damage a company’s reputation, affecting stock prices and consumer trust.
  • Cyberbullying: Deepfake videos or images are used to harass, defame, or humiliate individuals online, often with devastating effects on their mental health and reputation.
  • Fraud schemes: Faces of public figures (such as Elon Musk) are used to convince victims into using new crypto exchange platforms as part of crypto scams. Cybercriminals can also steal facial recognition data to create deepfakes, replacing their own faces with those of the victims, to gain unauthorized access to banking apps. The first ever Trojan with such capabilities called GoldPickaxe.iOS was recently uncovered by Group-IB.

How to spot a deepfake

Identifying deepfake videos can be tricky. Signs to watch out for include:

  • Obvious borders: Clear boundaries where images have been overlaid
  • Head movement errors: Graphical issues when the person turns their head (deepfakes are most effective for calm, forward-facing expressions)
  • Inconsistent color, lighting, and/or image quality
  • Facial expression anomalies: Unnatural blinking or unsynchronized lip movements
  • Eye rendering issues: Abnormalities seen in pupils or eye movements
  • Disappearance of moles and scars: Natural facial features may vanish
  • Temporary overlapping: Elements from the original image may appear over the deepfake
  • Mismatch in physical features: Differences in head shape, hairstyle, body shape, and voice
  • Visual artifacts: Flickering, pixelation, or blurring
  • Quality discrepancies: The overlaid fragment might even look better than the original video due to various enhancers

How to identify a deepfake voice?

It is possible to recognize a deepfake audio by listening out for the following:

  • Digital artifacts: Robotic sounds or digital noise
  • Unnatural speech tempo: Odd slowdowns or speed-ups
  • Inconsistent tone: The expression or articulation may not match the person’s typical speech or the context (e.g., overly emotional during a calm segment or vice versa)
  • Monotone delivery: A lack of variation in intonation, which makes speech sound unnatural
  • Uncharacteristic vocabulary: Use of words or phrases that the individual doesn’t normally use

Unlike fake videos, where various artifacts created by neural networks can be seen with the naked eye (such the “uncanny valley” effect), detecting fake audio by ear is much harder. Scammers can easily mask the digital origins of the audio with slight alterations and noise. To make detection even harder, such deepfakes may be based on well-established and highly developed sound analysis and modulation technologies that are widely applied in the music industry and smart voice assistants like Siri. Group-IB Digital Risk Protection experts therefore recommend calling the person back to verify their identity if you have any doubts.

Important note: In many fraud schemes, instead of using deepfakes, cybercriminals often rely on real fragments of archival videos featuring famous people. For instance, celebrities who have taken part in advertising or marketing campaigns are at risk of having their videos edited and re-dubbed. Such manipulated videos, which do not involve complex face-swapping techniques, can be just as misleading as deepfakes.

How to defend against deepfakes with Group-IB

To protect against deepfake threats:

Instead of fearing emerging technologies, you can harness the power of AI to your advantage by signing up to Group-IB Digital Risk Protection, which uses advanced AI algorithms to detect unauthorized use of your logos, trademarks, content, and design layouts across your digital landscape and offers robust protection against scams, phishing, and VIP impersonation attacks.